Configuration and Deployment
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendices
Introduction
This document provides comprehensive configuration and deployment guidance for the bi-analysis service. It covers environment-specific configuration files, parameter definitions, Docker containerization, Kubernetes deployment manifests, and service orchestration patterns. It also explains configuration management via Nacos, environment variable overrides, runtime configuration updates, deployment automation, health checks, monitoring, scaling strategies, resource requirements, performance tuning, backup procedures, disaster recovery, and maintenance windows for production deployments.
Project Structure
The bi-analysis service follows a layered structure:
- Command entrypoint initializes Kratos application, loads environment-specific Nacos configuration, sets up logging and service registry, and starts servers.
- Internal configuration defines the Bootstrap schema used to scan Nacos-provided YAML configurations.
- Nacos integration provides a Kratos config source and watcher for dynamic configuration updates.
- Kubernetes manifests define container images, environment variables, ports, and resource requests/limits.
- Dockerfile builds a minimal Alpine-based image with timezone and binary copy.
Diagram sources
- [main.go]
- [conf.proto]
- [common.proto]
- [config.go]
- [loader.go]
- [client.go]
- [discovery.go]
- [deployment.yaml]
- [service.yaml]
- [Dockerfile]
Section sources
- [main.go]
- [conf.proto]
- [common.proto]
- [config.go]
- [loader.go]
- [client.go]
- [discovery.go]
- [deployment.yaml]
- [service.yaml]
- [Dockerfile]
Core Components
- Environment-specific Nacos configuration files:
- Local development: application-local.yaml
- Development cluster: application-dev.yaml
- Test cluster: application-test.yaml
- Production: application-prod.yaml
- Bootstrap configuration schema:
- Bootstrap includes name, version, server, data, log, snowflake, and auth.
- Nacos configuration source:
- Loads multiple DataIDs per environment and supports watch for runtime updates.
- Kubernetes deployment:
- Defines image pull secrets, environment variables, ports, and resource requests/limits.
- Docker image:
- Multi-stage build with Go proxy mirrors, timezone setup, and minimal runtime image.
Section sources
- [application-local.yaml]
- [application-dev.yaml]
- [application-test.yaml]
- [application-prod.yaml]
- [conf.proto]
- [common.proto]
- [loader.go]
- [deployment.yaml]
- [Dockerfile]
Architecture Overview
The bi-analysis service integrates with Nacos for centralized configuration and service discovery. On startup, the service loads environment-specific configuration files and watches for changes. Kubernetes manages container lifecycle, exposing HTTP and gRPC ports and applying resource constraints.
Diagram sources
Detailed Component Analysis
Environment-Specific Configuration Files
- Local environment:
- Nacos server address/port/grpc_port/context_path
- Authentication credentials
- Client settings: namespace, group, timeout, beat interval, log/cache directories
- DataIDs: bi-common.yaml, bi-analysis.yaml
- Advanced toggles for cache loading, snapshot usage, and update thread count
- Dev/Test environments:
- Nacos server address and context path
- Client settings and DataIDs differ per environment
- Prod environment:
- Nacos server address and context path
- Client settings and DataIDs differ per environment
Parameter definitions:
- Nacos server: scheme, address, HTTP port, gRPC port, context path
- Client: namespace ID, group, app name, timeout, beat interval, log level/dir, cache dir
- Config: single or multiple DataIDs
- Auth: username/password and optional cloud keys
- Advanced: cache loading behavior, snapshot usage, update thread number
Section sources
- [application-local.yaml]
- [application-dev.yaml]
- [application-test.yaml]
- [application-prod.yaml]
- [config.go]
Bootstrap Configuration Schema
The Bootstrap message defines top-level service configuration:
- name: service name
- version: service version
- server: HTTP and gRPC transport settings
- data: datasource, Redis, Kafka configuration
- log: logging level/format/output and optional Aliyun logstore
- snowflake: worker and data center IDs
- auth: JWT and white-listed paths
These fields map to the common configuration schema and are populated from Nacos-provided YAML.
Section sources
Nacos Integration and Runtime Updates
- ConfigSource implements Kratos config.Source and supports loading multiple DataIDs and watching for changes.
- Watcher aggregates changes per DataID and exposes Next() to receive updated content.
- Client provides convenience methods for registering/deregistering services, discovering instances, subscribing/unsubscribing, and listing services.
Diagram sources
Section sources
Docker Containerization
- Multi-stage build:
- Builder stage installs Go proxy mirrors, configures git credentials, downloads modules, and builds the server binary.
- Final stage copies the binary and configs into a minimal Alpine image, sets timezone, exposes ports, and runs the server.
- Ports exposed:
- HTTP: 10002
- gRPC: 20002
- Build-time arguments:
- GIT_USERNAME, GIT_TOKEN, PROJECT_NAME, APP_NAME for private module access during build.
Diagram sources
Section sources
Kubernetes Deployment and Service Orchestration
- Deployment:
- Uses imagePullSecrets for registry authentication.
- Sets DEPLOY_ENV via ConfigMap and passes it to the container command.
- Injects NACOS_AUTH_USERNAME/PASSWORD from Kubernetes Secrets.
- Exposes HTTP and gRPC ports and applies CPU/memory limits/requests.
- Service:
- NodePort service for development/testing with explicit nodePort allocation.
- Selects pods by label app=${name}.
Diagram sources
Section sources
Configuration Management Through Nacos
- Environment selection:
- The service reads configs/application-${env}.yaml based on the -env flag.
- DataID resolution:
- Multiple DataIDs are supported per environment; loader fetches each and formats the key/value pairs for Kratos.
- Runtime updates:
- Watcher listens to changes for all configured DataIDs and emits updates via Next().
- Defaults and validation:
- Default Nacos configuration includes scheme, address, ports, group, timeouts, log/cache directories, and advanced toggles.
- Validation ensures server address and port are valid.
Section sources
Environment Variable Overrides and Runtime Configuration Updates
- Environment variables:
- DEPLOY_ENV selects the environment-specific YAML.
- NACOS_AUTH_* injected from Kubernetes Secrets override Nacos credentials.
- Runtime updates:
- Use the Watcher to receive configuration changes and re-scan into Bootstrap.
- Ensure downstream components handle configuration reload safely.
Section sources
Deployment Automation Scripts
- Makefile targets:
- init: installs required protobuf and Kratos tooling.
- api-doc: generates OpenAPI documentation from proto files.
- api-sync: syncs OpenAPI to Apifox.
- api: generates Go, HTTP, gRPC, and OpenAPI code from proto files.
- config: generates internal proto bindings.
- build: builds the binary with version metadata.
- generate: runs go generate and go mod tidy.
- all: runs api, api-doc, api-sync, config, generate.
- help: prints available targets.
Section sources
Dependency Analysis
The bi-analysis service depends on bi-common for shared configuration schemas and Nacos integration. The Kratos configuration source bridges Nacos and the application’s Bootstrap configuration.
Diagram sources
Section sources
Performance Considerations
- Resource sizing:
- Current deployment requests low CPU/memory; adjust based on load testing and latency targets.
- Network and timeouts:
- Tune Nacos client TimeoutMs and BeatInterval according to cluster latency.
- Logging and observability:
- Set log level and output appropriately for environments; consider structured logs and rotation.
- Database and caching:
- Configure datasource pool sizes and Redis connection pools aligned with workload.
- Kafka throughput:
- Adjust producer/consumer batch sizes and worker counts for event volume.
[No sources needed since this section provides general guidance]
Troubleshooting Guide
- Nacos connectivity:
- Verify server address, port, and context path in environment-specific YAML.
- Confirm credentials via NACOS_AUTH_* environment variables.
- Configuration loading:
- Ensure DataIDs exist in Nacos group/namespace and content is valid YAML.
- Check Watcher registration and logs for listener errors.
- Port conflicts:
- Confirm HTTP (10002) and gRPC (20002) ports are free and mapped in Service/Deployment.
- Health checks:
- Implement readiness/liveness probes on the container and expose metrics/metrics endpoints.
- Logs:
- Review container logs and Nacos client logs (log_dir) for initialization and runtime errors.
Section sources
- [application-dev.yaml]
- [application-test.yaml]
- [application-prod.yaml]
- [application-local.yaml]
- [loader.go]
- [client.go]
- [deployment.yaml]
- [service.yaml]
Conclusion
The bi-analysis service leverages Nacos for centralized configuration and service discovery, Kratos for application lifecycle, and Kubernetes for orchestration. By aligning environment-specific YAMLs, Docker images, and Kubernetes manifests, teams can achieve reliable deployments with dynamic configuration updates, robust monitoring, and scalable resource management.
[No sources needed since this section summarizes without analyzing specific files]
Appendices
Scaling Strategies
- Horizontal scaling:
- Increase replicas in Deployment; ensure stateless design and shared configuration via Nacos.
- Vertical scaling:
- Adjust CPU/memory requests/limits based on measured utilization.
- Auto-scaling:
- Use HPA targeting CPU/utilization or custom metrics.
[No sources needed since this section provides general guidance]
Backup Procedures and Disaster Recovery
- Configuration backups:
- Export Nacos DataIDs regularly and store in version control or secure storage.
- Rollback strategy:
- Tag container images and roll back to previous versions if configuration changes cause issues.
- DR sites:
- Deploy standby clusters in separate regions with synchronized Nacos namespaces/groups.
[No sources needed since this section provides general guidance]
Maintenance Windows
- Scheduled downtime:
- Perform rolling updates during off-peak hours; set preStop hooks if needed.
- Zero-downtime updates:
- Use blue/green or canary deployments with proper health checks and rollback plans.
[No sources needed since this section provides general guidance]