vitepress-template

Appearance

Configuration Management ​

**Referenced Files in This Document** - [[application-dev.yaml]](file/bi-basic/configs/application-dev.yaml) - [[application-prod.yaml]](file/bi-basic/configs/application-prod.yaml) - [[conf.pb.go (bi-common)]](file/bi-common/conf/common.pb.go) - [[conf.pb.go (bi-basic)]](file/bi-basic/internal/conf/conf.pb.go) - [[config.go (nacos)]](file/bi-common/registry/nacos/config.go) - [[nacos-default.yaml]](file/bi-common/registry/nacos/nacos-default.yaml) - [[loader.go (nacos)]](file/bi-common/registry/nacos/loader.go) - [[client.go (nacos)]](file/bi-common/registry/nacos/client.go) - [[env.go (nacos)]](file/bi-common/registry/nacos/env.go) - [[config.go (bi-cron)]](file/bi-cron/internal/config/config.go) - [[architect.md (bi-notify)]](file/bi-notify/architect.md) - [[spec-domain.md]](file/docs/spec-domain.md) - [[SKILL.md (bi-nacos-config)]](file/.agent/skills/bi-nacos-config/skill.md)

Table of Contents ​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion

Introduction ​

This document explains the configuration management system used across bi-server and related services. It covers environment-specific configuration files, YAML structure, parameter definitions, configuration loading hierarchy, override mechanisms, and environment variable integration. It also documents service-specific configurations, database connections, external service endpoints, validation and defaults, secrets management, hot-reloading, and deployment-specific overrides. Finally, it provides troubleshooting guidance for common configuration issues.

Project Structure ​

The configuration system is organized around:

  • Environment-specific application YAML files under each service’s configs directory
  • A shared bi-common configuration model defined via Protocol Buffers
  • A Nacos-based configuration source integrated with Kratos config
  • Optional environment variable overrides for dynamic tuning

Diagram sources

Section sources

Core Components ​

  • Nacos configuration model and defaults
    • Root configuration structure with nested domains for server, client, config, auth, and advanced settings
    • Embedded default YAML with sensible defaults and optional auto-derived DataId
  • Kratos config source for Nacos
    • Loads multiple DataIds and supports watch/OnChange callbacks for hot-reload
  • Environment variable integration
    • Extensive environment variable constants and a LoadFromEnv method to override defaults
  • Service configuration model
    • Bootstrap and typed sections for server, data, log, snowflake, and auth
  • Example service configuration
    • bi-basic application-{env}.yaml files define Nacos connection and DataIds

Section sources

Architecture Overview ​

The configuration pipeline merges multiple sources with strict precedence and supports hot-reloading.

Diagram sources

Detailed Component Analysis ​

Nacos Configuration Model and Defaults ​

  • Structure
    • RootConfig wraps Config with nacos namespace
    • Config groups: Server, Client, Config, Auth, Advanced
  • Defaults
    • Embedded default YAML supplies sane defaults; missing DataId is auto-derived from calling module
  • Validation
    • Validates host, port, and timeout before client creation
  • Merge and Clone
    • Merge propagates non-empty fields; Clone duplicates with safe copying of slices
  • Getter helpers
    • Convenience getters for common fields

Diagram sources

Section sources

Environment Variable Integration ​

  • Overridable fields
    • Server: scheme, addr, port, grpc_port, context_path
    • Client: namespace_id, group, app_name, timeout_ms, beat_interval, log_level, log_dir, cache_dir
    • Config: data_id
    • Auth: username, password, access_key, secret_key
  • Legacy compatibility
    • Backward-compatible environment variables for port, namespace_id, group, data_id
  • Behavior
    • LoadFromEnv reads environment variables and merges into Config; later steps can override

Diagram sources

Section sources

Kratos Nacos Config Source ​

  • Loading
    • Resolves multiple DataIds and returns Kratos KeyValue entries with format derived from file extension
  • Watching
    • Registers ListenConfig per DataId and emits Next() updates
    • Stop/Close cancels listeners and closes watcher
  • Usage
    • NewConfigSourceWithOptions builds a Kratos Source backed by Nacos client

Diagram sources

Section sources

Service-Specific Configuration Model ​

  • Bootstrap
    • Contains name, version, and typed sections: server, data, log, snowflake, auth
  • Typed sections
    • Server: HTTP and gRPC network, address, and timeouts
    • Data: datasource, redis, kafka
    • Log: level, format, output, file, Aliyun, skip paths
    • Snowflake: worker_id, data_center_id
    • Auth: JWT and white list

Diagram sources

Section sources

Environment-Specific Application Configurations ​

  • bi-basic application-{env}.yaml
    • Define Nacos server endpoint, client namespace/group, and list of DataIds
    • Production example includes explicit auth credentials and grpc_port

Section sources

Example: bi-cron gRPC Endpoint Configuration ​

  • AppConfig loads remote gRPC service endpoints from Nacos-backed YAML
  • Provides typed service configs with service_name, group, and timeout parsing

Section sources

Configuration Loading Hierarchy and Override Mechanisms ​

  • Precedence (highest to lowest)
    • Environment variables (LoadFromEnv)
    • Function options and programmatic overrides
    • Local application-{env}.yaml (Nacos connection info and DataIds)
    • Remote Nacos DataIds (merged in order)
  • Automatic defaults
    • Embedded default YAML plus auto-derived DataId from module name
  • Validation
    • Host/port/timeout validated before client creation

Section sources

Parameter Definitions and YAML Structure ​

  • Nacos configuration (nacos:)
    • server: scheme, addr, port, grpc_port, context_path
    • client: namespace_id, group, app_name, timeout_ms, beat_interval, log_level, log_dir, cache_dir
    • config: data_id or data_ids
    • auth: username, password, access_key, secret_key
    • advanced: flags and thread count
  • Service configuration (Bootstrap)
    • server.http/grpc: network, addr, timeout
    • data.datasource: driver, host, port, database, username, password, charset, optimize_for_starrocks, pool, log, streamload
    • data.redis: mode, addrs, username, password, db, protocol, max_retries, master_name, route_by_latency, route_randomly, pool, timeout
    • data.kafka: producer, consumer, consumers
    • log: level, format, output, file, aliyun, skip_paths
    • snowflake: worker_id, data_center_id
    • auth: jwt, white_list

Section sources

Database Connections and External Service Endpoints ​

  • Database
    • Data_Datasource defines driver, host, port, database, credentials, charset, StarRocks optimization flags, pool, log, and streamload
  • External endpoints
    • bi-cron demonstrates loading remote gRPC endpoints from Nacos YAML via AppConfig

Section sources

Configuration Validation and Default Value Handling ​

  • Default values
    • Embedded YAML default values; fallback to hardcoded defaults if embedded YAML fails
  • Validation
    • Validate host, port, and timeout prior to client creation
  • Parsing and defaults
    • bi-cron parses timeout strings with fallback to a default duration

Section sources

Secure Parameter Management ​

  • Secrets storage
    • Use Nacos encrypted values or inject via Kubernetes Secrets for sensitive fields
  • Authentication
    • Nacos auth credentials supported; production example shows explicit credentials in application-{env}.yaml
  • Environment isolation
    • Separate namespaces/groups per environment

Section sources

Configuration Hot-Reloading ​

  • Watch mechanism
    • ConfigSource.ListenConfig registers callbacks per DataId
    • Watcher.Next emits updated KeyValue entries on change
    • Stop/Close cancels listeners
  • Practical guidance
    • Register watchers for hot-reloadable settings (e.g., log level)
    • Merge updates into active configuration safely

Section sources

Deployment-Specific Overrides ​

  • Environment files
    • application-dev.yaml, application-prod.yaml define Nacos endpoints and DataIds
  • Environment variables
    • Override any Nacos setting via environment variables
  • Auto-derived DataId
    • If no DataId configured, derive from module name automatically

Section sources

Dependency Analysis ​

Diagram sources

Section sources

Performance Considerations ​

  • Minimize DataIds to reduce fetch overhead
  • Use watchers judiciously; avoid excessive listeners
  • Prefer environment variables for hot-swappable settings to avoid frequent reloads
  • Keep default timeouts reasonable to prevent long blocking during startup

Troubleshooting Guide ​

  • Cannot connect to Nacos
    • Verify server.addr, server.port, server.grpc_port, and context_path in application-{env}.yaml
    • Confirm environment variables if overridden
  • Invalid host or port
    • Validation occurs before client creation; fix values in YAML or environment variables
  • Missing DataId
    • Ensure data_id or data_ids is set; if omitted, auto-derived from module name
  • Authentication failures
    • Set NACOS_AUTH_USERNAME/PASSWORD or configure in application-{env}.yaml
  • No hot-reload updates
    • Ensure Watch() is called and listeners registered; confirm group matches
  • Conflicting configuration sources
    • Environment variables override YAML; confirm precedence and intended overrides

Section sources

Conclusion ​

The bi-server configuration system combines environment-specific YAML files, Nacos-based remote configuration, Kratos-native merging, and environment variable overrides. It provides robust defaults, strong validation, and hot-reload capabilities while supporting secure secret management and environment isolation. Following the documented hierarchy and best practices ensures reliable deployments across environments.