vitepress-template

Appearance

Architecture Overview ​

**Referenced Files in This Document** - [[main.go]](file/bi-analysis/cmd/bi-analysis/main.go) - [[application-dev.yaml]](file/bi-analysis/configs/application-dev.yaml) - [[client.go]](file/bi-common/registry/nacos/client.go) - [[registry.go]](file/bi-common/registry/nacos/registry.go) - [[config.go]](file/bi-common/registry/nacos/config.go) - [[consumer.go]](file/bi-common/mq/kafkax/consumer.go) - [[producer.go]](file/bi-common/mq/kafkax/producer.go) - [[config.go]](file/bi-common/mq/kafkax/config.go) - [[client.go]](file/bi-common/database/gormx/client.go) - [[config.go]](file/bi-common/database/gormx/config.go) - [[client.go]](file/bi-common/database/starrocks/streamload/client.go) - [[logger.go]](file/bi-common/observability/logger/logger.go) - [[middleware.go]](file/bi-common/auth/middleware.go) - [[jwt.go]](file/bi-common/auth/jwt.go) - [[services.yaml]](file/bi-intra/apisix/services.yaml)

Table of Contents ​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion

Introduction ​

This document presents the architecture of the BI Analysis Platform, a microservices-based system designed for scalable business intelligence analytics. It covers:

  • Microservices boundaries and communication patterns
  • Layered architecture separating UI, AI analysis, business services, and infrastructure
  • Event-driven data ingestion via Apache Kafka
  • Service discovery with Nacos
  • API gateway powered by APISIX
  • External integrations and deployment topology
  • Scalability, fault tolerance, and operational practices

Project Structure ​

The platform is organized around multiple Go microservices under the bi-* modules, unified by shared libraries in bi-common. The key structural elements are:

  • Service entry points: per-service main.go bootstraps Kratos applications, load configuration from Nacos, initialize logging, and register with Nacos.
  • Shared infrastructure: Nacos for service discovery and configuration, Kafka for event streaming, APISIX for API routing and security, and StarRocks for analytical storage.
  • UI frontends: Next.js-based web apps for client and admin experiences.

Diagram sources

Section sources

Core Components ​

  • Kratos Application Bootstrap: Each service initializes Kratos servers, loads configuration from Nacos, sets up logging, registers with Nacos, and wires dependencies via a wiring function.
  • Nacos Integration: Provides centralized configuration and service registration/discovery across environments.
  • Kafka Streaming: Producer/consumer abstractions for event-driven ingestion and processing.
  • Database Clients: GORM-based MySQL/StarRocks client with connection pooling and TLS support.
  • Stream Load: StarRocks Stream Load client for high-throughput data ingestion.
  • Observability: Structured logging with file rotation and optional Aliyun log forwarding.
  • Authentication: JWT middleware with optional RBAC provider and permission caching.
  • API Gateway: APISIX routes traffic to services, enforces security policies, and manages cross-cutting concerns.

Section sources

Architecture Overview ​

The system follows a layered architecture:

  • UI Layer: Next.js applications serving client/admin experiences.
  • Business Services: Per-domain microservices exposing HTTP/gRPC APIs.
  • Infrastructure Layer: Nacos, Kafka, APISIX, StarRocks, Redis.

Communication patterns:

  • API Gateway (APISIX) routes requests to services by route prefix and host.
  • Services discover each other via Nacos and exchange events through Kafka topics.
  • Data persistence leverages StarRocks for analytics workloads with GORM for transactional data.

Diagram sources

Detailed Component Analysis ​

Service Discovery with Nacos ​

  • Centralized configuration and service registry.
  • Services load bootstrap configuration from Nacos and register endpoints.
  • Registry supports health checks, clustering, and metadata propagation.

Diagram sources

Section sources

Event-Driven Architecture with Apache Kafka ​

  • Producers publish domain events to Kafka topics.
  • Consumers subscribe to topics and process messages with configurable batching, retries, and offsets.
  • Supports single-partition and consumer-group modes for different scaling needs.

Diagram sources

Section sources

API Gateway with APISIX ​

  • Centralized routing, authentication, rate limiting, and WAF protection.
  • Routes map to service names and hosts; supports whitelisted paths and JWT verification.

Diagram sources

Section sources

Data Persistence and Stream Loading ​

  • GORM client for relational/transactional data with connection pooling and TLS.
  • StarRocks Stream Load client for bulk ingestion with retry and redirect handling.

Diagram sources

Section sources

Authentication and Authorization Middleware ​

  • JWT middleware validates tokens and optionally integrates RBAC providers for granular permissions.
  • Supports separate providers for client and management endpoints.

Diagram sources

Section sources

Dependency Analysis ​

  • Coupling: Services depend on bi-common for shared infrastructure (Nacos, Kafka, DB clients, auth, logging).
  • Cohesion: Each bi-* module encapsulates domain-specific business logic and data access.
  • External Dependencies: APISIX for routing, Nacos for discovery/configuration, Kafka for messaging, StarRocks for analytics, Redis for caching.

Diagram sources

Section sources

Performance Considerations ​

  • Kafka
    • Use consumer groups for horizontal scaling and partitioned topics for throughput.
    • Tune batch sizes/timeouts and compression to balance latency and bandwidth.
  • Database
    • Apply StarRocks connection pool optimizations when applicable.
    • Monitor slow queries and adjust thresholds.
  • Logging
    • Enable file rotation and consider structured JSON logs for observability.
  • Gateway
    • Configure rate limits and WAF rules to protect upstream services.

[No sources needed since this section provides general guidance]

Troubleshooting Guide ​

  • Service Discovery
    • Verify Nacos connectivity and that services register with correct endpoints and metadata.
  • Kafka
    • Confirm producer/consumer connectivity, topic existence, and partition distribution.
    • Monitor lag and handle dead-letter queues for failed messages.
  • Database
    • Check connection pool stats and TLS settings; validate DSN generation.
  • Logging
    • Ensure log outputs are configured for stdout/file and cleanup resources on shutdown.
  • Authentication
    • Validate JWT secret and token expiration; confirm RBAC provider availability.

Section sources

Conclusion ​

The BI Analysis Platform employs a robust microservices architecture with clear service boundaries, event-driven ingestion, and centralized infrastructure management. APISIX provides secure, scalable routing; Nacos ensures dynamic configuration and discovery; Kafka enables decoupled, high-throughput data processing; and StarRocks powers analytics workloads. The layered design promotes maintainability, while shared libraries reduce duplication and improve consistency across services.