Infrastructure and Deployment Architecture
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
Introduction
This document describes the infrastructure and deployment architecture for the BI platform, focusing on containerized deployment using Docker and Kubernetes orchestration, APISIX API gateway configuration for traffic management and security, Nacos integration for service discovery and configuration management, Helm charts for infrastructure components, deployment patterns, infrastructure requirements, scaling strategies, and monitoring/observability setup.
Project Structure
The infrastructure repository is organized around three primary areas:
- APISIX routing and traffic management configuration
- Helm charts for APISIX, Nacos, Kafka, and Grafana
- Supporting scripts for publishing services and managing APISIX routes
Diagram sources
- [bi-infra README]
- [APISIX Services Config]
- [APISIX Manager Script]
- [APISIX Gateway LB Test]
- [APISIX Helm Values]
- [Nacos Helm Values]
- [Kafka Helm Values]
- [Grafana Helm Values]
- [Nacos Client]
- [Nacos Config]
- [Nacos Discovery]
- [Nacos Loader]
Section sources
Core Components
- APISIX API Gateway: Central traffic management, load balancing, security policies, and service discovery integration.
- Nacos: Service discovery and configuration management for microservices.
- Kafka: Event streaming and messaging backbone.
- Grafana: Visualization and observability dashboards.
- Deployment Scripts: Automated Docker builds, pushes, and Kubernetes image updates.
Section sources
- [APISIX Services Config]
- [APISIX Manager Script]
- [Nacos Helm Values]
- [Kafka Helm Values]
- [Grafana Helm Values]
- [Publish Services Script]
Architecture Overview
The system uses APISIX as the ingress and API gateway, integrating with Nacos for service discovery and configuration. Kafka provides asynchronous messaging, while Grafana visualizes metrics. Helm charts standardize deployments across namespaces.
Diagram sources
- [APISIX Services Config]
- [APISIX Manager Script]
- [Nacos Helm Values]
- [Kafka Helm Values]
- [Grafana Helm Values]
Detailed Component Analysis
APISIX API Gateway Configuration
APISIX serves as the central traffic plane with integrated security, routing, and service discovery. It supports:
- Route creation per service with dynamic upstream resolution via Nacos
- Security policies: IP whitelist, API key, rate limiting, WAF (uri-blocker), JWT authentication
- Path rewriting and host-based routing
- Health checks and retries
Diagram sources
Key configuration highlights:
- Default security policies (IP whitelist, rate limit, WAF rules) defined centrally
- JWT authentication with configurable algorithm and environment-based secret loading
- Upstream discovery via Nacos with round-robin load balancing and retries
- Host-based routing and path rewriting for frontend and backend services
Operational controls:
- Batch route creation/deletion via manager script
- Environment-specific URI prefixes and namespace handling
- Support for both fixed upstream URLs (frontend/static) and Nacos-discovered services
Section sources
Nacos Integration for Service Discovery and Configuration
Nacos provides:
- Service registration and discovery for APISIX and microservices
- Centralized configuration management via DataIDs
- Authentication and secure communication
Diagram sources
Implementation details:
- Client initialization with embedded defaults and environment overrides
- Kratos config integration for seamless remote configuration loading
- Watcher supports multiple DataIDs with cancellation and context control
- Discovery convenience methods for registering, deregistering, and subscribing to service instances
Section sources
Helm Charts for Infrastructure Components
Helm charts standardize deployments across environments:
- APISIX: Gateway, Admin API, metrics, and service exposure
- Nacos: Clustered service registry with external MySQL storage
- Kafka: KRaft mode cluster with persistence and resource limits
- Grafana: Dashboards and provisioning
Diagram sources
Section sources
Deployment Patterns
- Blue-green deployments: Alternate between two identical environments; switch traffic after validation
- Rolling updates: Gradual replacement with controlled restarts and readiness probes
- Canary releases: Route a small percentage of traffic to new versions for gradual rollout
Recommended practices:
- Use APISIX route weights or header-based routing for canary
- Maintain stable service names and Nacos registration for seamless failover
- Employ Git commit-based tagging for deterministic rollouts
[No sources needed since this section provides general guidance]
Infrastructure Requirements and Scaling Strategies
- APISIX: Horizontal scaling via replicas; resource requests/limits configured; optional HPA
- Nacos: Clustered deployment with persistent storage; resource requests/limits defined
- Kafka: Replicated brokers with disk-backed logs; tuned heap and partitions
- Grafana: Optional persistence; resource limits defined
Scaling levers:
- Increase APISIX replicas and enable autoscaling
- Scale Nacos statefulset replicas
- Grow Kafka cluster and adjust replication factors/partitions
- Provision additional Grafana replicas if needed
Section sources
Monitoring and Observability Setup
- APISIX metrics: Enable ServiceMonitor and Prometheus scraping
- Grafana dashboards: Predefined APISIX and Kubernetes dashboards included
- Tracing: Integrate OpenTelemetry/Jaeger via APISIX plugins or sidecars
Implementation pointers:
- Enable APISIX Prometheus exporter and ServiceMonitor in chart values
- Configure Grafana datasources and dashboards via values
- Use APISIX tracing plugins for distributed tracing
Section sources
Backup and Disaster Recovery Procedures
- Nacos: Back up MySQL database; restore on new cluster
- Kafka: Snapshot and restore topics; ensure replication factor and ISR alignment
- APISIX: Export and version control route configurations; rebuild from manifests
- Grafana: Back up dashboards and datasources; re-provision on recovery
[No sources needed since this section provides general guidance]
Security Configurations and Network Policies
- APISIX: IP whitelist, rate limiting, WAF rules, JWT authentication
- Nacos: Authentication enabled; secure connections and access controls
- Network policies: Restrict east-west traffic; allow only necessary ports
- Secrets management: Store JWT and gateway signing secrets in Kubernetes secrets
Section sources
Dependency Analysis
APISIX depends on Nacos for service discovery and on Kafka for messaging. The deployment pipeline integrates Docker builds with Kubernetes image updates.
Diagram sources
- [Publish Services Script]
- [APISIX Helm Values]
- [Nacos Helm Values]
- [Kafka Helm Values]
- [Grafana Helm Values]
Section sources
Performance Considerations
- Tune APISIX worker connections and timeouts for latency and throughput
- Enable HPA for APISIX and application pods
- Use Nacos caching and heartbeat intervals appropriately
- Optimize Kafka partition counts and replication for throughput
- Monitor Grafana dashboards for bottlenecks
[No sources needed since this section provides general guidance]
Troubleshooting Guide
- APISIX route issues: Use manager script to list, show, and recreate routes; verify security plugin configuration
- Nacos connectivity: Confirm server address/port/context path; validate authentication
- Service discovery: Ensure services are registered and healthy; check group/namespace
- Deployment failures: Review image tags, registry credentials, and resource limits
Section sources
Conclusion
The infrastructure leverages APISIX for robust traffic management and security, Nacos for reliable service discovery and configuration, and Helm for standardized deployments. With proper monitoring, scaling strategies, and operational procedures, the platform supports resilient, observable, and secure microservices delivery.