Monitoring and Analytics
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendices
Introduction
This document describes the Monitoring and Analytics dashboard within the Admin Panel. It covers the system monitoring interface, service health checks, performance metrics, operational visibility, log aggregation and analysis, alerting mechanisms, incident response workflows, metrics visualization, trend analysis, capacity planning, audit trails, user activity tracking, compliance reporting, integration with observability tools, metrics collection, and real-time dashboards. It also provides practical examples for investigating system issues, analyzing performance trends, and generating operational reports.
Project Structure
The Monitoring and Analytics capability is primarily implemented in the bi-analysis service, which exposes a query engine and integrates with observability primitives. Supporting components include:
- Metrics instrumentation via Prometheus-compatible counters and histograms
- Structured logging with local and cloud-backed sinks
- Kubernetes deployment artifacts for service exposure and scaling
- Configuration files for environment-specific behavior
Diagram sources
Section sources
Core Components
- Query Engine: Builds and executes analytical queries across dimensional and metric datasets, supporting ranking, aggregation, pagination, and formula-driven fields.
- Metrics Instrumentation: Exposes Prometheus-compatible counters and histograms for HTTP requests, database operations, cache, and business operations.
- Logging: Provides structured logging with configurable sinks (stdout, file, Aliyun SLS) and filtering.
- Admin Dashboard Areas: The Admin Panel includes dedicated areas for system logs, configurations, departments, dictionaries, menus, roles, and users, enabling operational oversight and compliance activities.
- Metrics Catalog: A curated mapping of business metrics and formulas to underlying identifiers for consistent reporting.
Key implementation references:
- Query builder and execution pipeline: [query.go]
- Metrics instrumentation API: [metrics.go]
- Logging configuration and sinks: [logger.go]
- Business metrics catalog: [metrics.go]
Section sources
Architecture Overview
The Monitoring and Analytics dashboard integrates:
- Frontend Admin Panel areas for system logs, configurations, and user management
- bi-analysis service exposing query APIs and metrics
- Observability stack (Prometheus scraping, structured logs, and optional cloud logging)
- Kubernetes service mesh and load balancing
Diagram sources
Detailed Component Analysis
Query Engine and Analytics Execution
The query engine parses user requests, resolves field and dimension metadata, handles formula dependencies recursively, constructs SQL, executes it against data sources, and formats results. It supports:
- Row/column grouping and field selection
- Search conditions with date ranges and filters
- Pagination and ranking
- Formula fields by extracting referenced fields and loading their definitions
- Shop-aware queries for order/finance domains
Diagram sources
Section sources
Metrics Instrumentation
The observability metrics module defines:
- HTTP request counters and durations
- Database query counters, durations, and error counters
- Cache hit/miss counters and operation durations
- Business operation counters
These metrics are recorded via helper functions and exported for Prometheus scraping.
Diagram sources
Section sources
Logging and Log Aggregation
Logging supports:
- Local stdout/file outputs with rotation and retention
- Optional integration with cloud log services
- Structured JSON or text formats
- Level-based filtering
Diagram sources
Section sources
Admin Panel Operational Areas
The Admin Panel includes:
- System logs: centralized log viewing and filtering
- Configurations: environment and feature toggles
- Departments, dictionaries, menus, roles, users: organizational and access control management
These areas support audit trails, compliance reporting, and operational oversight.
[No sources needed since this section does not analyze specific files]
Metrics Catalog and Formulas
The metrics catalog maps human-readable metric names to underlying identifiers across categories:
- Traffic metrics (impressions, clicks, conversion rates)
- Finance metrics (payments, refunds, costs)
- Consultation metrics
- WuJie metrics
- Other operational indicators
- Custom metrics and formula fields
- Derived metrics
This enables consistent reporting and reduces ambiguity in dashboards.
Section sources
Dependency Analysis
The bi-analysis service depends on:
- Observability modules for metrics and logging
- Kubernetes manifests for deployment and service exposure
- Environment-specific configuration loaded from Nacos or local YAML
Diagram sources
- [main.go]
- [metrics.go]
- [logger.go]
- [query.go]
- [home.go]
- [metrics.go]
- [service.yaml]
- [deployment.yaml]
- [application-dev.yaml]
- [application-prod.yaml]
Section sources
Performance Considerations
- Query performance: Use pagination, limit result sets, and avoid heavy joins when possible. Prefer indexed dimensions and filtered date ranges.
- Metrics overhead: Instrument only critical paths; batch or aggregate metrics where feasible.
- Logging volume: Tune retention and rotation; consider sampling high-frequency logs.
- Scaling: Deploy multiple replicas behind the Kubernetes Service and enable horizontal pod autoscaling based on CPU or custom metrics.
[No sources needed since this section provides general guidance]
Troubleshooting Guide
Common scenarios and actions:
- Investigating slow queries
- Use the query engine’s SQL generation endpoint to inspect generated SQL and confirm filters and joins.
- Validate date ranges and shop filters to ensure they reduce dataset size.
- Reference: [query.go]
- Identifying bottlenecks
- Monitor HTTP request durations and error rates via Prometheus metrics.
- Review database query durations and error counters to locate slow or failing operations.
- Reference: [metrics.go]
- Debugging logs
- Increase log level temporarily and verify output routing (stdout/file/cloud).
- Confirm log format and retention settings.
- Reference: [logger.go]
- Capacity planning
- Track business operation totals and trends to forecast growth.
- Reference: [metrics.go]
- Compliance reporting
- Use Admin Panel areas for configurations, roles, and user management to maintain audit trails.
- Reference: [README.md]
Section sources
Conclusion
The Monitoring and Analytics dashboard leverages a robust query engine, standardized metrics instrumentation, and structured logging to deliver operational visibility, performance insights, and compliance support. Together with the Admin Panel’s operational areas, it enables effective incident response, trend analysis, and capacity planning.
[No sources needed since this section summarizes without analyzing specific files]
Appendices
Example Tasks and Workflows
- Investigate system issues
- Inspect recent system logs and filter by severity.
- Correlate HTTP error spikes with database query failures.
- Reference: [logger.go], [metrics.go]
- Analyze performance trends
- Plot HTTP request durations and business operation totals over time.
- Reference: [metrics.go]
- Generate operational reports
- Aggregate finance and traffic metrics across departments and time windows.
- Reference: [metrics.go], [query.go]
[No sources needed since this section provides general guidance]