bi-basic - Foundation Data Services
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
Introduction
bi-basic is the foundational data services layer that powers business entities and data access patterns across the platform. It exposes gRPC and HTTP APIs for core entities such as goods, orders, shops, categories, and related services. The layer integrates tightly with Kafka for asynchronous event processing, supports dynamic sharding of goods and order tables per tenant/platform/shop, and provides robust transaction handling, caching, and validation mechanisms. This document explains the service architecture, data models, CRUD and business logic, database design principles, indexing strategies, integration patterns, and operational considerations.
Project Structure
The bi-basic module follows a layered architecture:
- Entry point initializes Kratos application, loads configuration from Nacos, sets up logging, and wires providers.
- Server layer registers gRPC and HTTP services and applies middleware (logging, validation, metadata, auth).
- Data layer manages database and Redis clients, transaction helpers, and repository implementations.
- Service layer implements business logic for goods, orders, shops, categories, and Kafka adapters.
- Kafka layer consumes multiple topics concurrently with batching and manual offset commits.
Diagram sources
- [main.go]
- [wire.go]
- [server.go]
- [grpc.go]
- [http.go]
- [data.go]
- [goods.go]
- [order.go]
- [shop.go]
- [good_category.go]
- [goods.go]
- [order.go]
- [consumer.go]
Section sources
- [main.go]
- [wire.go]
- [server.go]
- [grpc.go]
- [http.go]
- [data.go]
- [goods.go]
- [order.go]
- [shop.go]
- [good_category.go]
- [goods.go]
- [order.go]
- [consumer.go]
Core Components
- Bootstrap and DI wiring: Kratos app bootstrapped via Nacos-configured YAML, with Wire provider sets for server, data, biz, and service layers.
- gRPC and HTTP servers: Register multiple domain services and apply middleware for logging, validation, metadata, and remote auth.
- Data layer: GORM-based ORM client, Redis client, transaction helpers, and repository constructors.
- Business services: Goods and Order services implement CRUD and business logic, map platform enums to names, and support SSE for long-running operations.
- Kafka consumers: Consume multiple topics (OrderCombined, AfterSaleOrder, GoodsSku) with configurable workers, batching, and manual offset commits.
Section sources
Architecture Overview
The system is a Kratos-based microservice exposing:
- gRPC services for internal and cross-service communication.
- HTTP endpoints for admin and external integrations, including SSE and Excel export.
- Data persistence via GORM with dynamic sharding and Redis caching.
- Asynchronous ingestion via Kafka with dedicated consumers per topic and worker pools.
Diagram sources
Detailed Component Analysis
Data Models and Dynamic Sharding
Core entities and their dynamic table naming:
- Goods: base_goods with dynamic suffixes based on tenant, platform, and shop identifiers.
- Order: base_order with dynamic suffixes for tenant, platform, and shop.
- Shop: base_shop with tenant-scoped dynamic suffix.
- Category: base_good_category for standardized category hierarchy.
Diagram sources
Section sources
Service Layer: Goods and Orders
- GoodsService: Implements list, create, delete, update, detail retrieval, batch updates, virtual flag toggles, import operations, SKU sync from external platforms, and link template management.
- OrderService: Provides get, create, and update operations, mapping platform enums to human-readable names and converting timestamps to/from protobuf.
Diagram sources
Section sources
Kafka Event Processing
- Consumers for OrderCombined, AfterSaleOrder, and GoodsSku topics.
- Configurable workers, batching, and manual offset commits.
- Dedicated worker pools and offset committers for throughput and reliability.
Diagram sources
Section sources
HTTP Authentication and Middleware
- Remote JWT verification via gRPC to bi-tenant for protected routes.
- White-listing for specific operations.
- SSE support for long-running tasks with custom streaming wrapper.
Section sources
Dependency Analysis
The dependency graph highlights the inversion-of-control achieved via Wire and Kratos providers.
Diagram sources
Section sources
Performance Considerations
- Dynamic table sharding: Tenant/platform/shop-aware table names reduce contention and improve locality for reads/writes.
- Transaction isolation: Centralized transaction helpers ensure consistent writes and rollback semantics.
- Caching: Redis client initialization supports cluster mode with address mapping for local development.
- Kafka throughput: Worker pools and batch processing minimize latency and maximize throughput for event-driven ingestion.
- Middleware overhead: Logging, validation, and metadata middleware applied consistently across transports.
[No sources needed since this section provides general guidance]
Troubleshooting Guide
- Kafka consumers disabled: The KafkaData consumer is intentionally disabled; only OrderCombined, AfterSaleOrder, and GoodsSku consumers are started.
- Manual offset commits: Consumers commit offsets after successful batch processing; failures do not advance offsets.
- Remote auth failures: HTTP endpoints requiring JWT must pass a valid token verified against bi-tenant; otherwise unauthorized errors are returned.
- SSE streaming: Ensure proper headers and context propagation for SSE endpoints.
Section sources
Conclusion
bi-basic provides a robust, modular foundation for business data operations with strong separation of concerns across layers. Its use of dynamic sharding, transaction helpers, Redis caching, and Kafka-based event processing enables scalability and resilience. The service layer encapsulates business logic, while the server layer ensures consistent middleware and authentication. Together, these components form a reliable backbone for downstream analytics and operational services.