vitepress-template

Appearance

Bi-Cron Distributed Task Scheduling Service ​

**Referenced Files in This Document** - [[main.go]](file/bi-cron/main.go) - [[root.go]](file/bi-cron/cmd/root.go) - [[bi_api_jushuitan.go]](file/bi-cron/cmd/bi-api-jushuitan.go) - [[bi_api_leke.go]](file/bi-cron/cmd/bi-api-leke.go) - [[config.go]](file/bi-cron/internal/config/config.go) - [[client.go]](file/bi-cron/internal/grpc/client.go) - [[jushuitan_client.go]](file/bi-cron/internal/grpc/bi-api-jushuitan/client.go) - [[jushuitan_inventory.go]](file/bi-cron/internal/grpc/bi-api-jushuitan/inventory.go) - [[leke_client.go]](file/bi-cron/internal/grpc/bi-api-leke/client.go) - [[leke_goods.go]](file/bi-cron/internal/grpc/bi-api-leke/goods.go) - [[leke_order.go]](file/bi-cron/internal/grpc/bi-api-leke/order.go) - [[nacos-dev.yaml]](file/bi-cron/configs/nacos-dev.yaml) - [[cronjob.yaml]](file/bi-cron/k8s/cronjob.yaml)

Table of Contents ​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction ​

Bi-cron is a distributed task scheduling service designed to execute time-bound operations across BI microservices. It integrates with bi-api-jushuitan and bi-api-leke to synchronize data from external systems (Taobao and Jushuitan) via gRPC. The service is containerized and orchestrated using Kubernetes Jobs, enabling reliable, scalable execution of batch and maintenance tasks. Configuration is managed centrally through Nacos, supporting dynamic updates and environment-specific deployments.

Project Structure ​

The bi-cron project follows a layered structure:

  • Entry point initializes the CLI and delegates to command handlers.
  • Command handlers encapsulate task-specific logic and orchestrate gRPC client initialization.
  • Configuration module loads application settings from Nacos and exposes service discovery helpers.
  • gRPC client layer manages connections to target services and exposes typed clients for each domain operation.

Diagram sources

Section sources

Core Components ​

  • CLI and Command Layer: Uses Cobra to define hierarchical commands for task execution. Each task command initializes configuration, establishes gRPC connections, executes the operation, and logs outcomes.
  • Configuration and Service Discovery: Loads application configuration from Nacos, merges multiple data IDs, and resolves service endpoints via Nacos naming client.
  • gRPC Client Layer: Manages long-lived connections per target service, with thread-safe initialization and graceful shutdown. Provides typed client accessors for domain operations.
  • Kubernetes Orchestration: Kubernetes Job manifests define task invocations, environment variables, and resource limits, enabling controlled execution across clusters.

Section sources

Architecture Overview ​

Bi-cron orchestrates distributed tasks by:

  • Loading configuration from Nacos at runtime.
  • Discovering target service endpoints dynamically.
  • Establishing secure gRPC connections (insecure credentials in current setup).
  • Executing domain-specific operations with per-operation timeouts.
  • Logging structured results and propagating errors.

Diagram sources

Detailed Component Analysis ​

Configuration and Service Discovery ​

  • AppConfig defines gRPC service configurations for bi-api-leke and bi-api-jushuitan, including service name, group, and timeout.
  • Load merges multiple Nacos data IDs into a single configuration object.
  • GetServiceAddr resolves healthy instances via Nacos naming client.

Diagram sources

Section sources

gRPC Client Initialization and Lifecycle ​

  • Client initializer ensures thread-safe connection establishment and reuse.
  • Per-service clients expose typed accessors for domain operations.
  • Close ensures proper cleanup of connections.

Diagram sources

Section sources

Task Execution Patterns ​

  • Universal wrapper: runJob handles configuration loading, gRPC initialization, task execution, and logging for Leke tasks.
  • Specialized wrapper: runJushuitanJob performs similar steps but initializes only the Jushuitan client.
  • Each task constructs a background context, applies per-service timeout, and invokes the corresponding gRPC method.

Diagram sources

Section sources

Domain Operations: Jushuitan Inventory ​

  • AsyncFetch triggers asynchronous inventory fetching from Jushuitan.
  • Uses a dedicated client with per-operation timeout derived from configuration.

Diagram sources

Section sources

Domain Operations: Leke Goods and Orders ​

  • Goods: TaobaoItemsOnsale retrieves currently selling items.
  • Orders: TaobaoSyncFullOrder synchronizes historical orders; TaobaoSyncIncrementOrder handles incremental updates.

Diagram sources

Section sources

Kubernetes Orchestration and Scheduling ​

  • Kubernetes Jobs define task invocations for full and incremental order synchronization, as well as goods on-sale synchronization.
  • Environment-driven configuration path resolution via DEPLOY_ENV variable.
  • Resource requests and limits are specified for predictable scheduling.

Diagram sources

Section sources

Dependency Analysis ​

  • CLI depends on Cobra for command parsing and delegation.
  • Configuration module depends on Nacos SDK for config and naming clients.
  • gRPC clients depend on generated protobuf stubs and Google gRPC library.
  • Kubernetes Jobs depend on container images and Nacos configuration availability.

Diagram sources

Section sources

Performance Considerations ​

  • Connection Reuse: gRPC clients are initialized once and reused across tasks to minimize overhead.
  • Per-Operation Timeouts: Each task applies a timeout derived from configuration to prevent hanging operations.
  • Resource Limits: Kubernetes Jobs specify CPU and memory requests/limits to ensure predictable scheduling and isolation.
  • Service Discovery: Health checks and naming client selection ensure traffic is routed to healthy instances.

[No sources needed since this section provides general guidance]

Troubleshooting Guide ​

  • Configuration Loading Failures: Verify Nacos connectivity and data ID correctness. Check environment variable resolution for DEPLOY_ENV.
  • Service Discovery Failures: Confirm service names and groups match Nacos registrations. Validate namespace and group settings.
  • gRPC Connection Issues: Ensure target services are reachable and credentials align with deployment (current implementation uses insecure transport).
  • Task Execution Failures: Review task logs for RPC errors and timeouts. Validate that per-operation timeouts are sufficient for workload characteristics.

Section sources

Conclusion ​

Bi-cron provides a robust, configurable framework for executing distributed tasks against BI microservices. Its integration with Nacos enables dynamic configuration and service discovery, while Kubernetes Jobs offer reliable scheduling. The modular command structure and typed gRPC clients simplify extending the system with new tasks and domains.

[No sources needed since this section summarizes without analyzing specific files]

Appendices ​

Cron Job Configuration Examples ​

  • Increment Order Synchronization: Invoked via Kubernetes Job with arguments targeting the Leke order increment endpoint.
  • Full Order Synchronization: Similar invocation pattern for historical order sync.
  • Goods On-Sale Synchronization: Dedicated Job for retrieving current items.

Section sources

Creating Custom Cron Jobs ​

  • Define a new command under the appropriate service group in the CLI.
  • Implement the task function with context and timeout handling.
  • Add a Kubernetes Job manifest with the new command arguments and environment configuration.

Section sources