vitepress-template

Appearance

gRPC Service Definitions ​

**Referenced Files in This Document** - [[Makefile]](file/bi-proto/makefile) - [[shop.proto]](file/bi-proto/basic/v1/shop.proto) - [[category.proto]](file/bi-proto/basic/v1/category.proto) - [[template.proto]](file/bi-proto/analysis/v1/template.proto) - [[notification.proto]](file/bi-proto/notify/v1/notification.proto) - [[user.proto]](file/bi-proto/tenant/v1/user.proto) - [[order.proto]](file/bi-proto/leke/v1/order/order.proto) - [[repeat_shopping.proto]](file/bi-basic/api/repeat-shopping/v1/repeat-shopping.proto) - [[openapi.yaml (analysis)]](file/bi-proto/analysis/v1/openapi.yaml) - [[openapi.yaml (basic)]](file/bi-proto/basic/v1/openapi.yaml)

Table of Contents ​

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction ​

This document describes the gRPC service definitions and associated protobuf contracts used across the bi-proto module. It covers service names, method signatures, input/output message types, streaming patterns, validation rules, enums, and OpenAPI/Swagger generation for hybrid API scenarios. It also outlines metadata handling, authentication via JWT tokens, error propagation using gRPC status codes, and client implementation guidance for Go and Python.

Project Structure ​

The bi-proto module organizes protobuf definitions by domain:

  • basic/v1: Dictionary services for shops and categories
  • analysis/v1: Internal template service
  • notify/v1: Notification dispatch service
  • tenant/v1: Tenant/user dictionary service
  • leke/v1/order: Leke order synchronization APIs
  • Additional services under bi-basic/api/repeat_shopping/v1: Streaming analytics service

Diagram sources

Section sources

Core Components ​

  • ShopDictService: Unary gRPC service returning shop dictionaries keyed by shop identifiers.
  • CategoryDictService: Unary gRPC service returning network category dictionaries keyed by platform+category ID.
  • TemplateService: Unary gRPC service for listing templates, retrieving template info, and fetching favorite templates.
  • NotificationService: Unary gRPC service for sending notifications, batch sending, and querying status.
  • UserDictService: Unary gRPC service for retrieving user dictionaries per tenant.
  • Order service (Leke): Hybrid HTTP/gRPC service exposing order synchronization endpoints with OpenAPI annotations.
  • RepeatShoppingService: Streaming gRPC service emitting progress updates while computing repeat shopping statistics.

Streaming patterns:

  • Unary: All services except RepeatShoppingService.
  • Server streaming: RepeatShoppingService.SetShoppingRepeat returns stream SetShoppingRepeatReply.

Validation and OpenAPI:

  • Validation rules applied via the validate plugin for selected fields.
  • OpenAPI annotations present in several services for hybrid HTTP/gRPC exposure.

Section sources

Architecture Overview ​

The gRPC services are designed for inter-service communication within the BI ecosystem. Some services expose HTTP endpoints alongside gRPC for hybrid consumption. Validation and OpenAPI generation are integrated during proto compilation.

Diagram sources

Detailed Component Analysis ​

ShopDictService ​

  • Service name: ShopDictService
  • Methods:
    • ListShopDict: unary
  • Input: ListShopDictRequest
    • tenant_id: int64
  • Output: ListShopDictReply
    • shop: map<string, ShopDictItem>
    • shop_ztai: map<string, ShopZtaiItem>
  • Message types:
    • ShopDictItem: id, i_user_id, c_name, c_ab_name, c_platform, c_seller_nick, b, i_source_type
    • ShopZtaiItem: id, i_user_id, c_name, c_ab_name, c_platform, c_seller_nick, b, b_ztai_auth, b_tg_auth

Diagram sources

Section sources

CategoryDictService ​

  • Service name: CategoryDictService
  • Methods:
    • ListNetCategoryDict: unary
  • Input: ListNetCategoryDictRequest
    • tenant_id: int64
  • Output: ListNetCategoryDictReply
    • net_category: map<string, NetCategoryDictItem>
  • Message types:
    • NetCategoryDictItem: c_platform, c_cid, c_cid_name, c_ab_name

Diagram sources

Section sources

TemplateService ​

  • Service name: TemplateService
  • Methods:
    • ListTemplates: unary
    • GetTemplateInfo: unary
    • GetFavoriteTemplates: unary
  • Messages:
    • TemplateInfo: id, tenant_id, oper_id, c_name, c_icon, c_color, c_row, c_col, c_field, c_field_config, c_remark, t_create_time, i_sort, b
    • ListTemplatesRequest/Reply, GetTemplateInfoRequest/Reply, GetFavoriteTemplatesRequest/Reply

Diagram sources

Section sources

NotificationService ​

  • Service name: NotificationService
  • Methods:
    • SendNotification: unary
    • BatchSendNotification: unary
    • GetNotificationStatus: unary
  • Messages:
    • SendNotificationRequest: request_id, biz, event_type, sender, recipients, content, channels, options, extra
    • NotificationSender: oper_id, name, avatar, tenant_id
    • NotificationRecipient: tenant_id, user_id
    • NotificationContent: content_type, title, body, template_id, variables
    • NotificationSendOptions: async, priority
    • SendNotificationResponse: code, message, request_id, event_id, results
    • NotificationRecipientResult: recipient_id, channel, success, error_msg, message_id
    • BatchSendNotificationRequest/Response: repeated requests/results, success_count, fail_count
    • GetNotificationStatusRequest/Response: event_id, status, results, create_time, update_time

Diagram sources

Section sources

UserDictService ​

  • Service name: UserDictService
  • Methods:
    • ListUserDict: unary
  • Input: ListUserDictRequest
    • tenant_id: int64
  • Output: ListUserDictReply
    • users: map<string, UserDictItem>
  • Message types:
    • UserDictItem: id (optional), c_name, c_mobile, c_headimgurl, b, b_login, b_auth

Diagram sources

Section sources

Order Service (Leke) ​

  • Service name: Order
  • Methods:
    • TaobaoSyncFullOrder: unary (HTTP mapping configured)
    • TaobaoSyncIncrementOrder: unary (HTTP mapping configured)
    • TaobaoGetOrderInfo: unary (HTTP mapping configured)
  • Validation:
    • TaobaoGetOrderInfoRequest fields annotated with min length and required behavior.
  • OpenAPI:
    • Document info and operation metadata included.

Diagram sources

Section sources

RepeatShoppingService ​

  • Service name: RepeatShoppingService
  • Methods:
    • SetShoppingRepeat: server streaming
    • GetShoppingRepeat: unary
    • GetShoppingOrder: unary
  • Streaming pattern:
    • SetShoppingRepeat returns a stream of SetShoppingRepeatReply.
  • Validation:
    • Request fields use validate rules for counts, requiredness, and value ranges.
  • Enums:
    • CFieldEnum: i_real_byr, i_buyers, i_buyers_2, ..., i_buyers_more.

Diagram sources

Section sources

Dependency Analysis ​

The bi-proto module integrates validation and OpenAPI generation during proto compilation. The Makefile orchestrates:

  • protoc-gen-go for Go stubs
  • protoc-gen-go-grpc for gRPC service stubs
  • protoc-gen-go-http for HTTP gateway bindings
  • protoc-gen-validate for Go validation code
  • protoc-gen-openapi for OpenAPI schema generation

Diagram sources

Section sources

Performance Considerations ​

  • Prefer unary RPCs for small, bounded payloads; use server streaming for long-running computations with periodic updates (e.g., RepeatShoppingService).
  • Apply validation close to the boundary to fail fast and reduce downstream processing overhead.
  • Use pagination in list-style queries (e.g., RepeatShoppingService) to bound response sizes.
  • For hybrid HTTP/gRPC services, leverage HTTP caching and rate limiting at the gateway layer.

Troubleshooting Guide ​

  • Validation failures: Ensure request fields satisfy validate rules (e.g., minimum lengths, requiredness, numeric bounds).
  • OpenAPI generation: Verify that openapi annotations are present and that protoc-gen-openapi is installed and invoked by the Makefile targets.
  • Streaming RPCs: Ensure clients handle server streams properly and drain messages until completion.
  • Status codes: Use gRPC status codes to propagate errors; map domain-specific errors to appropriate canonical codes.

Section sources

Conclusion ​

The bi-proto module defines a set of gRPC services for internal BI workflows, with optional HTTP/gRPC hybrid exposure and robust validation and OpenAPI integration. Services cover dictionary retrieval, template management, notifications, user dictionaries, order synchronization, and streaming analytics. Clients should implement proper connection management, retries, and circuit breaking, and leverage JWT-based authentication as described below.

Appendices ​

Authentication and Metadata Handling ​

  • JWT tokens: Propagate JWT tokens via gRPC metadata (e.g., Authorization header) to authenticate requests.
  • Metadata keys: Use standard keys such as Authorization or a custom tenant-aware key as per your deployment policy.
  • Middleware: Implement gRPC interceptors to extract tokens from metadata, validate them, and attach contextual claims to the request context.

Error Propagation with gRPC Status Codes ​

  • Map domain errors to canonical gRPC status codes (e.g., InvalidArgument, NotFound, PermissionDenied, DeadlineExceeded).
  • Include structured error details using status details or trailers for richer diagnostics.

Client Implementation Examples ​

Go ​

  • Use the generated gRPC stubs and HTTP gateway bindings.
  • Example steps:
    • Initialize a gRPC client connection with TLS and keepalive.
    • Inject JWT via metadata using a per-RPC interceptor.
    • Call unary or streaming methods depending on service contract.
    • Handle validation errors returned as InvalidArgument with details.
    • For hybrid services, use HTTP clients to call mapped endpoints.

Python ​

  • Use grpcio and grpc-aio for asynchronous clients.
  • Example steps:
    • Create a secure channel and attach JWT metadata.
    • Invoke RPCs using the generated stubs.
    • For streaming RPCs, iterate over the response stream until completion.
    • Convert validation errors to appropriate exceptions.

Other Supported Languages ​

  • Java: Use the generated Java stubs with Netty or OkHttp transport.
  • Node.js: Use @grpc/grpc-js with metadata injection and error handling.
  • C++: Use the generated C++ stubs with proper channel configuration.

Connection Management, Load Balancing, Retry Policies, and Circuit Breakers ​

  • Connection management:
    • Use persistent connections with keepalive and retry on transient failures.
    • Configure timeouts per RPC based on service SLAs.
  • Load balancing:
    • Use gRPC load balancers (e.g., round_robin, ring_hash) behind service discovery.
  • Retry policies:
    • Implement idempotency-safe retries for unary calls with backoff.
    • Avoid retrying streaming RPCs unless explicitly idempotent.
  • Circuit breakers:
    • Track failure rates and latency; open the circuit on thresholds exceeded.
    • Expose health endpoints and degrade gracefully when open.

OpenAPI/Swagger Generation for Hybrid Scenarios ​

  • Enable OpenAPI generation via protoc-gen-openapi during proto compilation.
  • Review generated openapi.yaml files for each service to validate schema coverage.
  • For services with HTTP mappings, ensure operation IDs and descriptions align with the OpenAPI document annotations.

Section sources