Redis Caching (Redisx)
Table of Contents
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendices
Introduction
This document describes the Redisx caching system used across BI services. It covers client configuration, connection pooling, supported deployment modes (single, sentinel, cluster), environment-driven configuration, TLS support, and operational patterns. It also outlines how Redisx integrates with Kratos-based services, how to set up clients programmatically or via protobuf configuration, and how to handle cluster routing and address mapping for local development scenarios.
Project Structure
Redisx resides under bi-common/cache/redisx and exposes a thin wrapper around the official go-redis client. The module provides:
- A unified Client type that encapsulates a redis.UniversalClient
- Configuration structures for mode, addresses, credentials, pools, timeouts, TLS, and routing
- Environment variable loading and YAML defaults
- Helpers to convert Kratos protobuf configurations into Redis options
- Example integrations in multiple services
Diagram sources
- [client.go]
- [config.go]
- [setup.go]
- [env.go]
- [option.go]
- [redisx-default.yaml]
- [README.md]
- [data.go]
- [data.go]
- [data.go]
Section sources
- [README.md]
- [client.go]
- [config.go]
- [setup.go]
- [env.go]
- [option.go]
- [redisx-default.yaml]
- [data.go]
- [data.go]
- [data.go]
Core Components
- Client: Encapsulates a redis.UniversalClient and exposes Ping, Close, and Client accessors. It is created via NewClient or Setup and performs a health check on startup.
- Config: Central configuration model supporting single, sentinel, and cluster modes, with nested PoolConfig, TimeoutConfig, TLSConfig, and routing flags.
- Setup/MustSetup: Converts Kratos protobuf Redis config into Options and constructs a Client.
- Environment loading: LoadFromEnv reads environment variables and applies them to Config.
- Options: Functional Option helpers to override defaults for mode, addresses, credentials, pools, timeouts, TLS, routing, and custom dialers.
- Defaults: Embedded YAML provides sensible defaults for local and CI environments.
Key capabilities:
- Single, Sentinel, and Cluster modes
- Connection pooling controls (size, idle counts, timeouts, lifetimes)
- RESP3 protocol support
- Latency-based and random routing for cluster/sentinel
- TLS with optional client certificates and CA trust
- Environment-driven configuration and YAML defaults
Section sources
Architecture Overview
Redisx sits between services and the Redis backend. Services initialize Redisx during startup using either:
- Setup(MustSetup) with Kratos protobuf config
- NewClient(...) with functional Options
The Client delegates to go-redis’s UniversalClient, which selects the appropriate client implementation based on Mode (single, sentinel, cluster). Environment variables and YAML defaults are merged into Config prior to client creation.
Diagram sources
Section sources
Detailed Component Analysis
Client Lifecycle and Creation
- NewClient applies defaults, loads environment variables, applies Options, and creates the underlying client based on Mode.
- Health check is performed immediately after creation.
- Cleanup closes the underlying client.
Diagram sources
Section sources
Configuration Model and Defaults
- RootConfig wraps Config for YAML loading.
- Config embeds PoolConfig, TimeoutConfig, TLSConfig, and routing flags.
- DefaultConfig loads embedded YAML defaults; otherwise falls back to safe defaults.
- Clone produces a deep copy of Config.
Diagram sources
Section sources
Environment Variables and YAML Defaults
- LoadFromEnv populates Config from environment variables with precedence over YAML defaults.
- parseAddrs splits comma-separated addresses.
- redisx-default.yaml provides sane defaults for local development.
Supported environment variables include mode, addresses, credentials, protocol, retries, routing flags, pool sizes/timeouts, dial/read/write timeouts, and TLS settings.
Section sources
Protobuf to Options Conversion
- convertProtoToOptions inspects a protobuf Redis config and generates Option functions for Mode, Addrs, Credentials, DB, Protocol, MaxRetries, MasterName, routing flags, PoolConfig, and Timeout.
- It supports both new-style nested fields and legacy top-level fields.
- MustSetup and Setup return a Client and cleanup function.
Diagram sources
Section sources
TLS Configuration
- buildTLSConfig constructs a tls.Config when TLS is enabled.
- Supports CA certificate loading, optional client certificate/key pair, and SNI ServerName.
- Used by createClient for all modes.
Section sources
Cluster Address Mapping (Local Development)
- bi-basic demonstrates a custom Dialer to map Redis cluster internal IPs to local port-forward endpoints.
- This pattern is useful when connecting to Kubernetes-hosted clusters locally.
Section sources
Service Integration Examples
- bi-common/README.md shows MustSetup and NewClient usage in Kratos projects.
- bi-analysis demonstrates optional Redis initialization; if no Redis config is present, Redis is skipped with a warning.
- bi-basic shows conditional logic for cluster mode with a custom Dialer.
Section sources
Dependency Analysis
Redisx depends on:
- go-redis/v9 for the underlying client implementations
- YAML parsing for defaults
- Environment utilities for environment variable loading
- Reflection for protobuf conversion
Diagram sources
Section sources
Performance Considerations
- Connection pooling: Tune PoolSize, MinIdleConns, MaxIdleConns, PoolTimeout, ConnMaxIdleTime, and ConnMaxLifetime to match workload characteristics.
- Routing: Enable RouteByLatency and/or RouteRandomly for cluster/sentinel to improve latency and distribute reads.
- Timeouts: Adjust Dial, Read, and Write timeouts according to network conditions and SLAs.
- Protocol: Prefer RESP3 for richer data types and improved performance where supported.
- TLS overhead: Certificate verification and handshakes add CPU cost; ensure CA trust is configured efficiently.
[No sources needed since this section provides general guidance]
Troubleshooting Guide
Common issues and resolutions:
- Connection failures: Verify Mode, Addrs, and credentials. Confirm environment variables or YAML configuration.
- Cluster MOVED/ASK errors: Use a custom Dialer to map internal IPs to local port-forward endpoints (see bi-basic example).
- TLS handshake errors: Ensure CAPath/CertPath/KeyPath are correct and accessible; configure ServerName if needed.
- Health check failure: Run Ping(ctx) after client creation to detect connectivity problems early.
- Configuration precedence: Remember environment variables override user Options and YAML defaults.
Section sources
Conclusion
Redisx provides a concise, configurable, and Kratos-native way to integrate Redis across BI services. It supports single, sentinel, and cluster deployments, offers robust connection pooling and TLS, and enables environment-driven configuration. The included examples demonstrate practical integration patterns, including cluster address mapping for local development and optional Redis initialization in services.
[No sources needed since this section summarizes without analyzing specific files]
Appendices
Configuration Reference
- Modes: single, sentinel, cluster
- Pool: pool_size, min_idle_conns, max_idle_conns, pool_timeout, conn_max_idle_time, conn_max_lifetime
- Timeouts: dial, read, write
- TLS: enabled, skip_verify, ca_path, cert_path, key_path, server_name
- Routing: route_by_latency, route_randomly
Section sources
Service Integration Patterns
- One-line setup with MustSetup/Setup using Kratos protobuf config
- Programmatic setup with NewClient and Options
- Optional Redis initialization with graceful fallback
- Cluster-specific address mapping for local port-forward scenarios
Section sources