Event System - Getting Started

Overview

Our event system provides a robust event streaming architecture built on top of Redis Streams. This allows services within the monorepo to communicate asynchronously through domain events with persistence and consumer groups for scaling.

Architecture Overview

The event system consists of three main components:

1. Redis Stream Publisher: Publishes events to Redis Streams (channels)
2. Redis Stream Subscriber: Subscribes to Redis Streams using Consumer Groups
3. BullMQ Bridge: Routes events from Redis to BullMQ queues for reliable processing

┌──────────────────────────────────────────────────────────────┐
│                    Service A (Publisher)                     │
│                                                              │
│  Domain Service → Domain Event → Publisher → Redis Stream    │
└───────────────────────────────────┬──────────────────────────┘
                                    │
                                    ▼
                            ┌───────────────┐
                            │ Redis Stream  │
                            └───────┬───────┘
                                    │
                                    ▼
┌──────────────────────────────────────────────────────────────┐
│                   Service B (Subscriber)                     │
│                                                              │
│  Consumer Group → Subscriber → BullMQ Queue → Consumer       │
└──────────────────────────────────────────────────────────────┘

Key Benefits

• Decoupling: Services don't need to know about each other
• Scalability: Multiple services can subscribe to the same events. Horizontal scaling is handled natively by Redis Consumer Groups.
• Reliability: Redis Streams persist events (1 hour retention) and Consumer Groups ensure at-least-once delivery. BullMQ adds further reliability for processing.
• Async Processing: Non-blocking event handling
• Observability: Events are logged with correlation IDs for tracing

Redis Client Configuration

The event system uses a wrapper on the Redis client which provides a RedisStreamPublisherClient and a RedisStreamSubscriberClient. The Redis clients are configured via environment variables:

REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_STREAM_CONSUMER_GROUP=my-service-group

These are automatically loaded from the NestJS configuration system and passed to the Redis clients through the module registration.

Important: The REDIS_STREAM_CONSUMER_GROUP should be unique per microservice (e.g., coding-service, analytics-service) but shared across all pods of that service to enable load balancing.