For the latest stable version, please use Korvet 0.12.5!

Redis Data Structures Reference

This page provides a comprehensive reference of all Redis data structures used by Korvet.

Korvet uses Redis Streams exclusively to implement Kafka semantics:

Redis Key Layout for Topic "orders" with 3 Partitions

Partition Streams:
  ├─ orders:0 (Stream)
  │    ├─ 1234567890123-0: {key: "order-123", value: "...", timestamp: "1234567890"}
  │    └─ 1234567890123-1: {key: "order-124", value: "...", timestamp: "1234567891"}
  │
  ├─ orders:1 (Stream)
  │    └─ 1234567890456-0: {key: "order-125", value: "..."}
  │
  └─ orders:2 (Stream)
       └─ 1234567890789-0: {key: "order-126", value: "..."}

Consumer Groups (Redis Streams native):
  ├─ orders:0 has consumer group "my-group"
  │    └─ Managed by Redis: XGROUP, XREADGROUP, XACK
  │
  └─ orders:1 has consumer group "my-group"
       └─ Managed by Redis: XGROUP, XREADGROUP, XACK

Stream Keys

Each Kafka topic partition maps to a single Redis Stream:

{topic}:{partition}                    # Stream: message log

Examples:

orders:0        # Topic "orders", partition 0
orders:1        # Topic "orders", partition 1
payments:0      # Topic "payments", partition 0

Offset Encoding

Kafka offsets are stateless - encoded from Redis Stream entry IDs:

Entry ID format:  {timestamp}-{sequence}
Kafka offset:     (timestamp << N) | sequence

Where N = number of bits for sequence (typically 10-16 bits)

Example:

Entry ID:     "1234567890123-5"
Offset:       (1234567890123 << 10) | 5 = 1264197008485893

Consumer Groups

Consumer groups use Redis Streams native consumer groups:

# Create consumer group
XGROUP CREATE orders:0 my-group 0

# Read as group member
XREADGROUP GROUP my-group consumer-1 COUNT 100 STREAMS orders:0 >

# Acknowledge (commit offset)
XACK orders:0 my-group {entryId}