This version is still in development and is not considered stable yet. For the latest stable version, please use Korvet 0.12.5!

Architecture Overview

This page explains the high-level architecture behind Korvet and points to deeper workflow, protocol, and internal references.

This section is for advanced users, operators, and contributors who need to understand how Kafka-compatible behavior is implemented on top of Redis. If you only need to run or use Korvet, start with the Getting Started, Kafka API, and Operations sections.

System Model

Korvet exposes a Kafka-compatible broker interface while persisting data in Redis-backed storage layers. At a high level:

  • Kafka protocol requests are terminated by the broker

  • Kafka topics and partitions map onto Redis-backed storage primitives

  • offsets are translated between Kafka semantics and Redis entry identifiers

  • consumer-group behavior combines broker-side coordination with Redis-native delivery primitives

Architecture Overview

Architecture Overview

Core Concepts

Kafka to Redis Model

Kafka Concept Redis Implementation Key Pattern

Topic Partition

Redis Stream

{topic}:{partition}

Message Offset

Stream Entry ID + Mapping

Sequential integers (0, 1, 2…​)

Consumer Group

Redis Consumer Group

Same name as Kafka group

Consumer Group Offset

Hash + Consumer Group State

Tracked in Redis

Message Key/Value/Headers

Stream Entry Fields

key, value, headers, timestamp

Design Principles

  • Offset Semantics: Kafka uses sequential integer offsets (0, 1, 2…​), Redis uses timestamp-based entry IDs. We maintain bidirectional mappings.

  • Atomicity: Use Redis transactions (MULTI/EXEC) and Lua scripts for atomic operations.

  • Performance: Leverage pipelining, connection pooling, and caching for high throughput.

  • Compatibility: Maintain full Kafka protocol compatibility for existing clients.

Choose The Right Depth

Start Here For Architecture

Use these pages when you want to understand overall behavior without reading every protocol detail:

Drill Into Protocol Mapping

Use these pages when you need request-by-request Kafka to Redis behavior:

Go Deeper Into Internals

Use these pages when debugging implementation details or contributing to the codebase: