Configuration

Korvet is configured using Spring Boot’s standard configuration mechanisms.

Configuration Files

Configuration can be provided via:

  • application.yml or application.properties

  • Environment variables

  • Command-line arguments

Basic Configuration

korvet:
  server:
    port: 9092
    host: 0.0.0.0
  redis:
    uri: redis://localhost:6379

Redis Configuration

Configure the Redis connection:

korvet:
  redis:
    uri: redis://localhost:6379
    username: default
    password: ${REDIS_PASSWORD}

You can also embed credentials in the URI:

korvet:
  redis:
    uri: redis://username:password@redis.example.com:6379

For TLS/SSL connections, use the rediss:// scheme:

korvet:
  redis:
    uri: rediss://redis.example.com:6379
    username: default
    password: ${REDIS_PASSWORD}

For Redis Cluster:

korvet:
  redis:
    uri: redis://node1:6379,node2:6379,node3:6379
    cluster: true
    username: default
    password: ${REDIS_PASSWORD}

Topic Configuration

Configure topic behavior:

korvet:
  topics:
    auto-create: true         # Automatically create topics when they don't exist
    partitions: 1             # Default number of partitions for auto-created topics
    retention-time: 7d        # Default topic retention
    bucket-duration: 1h       # Default bucket duration for time-bucketed streams

When auto-create is disabled, topics must be created explicitly using the Kafka AdminClient or command-line tools before they can be used.

Offset Encoding Configuration

Korvet uses a stateless encoding scheme to convert Redis Stream entry IDs (timestamp-sequence pairs) into Kafka offsets. The offset-sequence-bits setting controls how many bits are allocated for the sequence number portion of the offset.

korvet:
  topics:
    offset-sequence-bits: 10  # Default: 10 bits (supports up to ~1 million messages/second)

Understanding Offset Encoding

Redis Stream entry IDs have the format {timestamp}-{sequence} (e.g., 1732896000000-0). Korvet encodes these into Kafka offsets using bit-packing:

Kafka offset = (timestamp << offset-sequence-bits) | sequence

When to Change This Setting

The default value of 10 bits is suitable for most use cases, supporting up to ~1 million messages/second per partition.

You should consider changing this setting if:

Scenario Recommended Value Maximum Throughput

Standard throughput
(up to 1M msg/sec)

10 (default)

~1 million messages/second

High throughput
(1M - 16M msg/sec)

14

~16 million messages/second

Very high throughput
(>16M msg/sec)

16

~65 million messages/second

Low throughput
(<100K msg/sec)

8

~256,000 messages/second

Trade-offs

Lower values (8-10 bits):

  • ✅ Smaller offset deltas between messages

  • ✅ Better compatibility with Kafka clients that have offset delta limits

  • ❌ Lower maximum throughput per partition

Higher values (14-16 bits):

  • ✅ Higher maximum throughput per partition

  • ❌ Larger offset deltas between messages from different milliseconds

  • ❌ May exceed Kafka’s maximum offset delta limit (Integer.MAX_VALUE) for messages far apart in time

Maximum Offset Delta Constraint

Kafka has a constraint that the offset delta between consecutive messages in a fetch response cannot exceed Integer.MAX_VALUE (2,147,483,647).

With different sequence bit settings, messages from different milliseconds have different offset deltas:

Sequence Bits Offset Delta/ms Max Time Span

8 bits

256

~97 days

10 bits (default)

1,024

~24 days

14 bits

16,384

~36 hours

16 bits

65,536

~9 hours
If you use higher sequence bits (14-16), ensure your consumers fetch messages regularly to avoid exceeding the offset delta limit.

Example Configurations

For a high-throughput logging system:

korvet:
  server:
    offset-sequence-bits: 14  # Support up to ~16 million messages/second

For a low-throughput event system with long retention:

korvet:
  server:
    offset-sequence-bits: 8  # Lower offset delta, supports longer time spans (~256K msg/sec max)

Consumer Group Configuration

Configure consumer group rebalancing behavior:

korvet:
  server:
    rebalance-delay: 3s  # Delay before completing rebalance (default: 3s)

The rebalance-delay setting controls how long Korvet waits before completing a consumer group rebalance to allow more members to join. Increase this value for larger consumer groups, especially in Kubernetes environments where pods may start at different times.

TLS Configuration

Enable TLS for the Kafka protocol endpoint:

korvet:
  server:
    tls: true
    cert-file: /path/to/server.crt
    key-file: /path/to/server.key
    key-password: ${KEY_PASSWORD}

Cold Storage Configuration

Enable tiered storage with Delta Lake for archival of old messages:

korvet:
  server:
    storage:
      path: s3a://my-bucket/korvet/delta  # S3 path or file:///path/to/delta for local
      s3:
        region: us-east-1
        credentials:
          type: iam  # Use IAM role for authentication
Cold storage is automatically enabled when path is set. There is no separate enabled property.

For static credentials instead of IAM role:

korvet:
  server:
    storage:
      path: s3a://my-bucket/korvet/delta
      s3:
        region: us-east-1
        credentials:
          type: static
          access-key-id: ${AWS_ACCESS_KEY_ID}
          secret-access-key: ${AWS_SECRET_ACCESS_KEY}

For LocalStack or MinIO:

korvet:
  server:
    storage:
      path: s3a://my-bucket/korvet/delta
      s3:
        region: us-east-1
        endpoint: http://localhost:4566  # LocalStack endpoint
        credentials:
          type: static
          access-key-id: test
          secret-access-key: test
Cold storage is optional. If not configured, all reads will be served from Redis Streams (hot storage only).

See Cold Storage for complete configuration options and per-topic retention settings.

Redis Metrics Configuration

Korvet can optionally enable Lettuce command latency metrics to track Redis operation performance.

These metrics are disabled by default to minimize overhead. Enable them when you need detailed Redis performance monitoring. 
korvet:
  redis:
    metrics:
      enabled: true  # Enable Lettuce command metrics (default: false)
      histogram: false  # Enable histogram buckets for percentiles (default: false)
      local-distinction: false  # Track per connection vs per host (default: false)
      max-latency: 5m  # Maximum expected latency (default: 5 minutes)
      min-latency: 1ms  # Minimum expected latency (default: 1 millisecond)

Configuration Properties:

  • enabled: Enable Lettuce command latency metrics (default: false)

  • histogram: Enable histogram buckets for aggregable percentile approximations (default: false)

  • local-distinction: Track metrics per connection instead of per host/port (default: false)

  • max-latency: Maximum expected latency for histogram buckets (default: 5m, only applies when histogram is enabled)

  • min-latency: Minimum expected latency for histogram buckets (default: 1ms, only applies when histogram is enabled)

When enabled, Lettuce will publish two timer metrics:

  • lettuce.command.firstresponse - Time to first response from Redis

  • lettuce.command.completion - Time to complete Redis command

Each metric includes tags: command (e.g., XADD, XREAD), local (if local-distinction enabled), and remote (Redis server address).

For more details, see Lettuce Redis Client Metrics.

Environment Variables

All configuration can be set via environment variables using Spring Boot’s relaxed binding. Property paths are converted to uppercase with underscores:

# Server configuration
export KORVET_SERVER_HOST=0.0.0.0
export KORVET_SERVER_PORT=9092
export KORVET_SERVER_BROKER_ID=0

# Redis configuration
export KORVET_REDIS_URI=redis://redis.example.com:6379
export KORVET_REDIS_USERNAME=default
export KORVET_REDIS_PASSWORD=secret

# Redis metrics (optional)
export KORVET_REDIS_METRICS_ENABLED=false
export KORVET_REDIS_METRICS_HISTOGRAM=false
export KORVET_REDIS_METRICS_LOCAL_DISTINCTION=false
export KORVET_REDIS_METRICS_MAX_LATENCY=5m
export KORVET_REDIS_METRICS_MIN_LATENCY=1ms

# Topic configuration
export KORVET_SERVER_AUTO_CREATE_TOPICS=true
export KORVET_SERVER_DEFAULT_PARTITIONS=1

# Cold storage configuration (optional)
export KORVET_STORAGE_PATH=s3a://my-bucket/korvet/delta
export KORVET_STORAGE_S3_REGION=us-east-1
export KORVET_STORAGE_S3_CREDENTIALS_TYPE=iam
# For static credentials:
# export KORVET_STORAGE_S3_CREDENTIALS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
# export KORVET_STORAGE_S3_CREDENTIALS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Next Steps