For the latest stable version, please use Korvet 0.12.5!

Configuration Reference

Complete configuration reference for Korvet.

Configuration Properties

Server Configuration

Property Default Description

korvet.server.host

0.0.0.0

Host address to bind the server

korvet.server.port

9092

Port to listen on

korvet.server.broker-id

0

Broker ID for this server

korvet.server.keyspace

korvet

Redis keyspace prefix for all keys

korvet.server.advertised-host

host

Advertised host for clients (if not set, uses host)

korvet.server.advertised-port

port

Advertised port for clients (if not set, uses port)

korvet.server.boss-threads

1

Netty boss thread pool size

korvet.server.worker-threads

0

Netty worker thread pool size (0 = 2 × CPU cores)

korvet.server.max-request-size

100MB

Maximum request size

korvet.server.partition-max-bytes

1MB

Maximum bytes per partition in fetch requests

korvet.server.fetch-max-wait

500ms

Maximum time to wait for fetch requests

korvet.server.auto-create-topics

true

Automatically create topics when they don’t exist

korvet.server.default-partitions

1

Default partitions for auto-created topics

korvet.server.default-offset-sequence-bits

10

Bits for sequence number in offset encoding (1-16)

korvet.server.default-retention-time

7d

Default time-based retention for topics

korvet.server.default-retention-size

-1

Default size-based retention (-1 = unlimited)

korvet.server.default-compression

none

Default compression type (none, gzip, snappy, lz4, zstd)

korvet.server.rebalance-delay

3s

Delay before completing consumer group rebalance

TLS Configuration

Property Default Description

korvet.server.tls

false

Enable TLS for the Kafka protocol endpoint

korvet.server.cert-file

none

Server certificate file (PEM format)

korvet.server.key-file

none

Server private key file (PEM format)

korvet.server.key-password

none

Password for the private key file

korvet.server.trust-cert-file

none

Trusted CA certificates file (PEM format)

korvet.server.client-auth-required

false

Require client certificate authentication

Redis Configuration

Property Default Description

korvet.redis.uri

none

Redis connection URI (e.g., redis://localhost:6379)

korvet.redis.host

localhost

Redis host (used if URI not set)

korvet.redis.port

6379

Redis port (used if URI not set)

korvet.redis.username

none

Redis username

korvet.redis.password

none

Redis password

korvet.redis.cluster

false

Enable Redis Cluster mode

korvet.redis.timeout

none

Connection timeout

korvet.redis.pool-size

8

Connection pool size

korvet.redis.io-thread-pool-size

CPU cores

Lettuce I/O thread pool size

Redis Metrics Configuration

Property Default Description

korvet.redis.metrics.enabled

false

Enable Lettuce command latency metrics

korvet.redis.metrics.histogram

false

Enable histogram buckets for percentiles

korvet.redis.metrics.local-distinction

false

Track metrics per connection vs per host

korvet.redis.metrics.max-latency

5m

Maximum expected latency for histograms

korvet.redis.metrics.min-latency

1ms

Minimum expected latency for histograms

Warm Redis Configuration (Optional)

The warm tier uses a separate Redis instance (typically Redis Flex with disk-backed storage) for intermediate storage between hot (RAM) and cold (Delta Lake) tiers.

Property Default Description

korvet.warm-redis.uri

none

Warm Redis connection URI (e.g., redis://flex-cluster:6379)

korvet.warm-redis.host

none

Warm Redis host (used if URI not set)

korvet.warm-redis.port

6379

Warm Redis port (used if URI not set)

korvet.warm-redis.username

none

Warm Redis username

korvet.warm-redis.password

none

Warm Redis password

korvet.warm-redis.cluster

false

Enable Redis Cluster mode for warm tier

korvet.warm-redis.timeout

none

Connection timeout

korvet.warm-redis.pool-size

same as main

Connection pool size (defaults to main Redis pool size)

korvet.warm-redis.io-thread-pool-size

CPU cores

Lettuce I/O thread pool size
Warm tier is only enabled when korvet.warm-redis.uri or korvet.warm-redis.host is configured.

Storage Configuration

Warm Tier Archiver

Property Default Description

korvet.server.storage.near.keyspace

same as main

Keyspace prefix for warm tier streams

korvet.server.storage.near.archiver.consumer-group

korvet-near-archiver

Consumer group name for warm archiver

korvet.server.storage.near.archiver.batch-size

10000

Messages per batch when archiving to warm tier

korvet.server.storage.near.archiver.block-duration-ms

100

Block duration for XREADGROUP in milliseconds

korvet.server.storage.near.archiver.read-workers

1

Number of worker threads for reading from hot tier

korvet.server.storage.near.archiver.stream-refresh-interval-ms

30000

Interval to refresh stream list in milliseconds

Cold Tier (Delta Lake)

Cold storage is enabled automatically when korvet.server.storage.path is configured.

Property Default Description

korvet.server.storage.path

none

Delta Lake storage path (e.g., s3a://bucket/path or file:///tmp/delta). Setting this enables cold storage.

korvet.server.storage.s3.region

none

AWS region (required for S3)

korvet.server.storage.s3.endpoint

none

Custom S3 endpoint (for MinIO or LocalStack)

korvet.server.storage.s3.credentials.type

iam

Credential type: iam or static

korvet.server.storage.archiver.consumer-group

korvet-archiver

Consumer group name for cold archiver

korvet.server.storage.archiver.batch-size

10000

Messages per batch when archiving to cold tier

korvet.server.storage.archiver.read-workers

1

Number of worker threads for reading

korvet.server.storage.archiver.commit-workers

4

Number of worker threads for writing to S3

See Cold Storage Configuration for complete configuration details and S3 performance tuning.

Default Configuration

The following shows the default application.yml configuration file:

# Korvet Server Configuration
#
# All properties have sensible defaults defined in code.
# Override using environment variables (e.g., KORVET_REDIS_URI) or this file.
# See docs/modules/ROOT/pages/reference/configuration.adoc for full reference.

spring:
  application:
    name: korvet

# Korvet Configuration
# Uncomment and modify properties to override defaults.
# korvet:
#   # Main Redis (hot tier) configuration
#   # Defaults: host=localhost, port=6379, pool-size=8
#   redis:
#     uri: redis://localhost:6379
#     # Alternative: specify host/port instead of URI
#     # host: localhost
#     # port: 6379
#     # cluster: false  # Enable for Redis Cluster (OSS Cluster API)
#     # username: default
#     # password: secret
#     # pool-size: 8
#     # io-thread-pool-size: 4
#
#   # Warm Redis (optional) - Redis Flex or secondary Redis cluster
#   # warm-redis:
#     # uri: redis://flex-cluster:6379
#     # cluster: false
#     # pool-size: 8
#
#   server:
#     # Network configuration (defaults: host=0.0.0.0, port=9092)
#     # host: 0.0.0.0
#     # port: 9092
#     # broker-id: 0
#     # keyspace: korvet
#     # advertised-host: localhost
#     # advertised-port: 9092
#
#     # Thread pool configuration
#     # boss-threads: 1
#     # worker-threads: 0  # 0 = use Netty default (2 * CPU cores)
#
#     # Request size limits
#     # max-request-size: 100MB
#
#     # Topic configuration
#     # auto-create-topics: true
#     # default-partitions: 1
#
#     # TLS/SSL configuration
#     # tls: true
#     # cert-file: /path/to/server.crt
#     # key-file: /path/to/server.key
#
#   # Tiered Storage Configuration (optional)
#   # storage:
#     # near:
#       # keyspace: korvet-warm
#     # remote:
#       # path: s3a://my-bucket/korvet/delta
#       # s3:
#         # region: us-east-1

# Spring Boot Actuator Configuration
management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus,metrics
  endpoint:
    health:
      show-details: always
  prometheus:
    metrics:
      export:
        enabled: true

# Logging Configuration
logging:
  level:
    root: INFO
    com.redis.korvet: DEBUG
    # Suppress Netty internal messages (native transport detection, buffer allocation, etc.)
    io.netty.util.internal: WARN
    io.netty.buffer: WARN

Environment Variables

All configuration can be set via environment variables using Spring Boot’s relaxed binding:

# Server Configuration
export KORVET_SERVER_HOST=0.0.0.0
export KORVET_SERVER_PORT=9092
export KORVET_SERVER_BROKER_ID=0
export KORVET_SERVER_KEYSPACE=korvet
export KORVET_SERVER_ADVERTISED_HOST=localhost
export KORVET_SERVER_ADVERTISED_PORT=9092

# Thread pool configuration
export KORVET_SERVER_BOSS_THREADS=1
export KORVET_SERVER_WORKER_THREADS=0  # 0 = use Netty default (2 * CPU cores)

# Request size limits
export KORVET_SERVER_MAX_REQUEST_SIZE=100MB
export KORVET_SERVER_PARTITION_MAX_SIZE=1MB

# Fetch Configuration
export KORVET_SERVER_FETCH_MAX_WAIT=500ms  # Maximum time to wait for fetch requests when no data is available

# Offset Encoding
# Number of bits for sequence number in offset encoding (1-16)
# Default: 10 (supports up to ~1 million messages/second per partition)
# See xref:quick-start/configuration.adoc#_offset_encoding_configuration[Offset Encoding Configuration] for details
export KORVET_SERVER_DEFAULT_OFFSET_SEQUENCE_BITS=10

# Topic Configuration
export KORVET_SERVER_AUTO_CREATE_TOPICS=true
export KORVET_SERVER_DEFAULT_PARTITIONS=1
export KORVET_SERVER_DEFAULT_RETENTION_TIME=7d
export KORVET_SERVER_DEFAULT_RETENTION_SIZE=-1  # -1 = unlimited
export KORVET_SERVER_DEFAULT_COMPRESSION=none  # none, gzip, snappy, lz4, zstd

# Consumer Group Configuration
export KORVET_SERVER_REBALANCE_DELAY=3s  # Delay before completing rebalance (default: 3s)

# TLS Configuration (optional)
export KORVET_SERVER_TLS=true
export KORVET_SERVER_CERT_FILE=/path/to/server.crt
export KORVET_SERVER_KEY_FILE=/path/to/server.key
export KORVET_SERVER_KEY_PASSWORD=secret
export KORVET_SERVER_TRUST_CERT_FILE=/path/to/ca.crt
export KORVET_SERVER_CLIENT_AUTH_REQUIRED=false

# Redis Connection
export KORVET_REDIS_URI=redis://localhost:6379
export KORVET_REDIS_HOST=localhost
export KORVET_REDIS_PORT=6379
export KORVET_REDIS_USERNAME=default
export KORVET_REDIS_PASSWORD=secret
export KORVET_REDIS_CLUSTER=false
export KORVET_REDIS_TIMEOUT=5s  # Connection timeout (optional)
export KORVET_REDIS_POOL_SIZE=8  # Connection pool size (default: 8)
export KORVET_REDIS_IO_THREAD_POOL_SIZE=4  # Lettuce I/O thread pool size (default: number of CPUs)

# Redis Metrics (optional)
# Enables Lettuce command latency metrics published to Micrometer
export KORVET_REDIS_METRICS_ENABLED=false  # Default: false
export KORVET_REDIS_METRICS_HISTOGRAM=false  # Default: false
export KORVET_REDIS_METRICS_LOCAL_DISTINCTION=false  # Default: false
export KORVET_REDIS_METRICS_MAX_LATENCY=5m  # Default: 5m
export KORVET_REDIS_METRICS_MIN_LATENCY=1ms  # Default: 1ms

# Warm Redis Connection (optional - enables warm tier)
# Connects to a secondary Redis instance (typically Redis Flex with disk-backed storage)
export KORVET_WARM_REDIS_URI=redis://flex-cluster:6379
export KORVET_WARM_REDIS_HOST=flex-cluster  # Alternative to URI
export KORVET_WARM_REDIS_PORT=6379
export KORVET_WARM_REDIS_USERNAME=default
export KORVET_WARM_REDIS_PASSWORD=secret
export KORVET_WARM_REDIS_CLUSTER=false
export KORVET_WARM_REDIS_TIMEOUT=5s
export KORVET_WARM_REDIS_POOL_SIZE=8

# Warm Archiver Configuration (optional, used when warm-redis is configured)
export KORVET_SERVER_STORAGE_NEAR_KEYSPACE=korvet  # Keyspace for warm tier (default: same as main)
export KORVET_SERVER_STORAGE_NEAR_ARCHIVER_CONSUMER_GROUP=korvet-warm-archiver
export KORVET_SERVER_STORAGE_NEAR_ARCHIVER_BATCH_SIZE=10000
export KORVET_SERVER_STORAGE_NEAR_ARCHIVER_BLOCK_DURATION_MS=100
export KORVET_SERVER_STORAGE_NEAR_ARCHIVER_READ_WORKERS=1
export KORVET_SERVER_STORAGE_NEAR_ARCHIVER_STREAM_REFRESH_INTERVAL_MS=30000

# Cold Storage (optional) - enabled when path is set
export KORVET_SERVER_STORAGE_PATH=s3a://my-bucket/korvet/delta
export KORVET_SERVER_STORAGE_S3_REGION=us-east-1
export KORVET_SERVER_STORAGE_S3_ENDPOINT=http://localhost:4566  # Optional: for LocalStack/MinIO
export KORVET_SERVER_STORAGE_S3_CREDENTIALS_TYPE=iam  # or 'static'
export KORVET_SERVER_STORAGE_S3_CREDENTIALS_ROLE_ARN=arn:aws:iam::123456789012:role/KorvetReaderRole
# For static credentials:
export KORVET_SERVER_STORAGE_S3_CREDENTIALS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export KORVET_SERVER_STORAGE_S3_CREDENTIALS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# S3 Performance Tuning (optional)
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_MAX_CONNECTIONS=96  # Default: 96
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_MAX_THREADS=64  # Default: 64
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_READAHEAD_RANGE=256K  # Default: 256K
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_INPUT_POLICY=normal  # normal, sequential, random
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_FAST_UPLOAD=true  # Default: true
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_FAST_UPLOAD_BUFFER=disk  # disk, bytebuffer, array
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_FAST_UPLOAD_ACTIVE_BLOCKS=8  # Default: 8
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_BLOCK_SIZE=64M  # Default: 64M
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_MULTIPART_SIZE=64M  # Default: 64M
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_CONNECTION_TIMEOUT=15000  # ms, Default: 15000
export KORVET_SERVER_STORAGE_S3_PERFORMANCE_SOCKET_TIMEOUT=60000  # ms, Default: 60000

# Cold Tier Archiver Configuration (optional, used when storage path is set)
export KORVET_SERVER_STORAGE_ARCHIVER_CONSUMER_GROUP=korvet-archiver  # Default: korvet-archiver
export KORVET_SERVER_STORAGE_ARCHIVER_BATCH_SIZE=10000  # Default: 10000
export KORVET_SERVER_STORAGE_ARCHIVER_BLOCK_DURATION_MS=100  # Default: 100
export KORVET_SERVER_STORAGE_ARCHIVER_READ_WORKERS=1  # Default: 1
export KORVET_SERVER_STORAGE_ARCHIVER_COMMIT_WORKERS=4  # Default: 4
export KORVET_SERVER_STORAGE_ARCHIVER_MAX_BATCHES_PER_COMMIT=10  # Default: 10
export KORVET_SERVER_STORAGE_ARCHIVER_FILES_PER_COMMIT=10  # Default: 10
export KORVET_SERVER_STORAGE_ARCHIVER_STREAM_REFRESH_INTERVAL_MS=30000  # Default: 30000

# Logging
export LOGGING_LEVEL_COM_REDIS_KORVET=DEBUG

Next Steps