Logging

Korvet uses Logback for logging with JSON output for easy parsing.

Log Format

Logs are output in JSON format by default:

{
  "timestamp": "2024-01-15T10:30:45.123Z",
  "level": "INFO",
  "logger": "com.redis.korvet.server.KorvetServer",
  "message": "Started KorvetServer in 2.5 seconds",
  "thread": "main"
}

Log Levels

Configure log levels via environment variables:

# Root level
export LOGGING_LEVEL_ROOT=INFO

# Korvet components
export LOGGING_LEVEL_COM_REDIS_KORVET=DEBUG

# Spring framework
export LOGGING_LEVEL_ORG_SPRINGFRAMEWORK=WARN

# Redis client
export LOGGING_LEVEL_IO_LETTUCE=INFO

Or in application.yml:

logging:
  level:
    root: INFO
    com.redis.korvet: DEBUG
    org.springframework: WARN
    io.lettuce: INFO

Structured Logging

Korvet includes structured fields in logs:

  • topic: Topic name

  • partition: Partition number

  • offset: Message offset

  • clientId: Kafka client ID

  • correlationId: Request correlation ID

Log Aggregation

Filebeat

Ship logs to Elasticsearch:

filebeat.inputs:
- type: container
  paths:
    - '/var/lib/docker/containers/*/*.log'
  processors:
    - decode_json_fields:
        fields: ["message"]
        target: ""

output.elasticsearch:
  hosts: ["elasticsearch:9200"]

Fluentd

<source>
  @type tail
  path /var/log/korvet/*.log
  pos_file /var/log/korvet/korvet.log.pos
  tag korvet
  <parse>
    @type json
  </parse>
</source>

<match korvet>
  @type elasticsearch
  host elasticsearch
  port 9200
  index_name korvet
</match>

Troubleshooting Logs

Enable debug logging for troubleshooting:

export LOGGING_LEVEL_COM_REDIS_KORVET_KAFKA=DEBUG
export LOGGING_LEVEL_COM_REDIS_KORVET_REDIS=DEBUG

Next Steps