Introduction
The Redis Connector for Spark provides integration between Redis and Apache Spark and supports reading data from and writing data to Redis. Within a Spark 3 environment the connector enables users to read data from Redis, manipulate it using Spark operations, and then write results back to Redis or to another system. Data can also be imported to Redis by reading it from any data source supported by Spark and then writing it to Redis.
The connector has the following system requirements:
-
Spark 3.5.4 is recommended but versions 3.1 to 3.5 should work as well.
-
Scala 2.12
-
When choosing a Spark distribution you must select one that uses Scala 2.12.
-
Redis version 5 or higher.
-
Java 17 or higher (if using Java to run Spark).
