Kafka Schema Registry

Kafka Schema Registry is a service that stores and enforces schemas for messages on Kafka topics. Without it, any producer can send any message format and consumers silently break when the format changes. With Schema Registry, every message is validated against a registered schema before it is written, and consumers always look up the exact schema used to encode a given message. This makes schema evolution safe and controlled.

Schema Registry works with Avro, Protobuf, or JSON Schema. Producers register a schema once and include only a 4-byte schema ID in each message instead of the full schema - keeping messages compact. Consumers use the schema ID to fetch the schema from the registry and deserialize correctly. The registry enforces compatibility rules: BACKWARD (new schema can read old messages), FORWARD (old schema can read new messages), or FULL (both). This prevents accidental breaking changes from reaching production consumers.

The below example shows registering an Avro schema and producing messages with schema validation using the Confluent Kafka Java client.

It gives the following output,

{"id": 1}

["orders-value"]

{
  "subject": "orders-value",
  "version": 1,
  "id": 1,
  "schema": "{"type":"record","name":"Order",...}"
}

It gives the following output,

Order sent with schema validation

# If you try to send a record missing a required field:
# org.apache.kafka.common.errors.SerializationException:
# Error registering Avro schema - field "status" is required

# The message never reaches the Kafka topic - caught at produce time

Schema evolution tip: to safely add a new field, always provide a default value in the Avro schema - for example {"name":"discount","type":"double","default":0.0}. This satisfies BACKWARD compatibility because old consumers reading new messages simply use the default. Adding a field without a default breaks BACKWARD compatibility and Schema Registry rejects the new schema version.