Event Driven Architecture: Building Reactive Systems
Event driven architecture decouples system components by communicating through asynchronous events rather than direct synchronous calls. Therefore, services can evolve independently while maintaining system-wide consistency through eventual consistency patterns. As a result, organizations build more resilient and scalable distributed systems. Rather than a service blocking on a downstream HTTP call and failing when that dependency is unavailable, a producer emits a fact about something that happened and moves on; consumers process that fact on their own schedule. Consequently, a slow or temporarily offline consumer no longer drags down the producer, and the system as a whole degrades gracefully instead of cascading into failure.
Core Patterns and Concepts
Event-driven systems use three primary patterns: event notification, event-carried state transfer, and event sourcing. Moreover, each pattern addresses different integration challenges and consistency requirements. Consequently, architects often combine multiple patterns within a single system based on domain needs. Event notification keeps payloads thin, emitting only an identifier and event type, which means consumers must call back to the source for details; this minimizes coupling but adds query load. Event-carried state transfer instead packs the full relevant state into the event, so consumers maintain local replicas and avoid the callback entirely, at the cost of larger messages and data duplication.
Event notification provides lightweight decoupling where producers emit events without concern for consumers. Furthermore, consumers independently decide how to react to events, enabling extensibility without modifying existing services. Therefore, adding a new analytics or audit consumer becomes a deploy rather than a change request against the producing team, which is the practical agility benefit teams cite most often.
Event Driven Architecture with Apache Kafka
Kafka serves as a durable event log that enables replay, time-travel debugging, and stream processing. Additionally, topic partitioning provides horizontal scalability while maintaining ordering within partition keys. For example, all events for a specific customer route to the same partition, ensuring ordered processing. Because Kafka retains events for a configurable window rather than deleting them on consumption, a newly deployed consumer can rewind to the beginning of a topic and rebuild its state from history, which is impossible with a traditional fire-and-forget message queue.
// Kafka event producer with schema registry
@Service
public class OrderEventProducer {
private final KafkaTemplate<String, OrderEvent> kafka;
public void publishOrderCreated(Order order) {
OrderEvent event = OrderEvent.builder()
.eventId(UUID.randomUUID().toString())
.eventType("ORDER_CREATED")
.timestamp(Instant.now())
.orderId(order.getId())
.customerId(order.getCustomerId())
.items(order.getItems())
.totalAmount(order.getTotal())
.build();
kafka.send("orders.events", order.getCustomerId(), event)
.whenComplete((result, ex) -> {
if (ex != null) log.error("Event publish failed", ex);
else log.info("Event published: offset={}", result.getRecordMetadata().offset());
});
}
}Schema evolution with a registry ensures backward compatibility as event structures change over time. Therefore, consumers can process events from different schema versions without breaking. In practice, registries enforce a compatibility mode such as backward or full, which rejects a producer attempting to publish a schema that would break existing consumers. As a guideline, add fields with defaults rather than removing or renaming them, and you preserve compatibility indefinitely.
The Outbox Pattern and Dual-Write Problem
A subtle failure mode haunts naive event-based systems: the dual-write problem. When a service writes to its database and then publishes an event as two separate operations, a crash between the two leaves the system inconsistent, since the order exists but no event was emitted. The transactional outbox pattern solves this by writing the event into an outbox table within the same database transaction as the business data. A separate relay process, often Debezium reading the database change log, then publishes those rows to Kafka and marks them sent. Because the business write and the outbox write commit atomically, you never lose an event, and the relay guarantees at-least-once delivery to the broker.
@Transactional
public void placeOrder(Order order) {
orderRepository.save(order); // business state
outboxRepository.save(OutboxEvent.builder() // same transaction
.aggregateType("Order")
.aggregateId(order.getId())
.eventType("ORDER_CREATED")
.payload(serialize(order))
.createdAt(Instant.now())
.build());
// A Debezium connector tails the outbox table and publishes to Kafka.
}Event Sourcing and CQRS
Event sourcing stores state changes as an immutable sequence of events rather than current state snapshots. However, querying event-sourced systems requires projections that materialize read-optimized views. In contrast to CRUD systems, event sourcing provides complete audit trails and enables temporal queries, letting you reconstruct the exact state of an aggregate at any past moment. CQRS, command query responsibility segregation, complements this by separating the write model that validates and appends events from one or more read models tuned for specific queries. Therefore, you can serve a dashboard from a denormalized projection while the canonical truth remains the event log. The trade-off is real complexity: you must handle eventual consistency between write and read sides, version your events forever, and often introduce snapshots so rebuilding an aggregate does not require replaying millions of events. For this reason, reserve full event sourcing for domains where auditability and temporal analysis genuinely justify the cost.
Error Handling and Dead Letter Queues
Failed event processing requires retry strategies with exponential backoff and dead letter queues. Additionally, idempotent consumers prevent duplicate processing when events are redelivered. Specifically, deduplication keys based on event IDs ensure exactly-once processing semantics in practice. Because most brokers guarantee at-least-once delivery, a consumer will occasionally see the same event twice after a rebalance or redelivery; therefore, the consumer must make processing idempotent by recording each handled eventId and skipping repeats. When an event fails repeatedly, blindly retrying it forever blocks the partition behind it, so a bounded retry followed by routing the poison message to a dead letter topic keeps the main stream flowing while preserving the failure for investigation.
When Not to Use This Style
For all its strengths, this style is not a default. Asynchronous flows are genuinely harder to reason about, debug, and test, because a single user action fans out into many eventually-consistent steps with no single stack trace to follow. Therefore, distributed tracing across the event chain becomes mandatory rather than optional. Where you need an immediate, strongly consistent answer, such as a synchronous payment authorization that must succeed before a response returns, a direct request-response call is simpler and safer. Small systems with a handful of services rarely earn back the operational overhead of a broker, schema registry, and dead letter handling. As a rule of thumb, adopt events when you have multiple independent consumers, genuine scaling or resilience requirements, or an audit mandate, and prefer synchronous calls when the interaction is inherently transactional and confined to two parties.
Related Reading:
Further Resources:
In conclusion, event driven architecture enables building resilient distributed systems that scale independently and evolve gracefully. Therefore, adopt event-driven patterns to achieve loose coupling and high throughput in your microservices ecosystem, while investing early in idempotency, the outbox pattern, and observability to manage the complexity they introduce.