Publishing an event to Kafka after saving to the database looks simple. It has a subtle, dangerous flaw: if the Kafka publish fails after the DB commit, or the app crashes between the two, your event is lost and your data is inconsistent. The Transactional Outbox Pattern solves this by writing the event to the database in the same transaction as the business data, then publishing to Kafka separately. This guide covers the pattern, the implementation, and idempotent consumers.

REST APIs are synchronous — the caller waits for a response. Sometimes you don’t want that. An order creation shouldn’t wait for the inventory system, the notification system, and the analytics system to all respond before confirming to the user. Kafka decouples these concerns. What Kafka Is Kafka is a distributed event streaming platform. It stores events (messages) in an ordered, immutable log. Producers write to Kafka; consumers read from it.

This article implements Kafka producers and consumers with the full production setup — error handling, retries, dead-letter topics, and idempotent consumers. Producer: KafkaTemplate @Service @RequiredArgsConstructor @Slf4j public class OrderEventPublisher { private final KafkaTemplate<String, Object> kafkaTemplate; public void publishOrderCreated(Order order) { OrderCreatedEvent event = new OrderCreatedEvent( order.getId(), order.getCustomerId(), order.getCustomerEmail(), order.getItems().stream().map(OrderItemDto::from).toList(), order.getTotalAmount(), Instant.now() ); // Key = customerId: all events for same customer go to same partition kafkaTemplate.send("order-events", order.getCustomerId().toString(), event) .whenComplete((result, ex) -> { if (ex !

There is a fundamental problem with publishing Kafka events after a database commit: if the application crashes between the commit and the publish, the event is lost forever. The Transactional Outbox Pattern solves this. The Problem @Transactional public Order createOrder(CreateOrderRequest request) { Order order = orderRepository.save(buildOrder(request)); // DB committed ✓ kafkaTemplate.send("order-events", event); // ← Crash here → event lost, DB already committed // → Inventory never updated, customer never notified return order; } Two distributed systems (PostgreSQL and Kafka) can’t be in a single transaction.