+91 98726 60544 hello@mitstech.co Mon–Sat · 09:00–18:30 IST

Event-driven architecture: the pitfalls nobody mentions

IT Strategy By Mits Engineering Team 2 min read
Event-driven architecture: the pitfalls nobody mentions

Events decouple producers from consumers, and that is a genuine benefit: a service can publish that something happened without knowing or caring who reacts. The cost, which is rarely discussed with the same enthusiasm, is that no single place in the system describes what happens when an order is placed. The behaviour is distributed across every consumer, and understanding it means reading all of them.

The first practical problem is delivery semantics. Most brokers guarantee at-least-once delivery, which means duplicates are not an edge case but a normal occurrence. Every consumer therefore has to be idempotent - processing the same event twice must produce the same result as processing it once. In practice that means each event carries an identifier and each consumer records what it has already handled. Teams that treat duplicates as unlikely rather than certain discover the gap through duplicate charges.

Ordering is the second. Most systems guarantee order only within a partition, so if related events are spread across partitions they can be processed out of sequence - an update arriving before the create it depends on. The fix is to partition by the entity identifier so everything about one order lands in one partition, and the discipline is to notice this before rather than after.

Then there is the outbox problem, which is subtle and bites everyone once. A service writes to its database and publishes an event. Those are two systems and there is no shared transaction, so a crash between them leaves the database updated and the event unpublished, forever. The reliable pattern is to write the event into an outbox table in the same transaction as the data, and have a separate process publish from that table. Without it your systems drift apart slowly and the reconciliation is manual.

Schema evolution is where event systems age badly. An event published today may be consumed by code deployed two years from now, and by replay tooling reading history. Adding fields is safe; removing or renaming them breaks consumers you may not know exist. A schema registry with compatibility checks in the pipeline is the mechanism that stops this, and it is much easier to adopt at the start than to retrofit across a hundred event types.

Finally, invest in the ability to answer a simple question: what happened to order 12345? In a request-response system you read a log. In an event system the answer is spread across producers, brokers and consumers, and without correlation identifiers propagated through every hop and a tool that assembles them, an ordinary customer support query becomes an afternoon of archaeology. Build that before you need it, because you will need it during an incident rather than at leisure.

Need help with this? Explore our Software Development services. Learn more Back to all news

Keep reading

More on IT Strategy