A high-throughput FinTech payment platform experienced severe cascading failures during real-world transaction spikes. Immediate symptoms included thread starvation, database connection pool exhaustion, rapidly increasing latency, and transaction timeouts. Forensic analysis showed that the underlying problem was not insufficient infrastructure. The concurrency model itself had become the bottleneck.
SWARCS redesigned the transaction write path around deterministic ownership rather than distributed lock arbitration. Transactions were keyed by account_id and routed through Kafka so that operations belonging to the same account were assigned to the same partition and processed in partition order by the consumer responsible for that partition. The redesigned flow combined partitioned event processing, in-memory state validation, and append-only persistence. This removed distributed lock operations from the transaction write path and changed the system from coordinating competing writers to routing related work through a deterministic processing path. The platform could therefore sustain substantially higher transaction volume without the cascading resource exhaustion observed in the original design.
The concurrency model changed from coordinating concurrent execution through distributed locking to making conflicting concurrent execution unnecessary for operations belonging to the same account. Measured results included p99 latency decreasing from 4,500 ms to 18 ms, stable throughput increasing from 3,200 TPS to 18,500+ TPS, transaction timeout rate decreasing from 8.5% to 0.0%, and database connection pressure decreasing from 100% to 26% under the tested target-load conditions.
The platform processes real-time FinTech payment transactions that affect account balances and ledger state across thousands of concurrent accounts. Its target workload includes transaction spikes reaching approximately 15,000 TPS.
The challenge was to sustain transaction processing under concurrency without allowing coordination overhead to undermine throughput, latency, or state consistency.
The legacy architecture used Redis through Redisson to enforce mutual exclusion when multiple application nodes attempted to update the same account state. As transaction concurrency increased, distributed lock acquisition and waiting became a bottleneck in the transaction path.
[Concurrent Transactions]
|
v
+-----------------------+
| Spring Boot Nodes |
| |
| Multiple JVM Threads |
+-----------+-----------+
|
v
+-----------------------+
| Redisson Distributed |
| Lock / Lease |
| |
| Contention |
+-----------+-----------+
|
v
+-----------------------+
| Account Update |
| |
| Database |
+-----------------------+
More traffic
|
v
More lock contention
|
v
More blocked workers
|
v
More connection pressure
|
v
Higher latency
|
v
More timeouts
|
v
More retries
|
v
More traffic
Client-side retries amplified the load on an already degraded processing path. Additional infrastructure capacity or different lock timeout settings could change when degradation occurred, but neither would remove distributed arbitration from the transaction path.
account_id
|
v
Deterministic Partition
|
v
Single Processing Path
|
v
Sequential Account State Transition
The key change is where concurrency is controlled. Instead of arbitrating ownership through distributed locks, the system routes transactions for the same account to the same Kafka partition and processes its records sequentially. Different partitions can progress concurrently, while unrelated accounts assigned to the same partition share its processing boundary.
+-------------------+
| Transaction |
+---------+---------+
|
v
+-------------------+
| Spring Boot Node |
+---------+---------+
|
v
+-------------------+
| Redis / Redisson |
| Distributed Lock |
+---------+---------+
|
+----+----+
| |
v v
WAIT EXECUTE
| |
+----+----+
|
v
Database Update
+-------------------+
| Incoming |
| Transaction |
+---------+---------+
|
v
+-------------------+
| Apache Kafka |
| |
| Key: account_id |
+---------+---------+
|
v
+-------------------+
| Kafka Partition |
| |
| Deterministic |
| Account Routing |
+---------+---------+
|
v
+-------------------+
| Single Processing |
| Path |
| |
| Sequential State |
| Transitions |
+---------+---------+
|
+------------------+
| |
v v
+-------------------+ +-------------------+
| Caffeine | | Append-Only |
| In-Memory State | | Event Persistence |
| & Validation | | |
+-------------------+ +-------------------+
|
v
+-----------------------+
| account_events |
| |
| Immutable Events |
+-----------------------+
Kafka records are keyed by account_id, routing records for the same account to the same partition under the configured partitioning scheme. Records are processed sequentially within a partition, preserving their partition order. A consumer group assigns a partition to one consumer instance at a time, although assignments can change during rebalancing. Parallel processing occurs across partitions, not between records sharing the same partition's processing boundary.
A Caffeine-backed in-memory layer keeps frequently accessed account state close to the processor, supporting transaction validation without repeatedly reconstructing active state through blocking database reads. Immutable state records and explicit enum-driven logic make state transitions explicit.
Persistence changed from competing updates to mutable account rows to an append-only account-event model.
The state transition and durable event write are covered by the same atomic database transaction. This establishes their atomicity within the database transaction; it does not imply atomic coordination with the in-memory cache or Kafka offset management.
The composite index supports account-scoped event lookup and access to ledger history by sequence number. The index alone does not establish how sequence numbers are assigned or guarantee a particular recovery procedure.
The redesign changed the concurrency model rather than merely replacing Redis with Kafka. Related transactions now follow a deterministic, ordered processing path, and durable transaction events are appended instead of competing updates being applied to the same mutable account row.
This places the processing boundary in the routing model and allows independent partitions to progress concurrently. The measured results demonstrate the improvement under the tested workload without implying that every possible timeout or operational failure has been eliminated.
99.6% reduction
~5.8× stable throughput
None observed under tested load
74% reduction
Removed
Across the measured workload, the redesign combined higher stable throughput with substantially lower tail latency and connection-pool utilization. The timeout result provides additional evidence of improved behavior under those conditions, while the removal of distributed lock operations confirms the underlying architectural change.
Deterministic routing changed how transaction concurrency was controlled: related work followed an ordered processing path without distributed lock operations on the write path. Under the tested workload, the measured results show improved throughput, tail latency, observed timeout behavior, and connection-pool utilization.
SWARCS performed architectural forensics on the transaction processing path, identified distributed lock contention as a central bottleneck, and redesigned processing around deterministic account routing. The work addressed the interaction between lock waiting, worker availability, database connection pressure, latency, and client retries rather than treating these symptoms independently.
STOP COMPETING FOR OWNERSHIP. MAKE OWNERSHIP DETERMINISTIC.
The central improvement was a change in concurrency control, not merely a change of infrastructure components. Deterministic routing established where related transactions are processed, while append-only persistence established how their durable events are recorded. The resulting architecture reduced contention in the transaction path while preserving the opportunity for parallel processing across partitions.