All projects
Data & ML pipelines
Reconciliation that closes in minutes instead of days
An event-driven reconciliation pipeline matching ledger, processor and bank records, with every unmatched penny surfaced the same day rather than at month end.
- Client
- Cross-border payments provider
- Industry
- Fintech
- Service
- Data & ML pipelines
- Year
- 2025
The challenge
Three systems disagreed about what had been paid, and the difference was found by a finance team exporting CSVs and sorting them by hand. Month-end close took days, breaks were discovered long after the counterparty could be asked about them, and nobody could say with confidence what the real exposure was.
What we did
- Streamed ledger, processor and bank events into one normalised transaction record keyed on a stable identifier
- Made matching idempotent and replayable, so a late or duplicated file corrects the picture instead of corrupting it
- Classified breaks by cause — timing, fees, FX, genuinely missing — so the finance team triaged rather than searched
- Gave every number a drill-through to the source events that produced it
- Alerted on unmatched value crossing a threshold rather than waiting for the close
Outcome
Days to under an hour
Close
Month-end to same day
Break discovery
Every figure traceable to source
Audit
- Python
- Kafka
- Airflow
- dbt
- PostgreSQL
- BigQuery
- Kubernetes