Pressure
Minutes-old answers on a seconds market — plus cost that grew faster than revenue.
SK Assist
· AISatyakabir Assistant
Satyakabir Technologies
Technology engineering · Digital transformation
Work · Case study
Real-time intelligence for a market that never pauses
2.3M
events / sec
−40%
decision latency
$12M
annual waste removed
7 mo
to production desk
Case study
We rebuilt a legacy financial analytics stack into a streaming lakehouse with an AI decision layer — processing 2.3M events per second, cutting decision latency 40%, and removing an estimated $12M in annual infrastructure waste.
Challenge
Traders needed answers in seconds; the legacy warehouse answered in minutes. Batch ETL, brittle dashboards, and opaque cost growth meant infrastructure spend outpaced revenue while operators lost trust in the numbers they traded on.
Solution
A principal-led platform squad replaced the batch core with a streaming lakehouse, added an evaluated AI decision layer behind policy gates, and shipped an operator console the desk actually uses — with FinOps and audit trails from day one.
How we framed the work
Minutes-old answers on a seconds market — plus cost that grew faster than revenue.
Streaming truth + evaluated AI, not another warehouse migration slide deck.
Weekly vertical slices behind flags; desk pilots before full cutover.
Latency, cost, and operator adoption measured in production — labeled honestly.
Engagement arc
5 stages · operator-owned handoffs
Frame
Desk pressure, latency SLOs, risk
Stream
Replace batch with lakehouse
Decide
AI layer with eval gates
Operate
Console, FinOps, on-call
Transfer
ADRs + runbooks to client SRE
Cadence
Latency SLOs, cost baseline, risk constraints, desk journeys.
Kafka + lakehouse foundation, schema contracts, replay drills.
Features, models, eval gates, console v1 under shadow traffic.
Flagged promotion, cost attribution, on-call transfer.
01 · Context
The estate had grown through acquisitions: multiple warehouses, nightly jobs, and dashboards that looked authoritative while serving stale aggregates. Traders compensated with spreadsheets. Platform cost rose without a clear owner for each dollar. Leadership needed a system that was fast, attributable, and explainable to risk and audit.
02 · Architecture
We collapsed batch paths into an event backbone with durable replay, contracted schemas, and a lakehouse that served both analytics and low-latency features. Models sat behind a router with offline regression suites and online monitors. Irreversible recommendations required human confirmation. Every decision left a lineage record suitable for desk and compliance review.
03 · Delivery
Discovery locked SLOs with trading and risk. We shadowed traffic for critical symbols, then promoted pods under feature flags. The operator console shipped early so the desk could reject designs that looked clever in demos but failed under live pressure. Cost attribution landed in the same waves as features — FinOps was not a later phase.
04 · Security & controls
Private connectivity, least-privilege IAM, encrypted at rest and in transit, and ACL-aware retrieval for any generative assist surfaces. Model providers were chosen only after residency and contractual review. Audit could reconstruct who saw what, when, and why a model suggested an action.
05 · Outcomes
Peak ingest hit 2.3M events per second. Median decision latency fell about 40% on the instrumented desk paths. FinOps attributed and retired idle clusters and duplicated pipelines — an estimated $12M annual waste removed. Operators stopped maintaining shadow spreadsheets for the promoted workflows.
06 · Transfer
We left ADRs for every major bet, runbooks for incident classes the desk actually sees, on-call pairing weeks, and an eval harness the client team extends. Satyakabir remains on retainer for hard problems — not as a black-box dependency for day-to-day operation.
01
Event streaming backbone with backpressure and replay
02
Feature + model serving path with offline/online evals
03
Trading-desk operator console with role-based views
04
Cost attribution and FinOps dashboards
05
Runbooks, ADRs, and on-call enablement pack
Kafka → lakehouse → feature store → model router
Policy gate + human-in-the-loop for irreversible actions
Private VPC endpoints; no raw desk data to public SaaS
Observability: traces, cost ceilings, decision lineage
From the engagement
Real-time intelligence for a market that never pauses
Stack & standards
“They rebuilt the path from event to decision without breaking the desk — and left us able to operate it.”
Continue exploring