Services

Resources

Company

Modernising Observability for a Billion Dollar Revenue Gaming Platform

Modernising Observability for a Billion Dollar Revenue Gaming Platform

Modernising Observability for a Billion Dollar Revenue Gaming Platform

Context.

A large online gaming and betting operator ran 18 business critical Java services across more than 3,000 VMs. The platform processed over 180 million casino spins per day and handled traffic peaks of up to 600,000 requests per second. In a regulated real-money environment, observability had become part of the revenue-protection and operational-resilience model.

Monitoring had evolved over a decade: SCOM, VMware and Nutanix for infrastructure; AppDynamics and Prometheus for application performance; Logstash and Elasticsearch for logs; and several visualisation platforms. Each tool remained useful, but incidents still had to be assembled across disconnected timelines. The organisation had extensive telemetry without a single path from business impact to technical cause.

Problem Statement.

Revenue exposure from delayed detection: Business signals travelled through a multi-hop log pipeline before scheduled evaluation, leaving degradation hidden for 10 to 15 minutes. At more than 2,000 spins per second, over one million transactions could pass through the platform before a reliable signal surfaced

Revenue exposure from delayed detection: Business signals travelled through a multi-hop log pipeline before scheduled evaluation, leaving degradation hidden for 10 to 15 minutes. At more than 2,000 spins per second, over one million transactions could pass through the platform before a reliable signal surfaced

Revenue exposure from delayed detection: Business signals travelled through a multi-hop log pipeline before scheduled evaluation, leaving degradation hidden for 10 to 15 minutes. At more than 2,000 spins per second, over one million transactions could pass through the platform before a reliable signal surfaced

A critical five-year blind spot: All 18 services depended on a shared custom TCP library that had remained largely unchanged for five years because it sat at the heart of transaction processing. The existing APM could not trace through it, while modifying its binary protocol and asynchronous execution paths carried material production risk

A critical five-year blind spot: All 18 services depended on a shared custom TCP library that had remained largely unchanged for five years because it sat at the heart of transaction processing. The existing APM could not trace through it, while modifying its binary protocol and asynchronous execution paths carried material production risk

A critical five-year blind spot: All 18 services depended on a shared custom TCP library that had remained largely unchanged for five years because it sat at the heart of transaction processing. The existing APM could not trace through it, while modifying its binary protocol and asynchronous execution paths carried material production risk

A reactive and expensive operating model: Infrastructure, APM and log platforms alerted independently and mostly on lagging indicators. Teams could confirm impact, but struggled to see the conditions building towards it. At 3,000+ VMs, duplicate agents, repeated queries and per-host vendor dependence also limited operating and commercial leverage.

A reactive and expensive operating model: Infrastructure, APM and log platforms alerted independently and mostly on lagging indicators. Teams could confirm impact, but struggled to see the conditions building towards it. At 3,000+ VMs, duplicate agents, repeated queries and per-host vendor dependence also limited operating and commercial leverage.

A reactive and expensive operating model: Infrastructure, APM and log platforms alerted independently and mostly on lagging indicators. Teams could confirm impact, but struggled to see the conditions building towards it. At 3,000+ VMs, duplicate agents, repeated queries and per-host vendor dependence also limited operating and commercial leverage.

Outcome/Impact.

10x

10x

10x

Faster Detection

Faster Detection

Faster Detection

180M+

180M+

180M+

Daily Transactions

Daily Transactions

Daily Transactions

9,648

9,648

9,648

Elasticsearch queries per day eliminated

Elasticsearch queries per day eliminated

Elasticsearch queries per day eliminated

10x faster detection: Improved from 10 to 15 minutes to under 60 seconds, reducing the period in which peak-hour degradation could compound unnoticed.

10x faster detection: Improved from 10 to 15 minutes to under 60 seconds, reducing the period in which peak-hour degradation could compound unnoticed.

10x faster detection: Improved from 10 to 15 minutes to under 60 seconds, reducing the period in which peak-hour degradation could compound unnoticed.

10x faster detection: Improved from 10 to 15 minutes to under 60 seconds, reducing the period in which peak-hour degradation could compound unnoticed.

Complete transaction visibility: All 18 services became traceable through previously invisible custom TCP calls, creating one path from business symptom to technical cause.

Complete transaction visibility: All 18 services became traceable through previously invisible custom TCP calls, creating one path from business symptom to technical cause.

Complete transaction visibility: All 18 services became traceable through previously invisible custom TCP calls, creating one path from business symptom to technical cause.

Complete transaction visibility: All 18 services became traceable through previously invisible custom TCP calls, creating one path from business symptom to technical cause.

88% monitor modernisation: 59 of 67 legacy monitor patterns were suitable for streaming metrics, preserving coverage while improving timeliness

88% monitor modernisation: 59 of 67 legacy monitor patterns were suitable for streaming metrics, preserving coverage while improving timeliness

88% monitor modernisation: 59 of 67 legacy monitor patterns were suitable for streaming metrics, preserving coverage while improving timeliness

88% monitor modernisation: 59 of 67 legacy monitor patterns were suitable for streaming metrics, preserving coverage while improving timeliness

Lower platform overhead: Eliminated 9,648 Elasticsearch queries per day and removed approximately 2 GB per service per day from the alerting path.

Lower platform overhead: Eliminated 9,648 Elasticsearch queries per day and removed approximately 2 GB per service per day from the alerting path.

Lower platform overhead: Eliminated 9,648 Elasticsearch queries per day and removed approximately 2 GB per service per day from the alerting path.

Lower platform overhead: Eliminated 9,648 Elasticsearch queries per day and removed approximately 2 GB per service per day from the alerting path.

Faster, safer legacy modernisation: AI shortened service-level planning from days to hours, while a phased rollout safely modernised a five-year-stable shared dependency without a fleet-wide cutover.

Faster, safer legacy modernisation: AI shortened service-level planning from days to hours, while a phased rollout safely modernised a five-year-stable shared dependency without a fleet-wide cutover.

Faster, safer legacy modernisation: AI shortened service-level planning from days to hours, while a phased rollout safely modernised a five-year-stable shared dependency without a fleet-wide cutover.

Faster, safer legacy modernisation: AI shortened service-level planning from days to hours, while a phased rollout safely modernised a five-year-stable shared dependency without a fleet-wide cutover.

Solution.

We framed the programme as controlled modernisation, not a tool replacement. The existing stack stayed active, each service was validated independently, and rollback remained available throughout. This protected business continuity while allowing value to be proven progressively.

  • Used AI to reduce uncertainty around the critical dependency: AI-assisted analysis mapped the TCP library's Netty flow, serialisation boundaries, metadata, thread hand-offs and error paths. Engineers used that evidence to select safe propagation points, verify compatibility and benchmark the change on Java 8 before rollout.

  • Turned one intervention into an estate-wide capability: OpenTelemetry client and server spans were added to the shared library with W3C trace context. One controlled framework change unlocked end-to-end tracing across all 18 services without redesigning each application.

  • Moved business detection closer to transaction time: Four bounded streaming metric families for spins, bets, wins and revenue replaced poll-based detection. Detailed logs remained available for audit and investigation, while latency drift, retry growth, queue pressure and spin-rate deceleration provided earlier warning of business impact.

  • Created fleet-wide control and portability: The OpenTelemetry Java Agent and local Collector were deployed across 3,000+ VMs. Batching, retry and export were kept outside the application process, and an open instrumentation standard reduced future switching cost.

  • Scaled the migration with AI assistance: AI mapped 67 monitor definitions across repositories to services, methods and candidate instrumentation points, then drafted configurations and documentation. Repetitive planning moved from days to hours; engineers retained accountability for semantics, performance and production approval.

Result.

The operator moved from fragmented, reactive monitoring to an observability capability aligned with business continuity. The value extended beyond better telemetry: earlier protection of peak-hour revenue, faster operational decisions, lower platform waste and a safer path to modernise a legacy estate. The five-year-stable library was not rewritten; it was understood, instrumented and converted into a platform-wide control point. AI accelerated discovery, while disciplined engineering kept the change safe.

Share
Share

Blogs.