ProjectX: Rebuilding a Modern Analytics Platform Without Stopping the Business

When Client X came to us, their analytics stack worked — but it had grown the way many successful products grow: fast, organically, and across too many moving parts. What started as a pragmatic setup had become hard to operate, hard to secure consistently, and expensive to evolve.

This is the story of how vByte Ltd (DevXo) redesigned that foundation into a production-grade platform — with clear cloud roles, GitOps delivery, managed data services, and a cutover measured in about one hour, not a weekend outage.

Where we started

The previous landscape looked familiar to anyone who has scaled analytics under delivery pressure:

• Many separate GitHub repositories, with uneven ownership and no single path from commit to production.
• Airbyte running on a small Kubernetes (K3s) footprint on a single EC2 instance — useful, but a fragile single point of failure.
• Almost every other service on its own EC2, packaged in Docker Compose / standalone containers: orchestration, BI, catalogs, warehouses, supporting APIs — each machine its own snowflake.
• Operational knowledge living in people’s heads and ad-hoc scripts, rather than in repeatable infrastructure and application definitions.
• Networking and access patterns that made “private by default” difficult to enforce uniformly.

It ran. Client X’s teams shipped. But every new environment, every security review, and every incident response cost more than it should. The platform needed an architecture that matched the ambition of the product.

How we split the platform (by design)

We did not put “everything in one account and hope.” We drew hard lines:

AWS — primary production

The commercial workload lives here:

• Amazon EKS as the single orchestration plane for analytics services (ingestion, orchestration, BI, lakehouse catalog, warehouse, and supporting apps).
• Managed PostgreSQL (RDS) for application metadata databases — Multi-AZ, automated backups, encryption in transit and at rest.
• Object storage (S3) for compute logs, lakehouse data, and backup-related workloads.
• Private networking for the control plane and data plane, with human access designed around a zero-trust overlay (Tailscale) rather than open public admin endpoints.
• IAM-centric access for people and workloads (including pod identity patterns), so credentials are not sprinkled across VMs.

GCP — disaster recovery

Recovery is not an afterthought bolted onto the same account:

• DR capacity and runbooks sit on GCP, deliberately decoupled from the AWS production blast radius.
• The point is simple: a regional or account-level incident on the primary cloud must not automatically take the recovery story with it.

Hetzner — platform control plane (GitLab & monitoring)

We moved “how we build and how we watch” off the production data path:

• Self-hosted GitLab (source of truth for application charts, CI, and container registry) on Hetzner — cost-efficient, under our operational control, independent of AWS quotas and console blast radius.
• Central monitoring stack (metrics, logs, alerting, dashboards) on a dedicated Hetzner host, reachable over the private mesh — so observability survives even when production is under stress, and so monitoring is not competing with analytics workloads for the same CPU and disks.

That three-way split — AWS for serving the product, GCP for DR, Hetzner for engineering systems — is one of the practices Client X valued most: clarity about where each risk lives.

Industry practices we actually implemented

Buzzwords only help if they show up in the daily workflow. For ProjectX we operationalized:

GitOps on Kubernetes

Applications are declared in Git and reconciled by Argo CD. Desired state is reviewable; drift is visible; promotions are intentional. Helm charts and values live next to the services they describe — not in a forgotten wiki page.

Infrastructure as code

Cloud resources are defined and applied with OpenTofu/Terraform-style workflows: VPC, EKS, RDS, IAM, storage, and edge pieces are reproducible. Environments stop being “the machine we SSHed into three years ago.”

Managed data plane for metadata

Application databases for orchestration, BI, and catalog metadata moved to RDS, with AWS Database Migration Service (DMS) used to perform an initial full load and then keep sources aligned through change data capture until cutover. That let us rehearse migration, shorten the final switch, and avoid a long freeze of Client X’s day-to-day work.

Private access by default

Operator UIs and SQL endpoints are published on a Tailscale fabric — HTTPS and database ports for humans without exposing admin surfaces to the public internet. Cluster APIs and private data stores stay inside the VPC; the mesh is how people reach them.

Autoscaling and right-sized workloads

Horizontal Pod Autoscaling for control-plane components, Cluster Autoscaler–aware scheduling for bursty job pods, and clear resource requests so the platform can grow with Client X’s pipelines instead of page-ing an engineer to launch another EC2.

Observability that pages the right people

Prometheus-compatible metrics, Grafana dashboards, and Alertmanager routes (including Slack/email) were wired for the warehouse and platform health — so memory pressure and query failures surface early, not when a stakeholder already sees a broken dashboard.

Least-privilege human access

Kubernetes RBAC and cloud access entries give engineers strong rights where they need them (application namespaces), without handing out cluster-admin to everyone who needs to debug a job.

The migration story (without the drama)

Modernization projects fail when cutover is improvisation. Ours was staged:

1. Build the new world in parallel — EKS, RDS, GitLab on Hetzner, monitoring, GitOps apps — while the old EC2/K3s estate continued to serve Client X.
2. Replicate state with DMS — metadata databases were brought onto RDS ahead of time, with continuous sync until we were ready to point applications at the new endpoints.
3. Move services chart by chart — BI, lakehouse catalog, orchestration, ingestion, and the warehouse stack landed on EKS under Argo CD, with secrets and identities handled as first-class platform concerns.
4. Cut over in a planned window of about one hour — DNS/mesh endpoints, application configs, and traffic paths switched together. Client X’s teams were briefed; rollback criteria were clear; the window held.

No multi-day “maintenance blackout.” No “we’ll see how Monday goes.” A deliberate, rehearsable switch.

What Client X runs on today

On the new platform, ProjectX consolidates the analytics value chain that used to be scattered across VMs:

One cluster. One delivery model. Clear homes for code, data, and recovery.

Outcomes — and a satisfied client

After go-live, Client X gained:

• Operational calm — fewer unique snowflake hosts, one Kubernetes vocabulary for deploys and incidents.
• Faster, safer change — merge → build → GitOps sync instead of SSH and hope.
• Stronger security defaults — private APIs, mesh-based human access, IAM-aligned workload identity.
• A credible DR story — production on AWS, recovery planning on GCP, tooling on Hetzner.
• Business continuity — the migration landed with roughly one hour of planned downtime, not a prolonged outage.

Client X’s stakeholders told us what matters most in plain language: the platform feels professional now — and we barely felt the move.

That is the standard we hold ourselves to at DevXo: ambitious engineering, boring cutovers, and clients who can get back to building product.

Why this matters beyond ProjectX

Scattered Docker hosts and solo K3s boxes are a rite of passage. They are not a destination.

If your analytics estate has outgrown “one EC2 per service,” the path forward is not a bigger VM — it is boundaries (which cloud owns what), automation (GitOps + IaC), managed state (RDS + disciplined migration with DMS), and respect for the business clock (cut over in an hour, not a quarter).

That is what we delivered for Client X on ProjectX.
That is what we deliver for teams who are ready for the same leap.

💡 Explore  DevXo

📞 Contact our team to see how we can help your business.