GPU & LLM inference
Running models across H100 / GH200 fleets and multiple providers (Lambda, RunPod, Vast, Novita, Friendli, bare metal). Load balancing, health-based failover, autoscaling and zero-downtime model swaps.
I help AI teams run GPU inference, Kubernetes and cloud infrastructure that stays up, is easy to see into, and doesn't burn cash.
GPU bills climbing every month, inference falling over at peak, dashboards nobody trusts, and a team stuck firefighting instead of shipping. Sound familiar?
I start with where the money and the incidents are, then ship the fixes myself with before/after numbers.
Most teams see quick wins in the first weeks: idle GPUs turned off, over-provisioned clusters right-sized, noisy alerts gone.
I work embedded in your Slack and repos, own problems end-to-end, and leave everything in code (Terraform, Helm, GitOps) so your team owns it after.
Running models across H100 / GH200 fleets and multiple providers (Lambda, RunPod, Vast, Novita, Friendli, bare metal). Load balancing, health-based failover, autoscaling and zero-downtime model swaps.
Finding where the spend actually goes, right-sizing nodes and GPUs, Karpenter and spot, killing idle capacity, and choosing rented vs. reserved vs. managed endpoints with numbers.
Prometheus, Grafana, logs and tracing that people actually use. SLOs, alerts that mean something, GPU and inference metrics, and DORA metrics for the engineering org.
EKS clusters with Terraform, Argo CD and Helm. High availability done properly (anti-affinity, PDBs, blast radius), secrets, ingress and bringing off-grid components back under GitOps.
On-call rotations that are sustainable, incident process and postmortems, runbooks and automation, so outages get shorter and stop repeating.
Internal developer platforms, ephemeral environments and CI/CD, plus help hiring, structuring and leading a platform team.
A deep look at cost, reliability and observability. You get a prioritized list of fixes with the estimated impact of each, and the first quick wins already shipped.
Embedded in your team, shipping improvements continuously and owning the infra problems end-to-end.
A sounding board for your CTO or platform lead: architecture reviews, GPU provider choices, hiring and roadmap.
Platform AI Engineering Manager. Multi-provider GPU fleet for LLM and image inference, EKS + Argo CD, observability on Prometheus/Grafana, incident and on-call tooling.
Platform Engineering Manager for Observability & Messaging, serving marketplaces across Europe. Highly available Kubernetes clusters.
Senior Platform Engineer. Internal developer and machine learning platforms at Latin American scale.
30 minutes, no pitch deck. Tell me what's hurting and I'll tell you honestly whether I'm the right person.
Book a call →Or email me at hello@sebacaldarola.com