Rahul Anand Sharma

I am an ML Systems Engineer and Researcher at Google Cloud, working at the intersection of LLM serving architecture, compiler runtimes, and distributed accelerator infrastructure. My work focuses on co-designing serving stacks and multi-node accelerator systems to push the latency–throughput Pareto frontier of frontier models across TPUs and GPUs.

At Google Cloud, I lead inference optimization initiatives for Gemini across GPU and TPU fleets—including architecting intra-node disaggregated serving for air-gapped Sovereign Cloud deployments on Google Distributed Cloud (GDC)—and drove multi-cube interconnect scaling, collective communication microbenchmarking, and Day-1 model enablement for TPU v7 (Ironwood) (Google Cloud Tech Impact Award, 2026).

Prior to Google, I was an Applied Scientist at AWS, where I architected Generative AI root-cause analysis for Apache Spark at hyperscale (Amazon Q) and active verification systems for cloud network security. I earned my Ph.D. in Electrical & Computer Engineering from Carnegie Mellon University (advised by Vyas Sekar and Anthony Rowe, CIT Dean’s Fellow) and previously built distributed edge-AI and IoT systems at Microsoft Research (FarmBeats, DeepEdge). My research spans 15+ publications at venues including SIGCOMM, NSDI, and USENIX Security, and 2 US patents.

---

Production Systems & Technical Scope