Loading...
Loading...
Building AI infrastructure tools in the open. These projects aim to solve real problems in the ML/AI platform space and are designed for potential CNCF contribution.
Multi-cloud MLOps reference platform on AWS EKS, Azure AKS, and GCP GKE with defense-in-depth security and full-stack observability. Designed around a 15-minute self-service path from experiment to production; end-to-end verification, LLM-serving benchmarks on real GPUs, and GitOps rollout are the current focus.
Currently focused on shipping Kortex, AI FinOps Platform, and MLOps Platform to production-grade quality.
Upstream contributions to CNCF projects (KServe, Karpenter, OpenCost) are planned as these projects mature and generate issues worth contributing back.
Complete MLOps platform showing how all the pieces fit together.
Kubernetes-native AI inference gateway built from scratch to explore the multi-model routing problem space: A/B testing, intelligent failover, circuit breakers with exponential backoff, OpenTelemetry tracing, and cost/latency/context-length-aware routing. The ecosystem has since standardized this layer (Gateway API Inference Extension, llm-d) — the learnings from building it independently now feed my upstream contributions there.
Reference implementation of a cost-optimization platform for AI/ML workloads: GPU utilization monitoring, budget forecasting with alerts, ML-based anomaly detection, automated right-sizing recommendations, and multi-cloud billing integration. Code-complete across all 3 phases; validation against live GPU clusters is the next milestone.
Check out my GitHub profile for more projects and contributions.