← All news

Software

Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Buoyant has released Kinference, an Apache 2.0 inference control plane for vLLM on Kubernetes. It adds declarative routing policies, KV-cache-aware load balancing and per-tenant metrics, and cut median TTFT 95% in Buoyant’s own benchmark.

Read the full story on MarkTechPost.

Read at source (opens in a new tab)

Latest from MarkTechPost

Latest digest

Week 44 EN

Open the digest

Sources

69 active sources

24 Turkish · 45 English

All sources