← All news

Software

AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU

AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances, with up to 48 concurrent invocations sharing one model per GPU, vLLM and SGLang base images that need no handler code, and 90-minute asynchronous timeouts.

Read the full story on InfoQ.

Read at source (opens in a new tab)

Latest from InfoQ

Software

How Google Traced GKE API Timeouts to a Linux Kernel TCP Bug

Google recently detailed how three interacting bugs across Google Kubernetes Engine (GKE), Private Service Connect, and the Linux kernel caused intermittent Kubernetes API connectivity failures, blocking around 800 developers from deploying to production. By Renato Losio

InfoQ