https://starptech.com/blog Notes on infrastructure, shipping software, and startups. en Fri, 29 May 2026 13:41:26 GMT https://starptech.com/blog/why-more-gpus-is-not-enough-for-llm-inference/ https://starptech.com/blog/why-more-gpus-is-not-enough-for-llm-inference/ What I learned deploying and tuning large-model inference: KV cache, routing, and cache hierarchy matter as much as raw GPU count. AI infrastructure Thu, 28 May 2026 00:00:00 GMT