Skip to content

Sign in See it live

Engineering

Inference, infrastructure, and making it fast enough to matter.

2 posts

Feb 2026

Structured Output Isn't Reliable Output

JSON mode, function calling and constrained decoding give you schema compliance, not semantic reliability. Valid JSON can be completely wrong.
Jun 2025

From 11% to 88% GPU Utilization: How We Built 8x Faster LLM Inference

PyTorch leaves 89% of GPU bandwidth on the table. We fixed it with custom Triton kernels. Here's what we learned building Accelerate.

Other topics