Google has announced two new eighth-generation AI accelerator systems, the TPU 8tx and TPU 8i Max, marking the first time the company has shipped four TPU variants within a single year. Built on existing silicon, the new configurations use modular packaging, networking, and memory setups to address specific enterprise workloads rather than introducing a new chip generation.
The TPU 8tx focuses on confidential computing by maintaining data and model weight encryption during active processing, matching a capability Nvidia offers in its GPUs. Meanwhile, the TPU 8i Max pairs two TPU 8i chips on one package to double compute, memory, and memory bandwidth in the same rack format, aiming to alleviate memory bottlenecks during large-scale agentic inference.
The releases highlight Google’s strategic transition from using TPUs internally to marketing them as a commercial cloud platform, with the company reporting nearly 100 external customers. Alongside the hardware, Google open-sourced TorchTPU and added support for frameworks like vLLM and SGLang to ease integration for PyTorch developers.
Why it matters
Provides alternative high-memory and confidential hardware options for startups and enterprises running continuous, memory-intensive inference.
TorchTPU open-sourcing simplifies porting PyTorch code bases to Google Cloud TPUs without major software refactoring.
Signals a broader cloud provider trend to offer proprietary silicon tailored specifically for lower-cost, high-throughput inference.
Source: the-decoder.com



