Why the AI era is forcing a redesign of the entire compute backbone
1 min read
Summary
The computing industry needs to undergo a revolution to fulfil the promise of artificial intelligence (AI), according to Amin Vahdat, vice president and general manager for machine learning, systems and cloud AI at Google Cloud.
To achieve this, the industry needs to innovate and collectively rethink the entire technology stack, creating a new blueprint for global infrastructure to deliver new capabilities.
The shift away from scale-out commodity hardware will see a proliferation of domain-specific compute units, including application-specific integrated circuits, GPUs and tensor processing units, that are optimised for narrower tasks.
To scale AI workloads, the use of specialised interconnects such as ICI for TPUs and NVLink for GPUs, will rise as such networks can bypass the overhead of layered networking stacks by using direct memory-to-memory transfers.
Machine learning (ML) models often rely on calculations across tens to hundreds of thousands of identical compute elements, consuming immense power, Vahdat said, adding this is driving the need for high-density systems that minimise physical distance between processors to reduce latency and power consumption.