Fast, Customized AI Chips Powering the Next Era of Computing

Artificial Intelligence (AI) promises to transform major industries through self-driving vehicles, automated business processes, and scientific discoveries guided by machine learning algorithms. Underlying these innovations are specialized computer chips engineered specifically for AI‘s intense computational demands.

In this comprehensive technology overview, we will decode the most exciting new AI chip developments – from incumbents like NVIDIA to hungry startups like Cerebras – what makes them uniquely fast learners and how they might impact products you use daily in the years ahead.

Why AI Needs Custom Hardware

First, what exactly is "AI" in the context of computer chips? Popular applications like Siri or Spotify recommendations rely on machine learning, a branch of AI focused on detecting hard-to-discern patterns across massive datasets. This could involve analyzing acoustic properties that characterize hit songs, speech patterns to transcribe videos, or subtle visual cues identifying skin cancers.

Machine learning typically uses model architectures called artificial neural networks loosely inspired by the interconnected neurons in animal brains. Much like we internally develop intuition through experiences over time, these networks also repeatedly adjust internal parameters when exposed to more labelled examples during training.

However, unlike our streamlined biological hardware, machine learning requires crunching through millions of examples and complex mathematical operations running simultaneously across thousands of processors to build its reasoning abilities. This computationally-intensive rapid iteration process underlies everything from better conversational assistants to early disease detection capabilities.

Specialized hardware offers 5-10X speedups versus repurposing stock server CPUs or even gaming GPUs for these unique machine learning workloads by optimizing for critical performance differences:

With Moore‘s Law ending, domain-specific architectures now provide the most direct path to improving price/performance for AI workloads from cloud to edge devices. Software customizability will define next-gen chips targeting different operational constraints from data centers to autonomous cars.

Let‘s overview bleeding-edge offerings pushing machine learning capabilities to unprecedented scales.

Giant AI Accelerators Tackling Industrial-Size Problems

Two figures dominate the booming data center market for companies harnessing machine learning across massive datasets – NVIDIA and start-up Cerebras Systems:

NVIDIA A100 Tensor Core GPU

NVIDIA pioneered using its graphics processing units (GPUs) for massively-parallel floating point operations ideal for AI workloads. The A100 consolidates its past innovations into an AI-specific giant:

• Packs 54 billion transistors enabling over 1000 trillion operations per second through parallelism
• Provides up to 80 GB high-bandwidth memory accessible by all cores
• Third-generation Tensor Cores designed specifically for accelerated matrix math
• Multi-instance GPU allowing up to 7 separate workloads/models to run concurrently
• Scale up to thousands of GPUs networked through NVIDIA‘s dedicated interconnect fabric and software

With thousands of A100s now powering AI clouds like Amazon EC2, Microsoft Azure, and Oracle Cloud Infrastructure, industrial researchers are achieving state-of-the-art results across climate forecasting, drug discovery, and natural language understanding. The latest "Megatron-Turing NLG 530B" – a 530 billion parameter language model built using A100 clusters – produces incredibly fluent conversational text that can pass many parts of the Turing test.

Source: NVIDIA A100 Product Page

Cerebras Wafer Scale Engine 2 (WSE-2)

While NVIDIA clusters multiple A100 GPUs networked together to act as a giant AI supercomputer, startup Cerebras Systems takes an unprecedented "go big or go home" approach by packing 850,000 cores across a massive 46,000 square mm silicon wafer – the largest computer chip ever produced. This Wafer Scale Engine sets incredible records:

• Contains 2.6 trillion transistors – 20X more than NVIDIA‘s flagship GPU
• Provides 2 petabytes per second memory bandwidth from on-wafer SRAM – no external memory needed!
• Enables customer workloads up to 20X faster runtimes depending on model sparsity optimizations compatible with its architecture
• cooler taller than a referee keeps the entire monolithic chip operating at a brisk 15 kilowatts!

While literal factory-scale fabrication constraints limit production volumes, Cerebras showcases just how far customization can push optimized silicon designs dedicated for AI alone. Its $500M funding including strategic manufacturing partnerships suggests the Wafer Scale approach warrants ongoing investment despite immense technical hurdles.

Sources: Cerebras Systems Homepage, WSE-2 Launch Press Release

While likely impractical for smaller organizations, these leviathan processors showcase manufacturing feats that expand perceived technical limits for specialized AI hardware. Their rapid development cycles reflect intense interest in such heavy accelerators across cloud infrastructure providers and well-funded research labs to stay competitive. Just a few racks full of these chips can train models with trillions of parameters in days instead of weeks.

But custom AI infrastructure requires major resource investments – what about embedding intelligent features more seamlessly into consumer devices at the edge?

AI Acceleration Gets Closer to Users

Delivering advanced neural network capabilities directly on end-user devices rather than round-tripping to distant cloud data centers confers key benefits:

• Enables responsive real-time experiences – no lags for speech transcription or AR effects
• Avoids privacy risks from continuously streaming sensitive user data externally
• Provides fault tolerance when no reliable network link for mission-critical systems like autonomous cars
• Unlocks emerging user context with access to raw on-device sensor data before external processing

Let‘s evaluate notable chips purpose-built to accelerate on-device AI experiences closer to end users:

Google Tensor Smartphone Processor

Historically, smartphones have relied on separate compute, graphics, and display-focused chips – great for versatility but lacking specialization. With Tensor, Google finally unveiled its first mobile processor tailored specifically for AI workloads it expects will define modern phone experiences:

• Google‘s proprietary AI acceleration (TPU) cores integrated into mobile SoC design
• Focused on real-time latency-sensitive workloads like speech recognition and photography
• Twice the AI performance at lower power budgets than prior mobile chipsets
• Enables phone exclusive features like real-time language translation and enhanced autofocus

By controlling its own chip design from the ground-up, Google can evolve phone capabilities most meaningful to users in coming years – less about vanity benchmark metrics but instead more assistive, intuitive software directly touching daily experiences.

Intel Loihi 2 Neuromorphic Processor

Rather than simply optimizing current code-based AI models to run faster on mobile hardware, Intel‘s research division pioneers completely different computing paradigms inspired by the inner workings of animal brains for smarter embedded systems:

The Loihi 2 neuromorphic processor uses spiking neural networks mimicking our neurons‘ pulse-coded signaling protocols:

• Networks trained with spike timing and bursts encode information differently from mainline AI models
• Event-driven operation allows sophisticated learned behaviors while using 100-1000X less power
• Significantly enhanced adaptation, generalization capabilities closer to biological systems
• Ideal for small form-factor robots, drones, and internet-of-thing sensory devices

This bio-inspired computing niche focuses more on replicating in-the-field learning rather than brute number crunching. By investigating radically unconventional architectures, projects like Loihi accelerate discoveries outside industry‘s current technical defaults.

Architecting the Future of AI Chips

While chips like Google‘s Tensor and Cerebras‘ giant wafer push machine learning capabilities tremendously over conventional hardware, they still rely on handcrafted configurations. The next frontier focuses on software-defined architectures that are highly customizable to different applications post-deployment without sacrificing efficiency.

Groq Architecture – Unified, Programmable Logic

Startup Groq argues that virtually all modern processor designs – even GPUs and AI accelerators – retain inherent bottlenecks from fixed divisions of labor across physically separate sub-components like CPU cores, memory clusters, and interconnects shufflings data between them.

Their architecture consolidates all key logic into a unified software-programmable array instead, directly accessing on-die unified memory to greatly simplify data movement:

This streamlined foundation paired with intelligent caching for predictable latency allows customized dataflow optimize different neural network topologies, data types and precisions for up to 10X runtime improvements. Such software-defined configurability will become increasingly vital as researchers rapidly evolve network architectures.

By questioning status quo assumptions, Groq represents the innovative thinking hardware startups can pioneer before capabilities later scale across broader applications.

The Future Is Custom & Configurable

From mammoth 85-inch wafer prototypes to smartphone-sized embedded processors inspired by biology, hardware propelling AI‘s current wave of adoption only hints at technology transformations still to come:

• Conversational assistants like Siri answering queries with encyclopedia-scale knowledge and nuanced wit derived from thousand-petaflop scale private cluster training
• Grade-school level reading comprehension and reasoning tutors accessible freely to expand youth education opportunities
• Autonomous vehicle networks and augmented workforce collaboration from redesigned occupations and mobility patterns
• Continuous discovery of more nutritious, sustainable nutrition balancing environmental resources amidst shifting demographics
• Differentiated preventative healthcare prioritizing wellbeing milestones customized across patient genetics and lifestyles

These examples showcase AI‘s future societal benefits enabled by specialized hardware. While still early, unrelenting exponential progress in AI compute from architecture innovations, advanced manufacturing, and supply chain investments clearly indicate we‘ve only begun unlocking technology‘s positive potential improving lives when thoughtfully, ethically directed.

What hardware breakthroughs excite you most in years ahead? Share your perspectives on the economic, policy, and ethics considerations influencing AI‘s global emergence.