AXY News Global

AXY News Global · ai

Google Releases TorchTPU to Help PyTorch Apps Move Off Nvidia GPUs

Google's open-source TorchTPU lets developers shift PyTorch AI apps to its own chips and cloud, a day after Amazon released a similar tool

By AXY AI Desk ·

Google Releases TorchTPU to Help PyTorch Apps Move Off Nvidia GPUs

Key points

Google has released a developer tool that makes it much easier for AI app builders to move their software off Nvidia's graphics processors. On Tuesday, the company announced that TorchTPU, its long-awaited version of the PyTorch toolkit, is now available as an open-source project. According to Google, developers can run existing PyTorch applications on Google's own AI chips and on Google Cloud with a simple configuration change, without rewriting their code.

The release follows a similar move by Amazon, which announced its own PyTorch version on Monday. Both companies are now offering developers a way around a habit that has long kept most AI software tied to Nvidia hardware, and that habit is the core of Nvidia's position in the industry.

What TorchTPU does

TorchTPU is designed to run PyTorch code on Google's Tensor Processing Units, or TPUs. A TPU is a custom chip Google built for machine learning, the heavy mathematical work of training AI models and running them once they are trained. Google first developed TPUs for its own services and later made them available to cloud customers.

The practical promise is speed and simplicity. Google says developers can move existing PyTorch apps onto TPUs and, by extension, onto Google Cloud, the company's cloud computing service. For a small team that has spent months tuning its software for one kind of chip, a switch that needs only a settings change could turn a costly rebuild into a manageable test.

The word open source matters here. Open-source software has publicly available code that anyone can inspect, use and modify. That means outside developers, researchers and companies can examine how TorchTPU works and build on it, rather than relying on a closed product from a single vendor.

Why PyTorch sits at the center of AI development

PyTorch is a popular open-source toolkit that AI developers and startups use to build, train and run AI models. Think of it as a set of ready-made building blocks. Instead of writing every mathematical operation from scratch, developers call functions that PyTorch already provides, and those functions do the heavy lifting on whatever hardware is available.

The source material says that some accounts put PyTorch's share of AI apps at 85 percent. That figure is attributed to unnamed accounts and has not been independently confirmed, so it should be read as an estimate. Even without that exact number, PyTorch is clearly one of the most widely used tools in the field, which is why changes to how it runs on different chips draw so much attention.

Meta, then called Facebook, originally developed PyTorch in 2016. That was years before large language models, the systems behind today's chatbots and writing tools, became a mainstream technology. In 2022, Meta handed PyTorch over to the Linux Foundation, a nonprofit that manages many important open-source projects. The move placed the toolkit under broader community governance rather than a single company's control.

How Nvidia became the default

Nvidia's graphics processing units, or GPUs, were first built to render images for video games. Their ability to perform many calculations at the same time later made them well suited to training neural networks, the type of AI model that now powers most modern systems. By the time PyTorch appeared, Nvidia's GPUs were already the standard hardware for AI research.

Because of that, many of PyTorch's features were built to work especially well with Nvidia chips. Over time, software written for PyTorch and tuned for Nvidia hardware became the default path. Switching to another chip often meant more than changing a machine. It could mean revisiting code, retesting models and accepting the risk that something would break in production.

That is why the stakes are about software as much as silicon. Nvidia's lead rests on its chips, but also on the habits, tools and code that developers already have. Breaking that lock-in requires making other chips feel just as comfortable to use, and that is the problem Google and Amazon are now trying to solve.

Amazon's parallel move

Amazon announced on Monday that it had released its own version of PyTorch, called TorchNeuron. The tool allows AI apps to be moved to Trainium, Amazon's homegrown AI chip, and to Amazon Web Services, known as AWS, without rewriting the software.

Amazon also said it is taking a further step. It is submitting code to the main PyTorch project that separates many features from Nvidia's chips. If that code is accepted into the core project, PyTorch would work more openly with any hardware by default, rather than being built around one brand. That would change the toolkit itself, not just the tools built on top of it.

Google's announcement came one day later, and the two companies are taking similar but not identical routes. Amazon is pushing changes into the shared PyTorch project, while Google's version is offered as its own open-source release tied to its chips and cloud.

What Google added to the release

Google says it worked directly with several companies and projects to make sure TorchTPU works well with widely used AI development tools. The companies named include Meta, Hugging Face, Inferact, which makes vLLM, and RadixArk, which makes SGLang. Hugging Face runs a large platform where developers share AI models, while vLLM and SGLang are tools used to run models efficiently. Google said the aim was for TorchTPU's features and code libraries to work with these tools in the same way the original PyTorch does.

The company also launched a dedicated tool called the TPU agent suite. It is designed to help developers move their apps to Google's chips and cloud, and to help them fix errors along the way. Those errors are often the most frustrating part of a migration, because a program can compile on new hardware but behave differently once it runs.

Google also updated LiteRT, a tool that runs AI on phones, laptops and embedded devices, so that it works better with PyTorch-based AI apps. Taken together, these steps go beyond a simple port of the software. They aim to reduce the everyday friction developers face when they change platforms.

What it could mean for developers and Nvidia

For developers, the most immediate effect is choice. A team that wants to test alternative hardware can now do so with less upfront work. Companies that have been limited by the supply or price of Nvidia chips may find it easier to compare options, and developers who stay with Nvidia may benefit if competition pushes prices or services to improve.

The source material is clear that this does not mean a rapid exodus from Nvidia. Developers are not expected to abandon Nvidia hardware en masse simply because switching has become easier. Many teams have years of code, tools and operating habits built around Nvidia systems, and moving that work takes time, testing and budget. The shift is more likely to happen gradually, starting with new projects, experiments and workloads where cost or availability matters most.

For Nvidia, the pressure is less about one product and more about the ecosystem around its chips. Its advantage has rested on hardware performance and on software that makes that hardware easy to use. If PyTorch becomes equally comfortable on other chips by default, that software advantage becomes harder to defend. Whether other chip makers and cloud providers follow with similar tools will help show how quickly that change spreads.

What to watch next

Several questions will determine how far this shift goes. The first is whether Amazon's proposed changes to the main PyTorch project are accepted. If they are, hardware flexibility would become a built-in feature of the core toolkit rather than an add-on from a single cloud provider.

The second is how well TorchTPU and TorchNeuron perform on real workloads. Developers will judge the tools by whether their applications run reliably, produce the same results and cost less or at least no more than they do on Nvidia systems. Early adopters' experiences will matter more than launch announcements.

The third is whether major AI companies and startups begin moving meaningful portions of their work to TPUs or Trainium. Google and Amazon have each given developers a path, but the path only matters if it is used. Developers and cloud customers should also watch how Nvidia responds, whether through pricing, software updates or new partnerships, because its reaction will shape how much room these alternatives get.

Questions and answers

What is Google TorchTPU?

TorchTPU is Google's version of the PyTorch AI toolkit, released as an open-source project on Tuesday. According to Google, it lets developers run existing PyTorch apps on Google's Tensor Processing Units and on Google Cloud.

Do developers need to rewrite their apps to use TorchTPU?

Google says developers can move existing PyTorch apps to TPUs with a simple configuration change and without rewriting their code.

How is TorchTPU different from Amazon's TorchNeuron?

Amazon's TorchNeuron lets AI apps move to its Trainium chip and AWS without rewriting, and Amazon is submitting code to make PyTorch more open to any hardware. Google's TorchTPU targets its TPU chips and Google Cloud, and Google also launched the TPU agent suite to help developers fix errors during the move.

Will developers stop using Nvidia chips?

The source material says developers will not automatically abandon Nvidia hardware en masse just because switching is easier. Amazon and Google are, however, clearly working to reduce Nvidia's advantage.

Topics: Google, TorchTPU, PyTorch, Nvidia, Amazon, Google Cloud, TPU chips, AI development tools

Sources: TechCrunch AI

https://axynewsglobal.com/article/google-releases-torchtpu-to-help-pytorch-apps-move-off-nvidia-gpus