Solely per week after Nvidia’s new AI-focused Volta GPU structure was introduced, Google goals to steal a few of its thunder with its new, second-generation, Tensor Processing Unit (TPU) that it calls a Cloud TPU. Whereas its first technology chip was solely appropriate for inferencing, and subsequently didn’t pose a lot of a risk to Nvidia’s dominance in machine studying, the brand new model is equally at house with each the coaching and working of AI programs.

A brand new efficiency chief amongst machine studying chips

At 180 teraflops, Google’s Cloud TPU packs extra punch, at the very least by that one measure, than the Volta-powered Tesla V100 at 120 teraflops (trillion floating level operations per second). Nonetheless, till each chips can be found, it gained’t be potential to get a way of an actual world comparability. Very like Nvidia has built servers out of multiple V100s, Google has additionally constructed TPU Pods that mix a number of TPUs to realize 11.5 petaflops (11,500 teraflops) of efficiency.

Google second-generation Cloud TPUFor Google, this efficiency is already paying off. As one instance, a Google mannequin that required a whole day to coach on a cluster of 32 high-end GPUs (in all probability Pascal), could be skilled in a day on one-eighth of a TPU Pod (a full pod is 64 TPUs, so meaning on eight TPUs). In fact, normal GPUs can be utilized for all types of different issues, whereas the Google TPUs are restricted to the coaching and working of fashions written utilizing Google’s instruments.

You’ll have the ability to hire Google Cloud TPUs to your TensorFlow purposes

Google is making its Cloud TPUs out there as a part of its Google Compute providing, and says that they are going to be priced just like GPUs. That isn’t sufficient info to say how they are going to evaluate in price to renting time on an Nvidia V100, however I’d count on it to be very aggressive. One downside, although, is that the Google TPUs presently solely assist TensorFlow and Google’s instruments. As highly effective as they’re, many builders is not going to wish to get locked into Google’s machine studying framework.

Nvidia isn’t the one firm that ought to be frightened

Whereas Google is making its Cloud TPU out there as a part of its Google Compute cloud, it hasn’t mentioned something about making it out there outdoors Google’s personal server farms. So it isn’t competing with on-premise GPUs, and positively gained’t be out there on aggressive clouds from Microsoft and Amazon. The truth is, it’s more likely to deepen their partnerships with Nvidia.

The opposite firm that ought to in all probability be frightened is Intel. It has been woefully behind in GPUs, which suggests it hasn’t made a lot of a dent within the quickly rising marketplace for GPGPU (Normal Function computing on GPUs), of which machine learning is a large half. This is only one extra means that chip that would have gone to Intel, gained’t.

Massive image, extra machine studying purposes might be shifting to the cloud. In some circumstances — should you can tolerate being pre-empted — it’s already inexpensive to hire GPU clusters within the cloud than it’s to energy them domestically. That equation is simply going to get extra lopsided with chips just like the Volta and the brand new Google TPU being added to cloud servers. Google is aware of that key to growing its share of that market is having extra vanguard software program working on its chips, so it’s making 1,000 Cloud TPUs out there without spending a dime to researchers prepared to share the outcomes of their work.