\n<\/p>\n
Every time a new Pixel phone comes out, you might hear that \u201con-device processing\u201d makes its cool new features possible. Just take a look at the new Pixel 9 phones \u2014 things like Pixel Studio and Call Notes run \u201con device.\u201d And it\u2019s not just phones: Nest cameras, Pixel smartwatches and Fitbit devices also use this whole \u201con-device processing\u201d thing. Given the devices that use it and the features it\u2019s powering, it sounds pretty important.<\/p>\n
It\u2019s safe to assume that the, er, processing, is happening on the, uh\u2026well, the device. But to get a better understanding of what that means, we talked to Trystan Upstill, who has been at Google for nearly 20 years working on engineering teams across Android, Google News and Search.<\/p>\n
You were on a team that helped develop some of the exciting features that shipped with our new Pixel devices \u2014 can you tell me a little about what you worked on?<\/b><\/p>\n
Most recently, I worked within Android where I led a team that focuses on melding Google\u2019s various technology stack into an amazing experience that\u2019s meaningful to the user. Then figuring out how to build it and ship it.<\/p>\n
Since we\u2019re improving technologies and introducing new ones quite often, it seems like that would be a never-ending job.<\/b><\/p>\n
Exactly! Within recent years, there\u2019s been this explosion in generative AI capabilities. At first when we started thinking about running large language models on devices, we thought it was kind of a joke \u2014 like, \u201cSure we can do that, but maybe by 2026.\u201d But then we began scoping it out, and the technology performance evolved so quickly that we were able to launch features using Gemini Nano, our on-device model, on Pixel 8 Pro in December 2023.<\/p>\n
That\u2019s what I want to know more about: \u201con-device processing.\u201d Let\u2019s break it down and start with what exactly \u201cprocessing\u201d means.<\/b><\/p>\n
The main processor, or system-on-a-chip (SoC), in your devices, has a number of what are called Processing Units designed specifically to handle the tasks you want to do with that device. That’s why you’ll see the chip (like the Tensor chip found in Pixels) referred to as a “system-on-a-chip: There’s not just one processor, but several processing units, memory, interfaces and much more, all together on one piece of silicon.<\/p>\n
Let\u2019s use Pixel smartphones as an example: The processing units include a Central Processing Unit, or CPU, as the main \u201cengine\u201d of sorts; a Graphics Processing Unit, or GPU, which renders visuals; and now today we have a Tensor Processing Unit, or TPU, specially designed by Google to run AI\/ML workloads on a device. These all work together to help your phone get things done \u2014 aka, processing.<\/p>\n
For example, when you take photos, you\u2019re often using all elements of your phone\u2019s processing power to good effect. The CPU will be busy running core tasks that control what the phone is doing, the GPU will be helping render what the lens is seeing and, on a premium Android device like a Pixel, there’s also a lot of work happening on the TPU to process what the optical lens sees to make your photos look awesome.<\/p>\n
Got it. \u201cOn-device\u201d processing implies there\u2019s off-device. Where is \u201coff-device processing\u201d happening, exactly?<\/b><\/p>\n
Off-device processing happens in the cloud. Your device connects to the internet and sends your request to servers elsewhere, which perform the task, and then send the output back to your phone. So if we wanted to take that process and make it happen on device, we\u2019d take the large machine learning model that powered that task in the cloud and make it smaller and more efficient so it can run on your device\u2019s operating system and hardware.<\/p>\n
What hardware makes that possible?<\/b><\/p>\n
New, more powerful chipsets. For example, with the Pixel 9 Pro, that\u2019s happening thanks to our SoC called Tensor G4. Tensor G4 enables these phones to run models like Gemini Nano \u2014 it\u2019s able to handle these high-performance computations.<\/p>\n
So basically, Tensor is designed specifically to run Google AI, which is<\/b> also<\/i><\/b> what powers a lot of Pixel\u2019s new gen AI capabilities.<\/b><\/p>\n
Right! And the generative AI features are definitely part of it, but there are lots of other things on-device processing makes possible, too. Rendering video, playing games, HDR photo editing, language translation \u2014 most everything you do with your phone. These are all happening on your phone, not being sent up to a server for processing.<\/p>\n<\/div>\n