Qualcomm’s new flagship mobile chipset is an on-device AI fest from top to bottom. The Snapdragon 8 Gen 3 brings generative AI capabilities right to the chipset, vastly speeding up the processing-intensive activities that are usually outsourced to the cloud. It’s one more leap forward in a year that has seen a lot of leaping when it comes to AI, and it will come sooner than later: the first phones with the new chipset are expected to debut in the coming weeks.
The 8 Gen 3 supports a chatbot trained on Meta’s Llama 2, and it can accept text, image, and voice input. It can also talk back to you as well as generate an image or text. The chipset also runs the AI image generator Stable Diffusion on-device, something that Qualcomm demoed earlier this year. This time around, the company says it can generate an image in less than one second — the previous tech generated an image in about 15 seconds, which is still faster than the couple of minutes it can take on even a well-equipped laptop.
It’ll even relay your favorite activities and “fitness level,” which sounds slightly dystopian
All of this is housed in what Qualcomm calls its AI engine, utilizing the company’s Hexagon neural processor. Meanwhile, the Sensing Hub uses OpenAI’s Whisper for speech recognition. The Sensing Hub also supplies the AI engine with information about the user, including location, for more personalized responses. It’ll even relay your favorite activities, age, and “fitness level,” which sounds slightly dystopian if you ask me.
Generative AI also plays a major role in the 8 Gen 3’s new image processing capabilities. It will support generative fill for image expansion so you can zoom out and re-crop photos — again, right on the device. The video features are even wilder: there’s an object eraser for video — just tap and an unwanted subject disappears! — and on-device night mode recording at up to 4K / 30p. Google announced its own Night Sight for video feature coming to the Pixel 8 Pro in the near future, but it will run in the cloud.
There’s also a feature called Vlogger’s View that will layer video from the selfie and rear cameras together into one view. It’s not a picture-in-picture thing — this feature uses improved image segmentation to remove the background from the selfie video to make it look like you’re standing in front of whatever your rear camera sees.
There’s good reason to be concerned about the misuse of these tools, and to that end, Qualcomm is working with a company called Truepic. The tech it’s using is compliant with the Coalition for Content Provenance and Authenticity’s open standard, which is kind of like metadata to…