- Google Gemini Live now provides visual guidance with real-time cues on your screen when you share your camera
- The feature is designed to help users solve tasks visually and identify objects by sight
- Gemini Live has also expanded its app integrations and introduced expressive voice upgrades
Google is continuing its quest to get people to use its Gemini AI assistant at all times and in all places with a new set of upgrades launched alongside the Pixel 10 series of smartphones. The centerpiece of the new and improved Gemini Live is a set of AI eyes, a feature called visual guidance.
Basically, you can give Gemini Live access to your camera, and it will look at the same things you’re looking at and help you figure out things like the right tool to use, the best choices to coordinate an outfit, or other tasks. The solutions will be right on the screen, with arrows or circles around the correct answer. For now, the feature will only be available on the Pixel 10, but other Android phones and even iOS devices will be able to use the feature in the near future.
Visual guidance might sound like a party trick, but it could prove to be a major draw for Gemini Live. Instead of receiving a flat, spoken answer when you ask Gemini to help assemble a new piece of furniture, you can now show the parts to your camera and have the assistant visually indicate which goes where. It doesn’t require special hardware; it’s like showing a friend who’s good at DIY what you have and asking for help.
Google clearly sees it as a way to bridge the awkwardness that sometimes happens when you ask an AI for help and it gives you vague or overly generic answers. “Use the blue-handled pliers,” might not help much if your toolbox has three tools with blue handles. A glowing circle over the right one is much more helpful. As someone who has tried to follow a YouTube tutorial while simultaneously wielding a screwdriver, I get the appeal.
Sweet talk and multitasking
Gemini Live will also sound better when it’s showing you things, thanks to new speech models capable of adjusting the tone, and even the character of the voice. So, Gemini might use an especially calm voice to talk about a stressful topic, speed up when you’re in a hurry, or perhaps tell you a story about pirates in the stereotypical pirate accent.
Gemini Live is also going to be better at multitasking thanks to new links to apps like Google Calendar, Messages, and Maps. So, when you’re chatting with Gemini, you could get it to handle your personal appointments and send texts to your friends with directions.
The revamp of Gemini Live fits with Google’s broader approach to AI, positioning it as an ambient, always-on platform rather than a standalone feature. AI assistance that is flexible enough for any event, while using context to be specifically valuable for individuals, is what Google and other AI developers have promised for a while. And while the visual guidance and other tools aren’t going to be perfect, the adaptability could make up for it. You don’t need to learn a new system or talk in commands. You just show Gemini what you see, ask it what you need, and get a reply that’s tuned to the topic.
