This Week in AI: Google Adds Even More AI for Photos and Translations

Google announced its latest series of new devices and AI tools this week. But amidst the new phones and smart watches, arguably the most interesting aspect of Google's announcement was that it echoes Apple's ambition to make AI an ambient assistant able to help you manage your daily life. (And at the same time, Apple is reportedly in talks with Google to use the latter's Gemini AI in a new version of Siri.)

Google's newest Magic Cue feature, which is starting in Pixel phones, draws information from across Google’s apps while you are using your device. So you might be texting someone about dinner reservations, for example, and suddenly Google's AI will recommend information from Gmail, drawing from the confirmation email you received when you set up those reservations in the first place. The same could happen with your travel. If you're about to call an airline, for example, the phone will automatically display booking information from Gmail to have it handy.

Most compelling is probably its universal translator (or "babel fish")-like feature, which takes an AI sample of your voice and translates what you're saying. The remarkable result is that you talk in one language, the AI hears you, and translates your voice into another language in near-real time.

This is a type of technology that's been long in the making. Language translation has been a key staple of sci-fi because it seems like one of those things that should be easy, but we all know is actually really hard. And with good reason, too, since many of us struggle to communicate well in one language, let alone try with a new one.

Google isn't the only company working on this. Each tech giant has their own take on the idea, with even Meta debuting a new translation tool for social video. "Meta AI Translations uses the sound and tone of your own voice so it feels authentically you," Meta said while announcing the feature. The company includes a "lip syncing" technology that matches mouth movements to sound, so "it looks like you're really speaking the language in the" video.

The technology still has a bit to go to ensure accuracy, expanded languages, and speed. But it's probably one of the most cool, yet least flashy, uses of AI yet.

Edit photos with your voice

Artificial intelligence has already had a massive impact on photography, offering new ways to remix images, edit out backgrounds, add elements and much more. Now, Google Photos will allow you to talk to your photo editor too, effectively giving it instructions like you would an expert sitting in front of you. Soon, Google said, you'll be able to tell your phone to remove someone from the background of an image, or to change a cloudy day into a sunny one.

Google said the tool is open-ended, so you don't have to know what the name of each tool is to get started. "And if you truly have no idea where to start, you can just start by typing or saying, 'make it better' or using one of the provided suggestions," Google said.

The feature draws on one of the most promising aspects of AI, which is its ability to understand us by translating our conversational speech into something a computer program can understand. That means many of us won't have to hunt around menus as much anymore, and or get anxious the next time Microsoft upgrades its Office software with a new look.

Google already seems aware of how impactful these types of new interfaces can be, and it has begun adding that idea around its suite of products. Last week, Google announced a change for its Flight Deals service, allowing you more flexibility when searching for trips. You can, for example, ask for where to go for a week-long hike, and Google will search around all the available flights everywhere to get an answer.

"Instead of playing with different dates, destinations and filters to uncover the best deals, you can just describe when, where and how you’d like to travel — as though you’re talking to a friend — and Flight Deals will take care of the rest," Google said.

AI toys are coming next 

The next generation of plushy toys is on its way, adding AI chatbot to your stuffed animals starting this holiday season.

Just a couple of years after ChatGPT took the world by storm, drawing hundreds of millions of users shortly after it launched, now the toy industry is getting in on the fun.

The New York Times reported on one of the newer entrants, called Curio, which makes a series of different AI-powered plushy toys, or what the reporter called "chatbots wrapped in stuffed animals."

"My children are already familiar with the idea of a mechanical friend, because as they watch television, they are served story after story about artificially intelligent sidekicks and their wondrous deeds," the NYT reporter noted.

While toys have often tried to piggyback off of AI type technologies, it hasn’t always gone well. Hello Barbie was probably the most dramatic example when it was hacked by researchers around its lunch in 2015. After some back-and-forth, Mattel ultimately pulled Hello Barbie from shelves.

One of the lessons that many companies have drawn from the toy industries' AI experiments is that understanding technology requires much more than just shoving the latest feature into your product. That said, AI companies have also learned that there is money to be made by offering easy tools to connect their products to any others out there, effectively allowing you to program anything to talk to a chat and get a response.

This first round of AI enabled toys will likely seem basic when compared to what's on the horizon. But it goes to show that the toy industry sees a lot of promise in what AI can do and how it can help people live the fantasy that their toys have always promised.

I remember how special it felt when I first heard a Teenage Mutant Ninja Turtles toy speak a catchphrase in the middle of a play session with a friend. More than three decades later, I may finally live the fantasy of being able to talk to my toy, and have it actually talk back...cowabunga?

More from MC News

• Everything You Need to Know About WiFi 7
• Keyboard 101: Intro to Computer Keyboards
• The NVIDIA 50 Series Laptop Buying Guide
• Can Your PC Run OpenAI's New GPT-OSS Large Language Models?
• Best 3D Printer Projects for Nintendo Switch 2
• How to Upgrade Your Laptop's RAM and Storage
• How to Build Your Own AI Personal Assistant
• The End is Coming for Windows 10: What You (Still) Need to Know
• Fix It Yourself: Talking to iFixit on Why Repairable Tech Matters

Ian Sherr is a widely published journalist who's covered nearly every major tech company from Apple to Netflix, Facebook, Google, Microsoft, and more for CBS News, The Wall Street Journal, Reuters, and CNET. Aside from writing, he tinkers with tech at home, is a longtime fencer -- the kind with swords -- and began woodworking during the pandemic.