Microsoft’s Bing Chat AI bot now lets you search using images

Microsoft on Tuesday announced a major update to its AI-powered chatbot: Visual Search. Users can now take or upload a photo to Bing Chat and request more information about it via the desktop or Bing apps.

“Bing can understand the context of an image, interpret it, and answer questions about it,” Microsoft wrote in a release. “Whether you’re traveling to a new city on vacation and asking about the architecture of a certain building, or at home trying to come up with dinner ideas based on the contents of your fridge, upload an image to Bing Chat and use it to tap into the knowledge of the web to get answers.”

The update comes as the AI ​​arms race heats up among chatbot leaders like Microsoft, Google, OpenAI and Anthropic. In an effort to develop the most advanced generative AI, the tech giants are rapidly rolling out new features in an effort to keep pace not only with their text-based chatbot rivals, but also with image-rich AI tools.

While image searches—and responses that include images—are now part of the chatbot user experience, it seems that none of the leading text-based chatbots can yet create their own images, unlike tools like like Midjourney, Stable Diffusion and DALL-E 2. However, Google says this feature is on the way for its Bard chatbot.

Microsoft’s decision to allow images for Bing Chat follows Google’s recent debut of an image search feature for Bard, its AI chatbot. Using Google Lens, users can ask Bard for information about an image they’ve uploaded, ask it to create a caption, or even add a highlight to the chatbot’s responses, such as asking for restaurant recommendations with photos of restaurant interiors. At the time of writing, OpenAI’s ChatGPT does not allow uploading photos as the chatbot is still entirely text-based, while Anthropic’s chatbot, Claude 2, works similarly.