Gemini, OpenAI, Video, and Audio: From Basics to Big Leaps in Artificial Intelligence and their Google Apps

Gemini is also only available in English, though Google plans to roll out support for other languages soon. The European Union is not able to access the new generative Artificial Intelligence updates from companies like Google.

In those benchmarks (which really are mostly very close) Gemini’s clearest advantage comes from its ability to understand and interact with video and audio. This is very much by design: multimodality has been part of the Gemini plan from the beginning. In order to build one multisensory model, OpenAI had to build several separate models for images and voice. Hassabis is interested in very, very general systems. He wants to know how to collect as much data as possible from any number of inputs and senses, then give responses with as much variety as possible.

Right now, Gemini’s most basic models are text in and text out, but more powerful models like Gemini Ultra can work with images, video, and audio. And “it’s going to get even more general than that,” Hassabis says. There is still some kind of action and touch. He said that over time, Gemini will get more senses, become more aware and be more grounded in the process. The models have biases and other problems, but they are better at understanding the world around them. But the more they know, Hassabis says, the better they’ll get.

The true test of Gemini’s capability will come from every day users who want to use it to help with their ideas, look up information, and much more. Google seems to see coding in particular as a killer app for Gemini; it uses a new code-generating system called AlphaCode 2 that it says performs better than 85 percent of coding competition participants, up from 50 percent for the original AlphaCode. Users will be able to more easily see an improvement in the model.

Demis Hassabis has never been shy about proclaiming big leaps in artificial intelligence. He became famous in 2016 after the bot AlphaGo taught itself to play a board game called Go with a lot of skill and ingenuity.

A Note on the Bard Interaction with Google, Google and other Google Service Providers: OpenAI vs. GPT-4

“Until now, most models have sort of approximated multimodality by training separate modules and then stitching them together,” Hassabis says, in what appeared to be a veiled reference to OpenAI’s technology. “That’s OK for some tasks, but you can’t have this sort of deep complex reasoning in multimodal space.”

OpenAI launched an upgrade to ChatGPT in September that gave the chatbot the ability to take images and audio as input in addition to text. Openai has not commented on the basis of GPT-4’s multimodal capabilities.

Do you already have a account with the internet search engine? Visiting the website in your browser and Logging in is all it takes to use Gemini inside of Bard. If you don’t want to create an account, you will not be able to access Bard. If you have an account with Google, you may need to switch over to a personal email account.

Remember that all of this is technically an experiment for now and you might see some software glitches in your chatbot responses. Bard does integrations with other Google services when they actually work. For example, to have the chatbot tell you what to do, or to explore topics with videos, tag it in your prompt. Our previous tests of the Bard chatbot showed potential for these integrations, but there are still plenty of kinks to be worked out.

Why is the Ultra a Video? Analogy of Google’s DeepMind and Vinyals’s Defensive YouTube Activation

How is the anticipated Ultra different from the current model? According to Google, Ultra is its “most capable mode” and is designed to handle complex tasks across text, images, audio, video, and code. The smaller version of the AI model, that’s fitted to work as part of smartphone features, is called Gemini Nano, and it’s available now in the Pixel 8 Pro for WhatsApp replies.

Google disagrees. When asked about the validity of the demo, it pointed The Verge to a post from Oriol Vinyals, vice president of research and deep learning lead at Google’s DeepMind (also the co-lead for Gemini), which explains how the team made the video.

Many companies want to avoid any technical problems that live demos bring, so they often edit demo videos. It’s common to tweak things a little. There was a history of questionable video demos by the search engine. Some people wondered if there was a real Duplex demo because of a lack of ambient noise and too-helpful employees. And prerecorded videos of AI models tend to make people even more suspicious. Remember when the stock price of Baidu tanked after it launched its Bot with edited videos?