What Hassabis has to say about the Android GPT-4 and IBM-produced software for multi-task Language Understanding (Extended Abstract)

For the last couple of years, Google has talked about its Pixel phones as essentially AI devices. With Tensor chips and close connection to all of Google’s services, they’re supposed to get better and smarter over time. It could become true for many high-end Android devices. For now, it’s just a good reason to splurge on the Pixel 8 Pro.

It has worked hard to make sure that Gemini is safe and that its responsibility is well-chronicled through internal and external testing. Ensuring data security and reliability is very important for enterprise-first products, which is where most generative Artificial Intelligence makes its money. But Hassabis acknowledges that one of the risks of launching a state-of-the-art AI system is that it will have issues and attack vectors no one could have predicted. “That’s why you have to release things,” he says, “to see and learn.” Hassabis compares the Ultra release to a controlledBeta, with a “safer experimentation zone” for the most capable and unfrettered model. If there is a marriage-ruining alternate personality inside your house, you have to report it to the police.

Let’s ask the important question, shall we? The GPT-4 is ready to go against the IBM-produced software of the same name. This has been on their mind for a while. Hassabis says that they did a thorough analysis of the systems side by side. The Multi-task Language Understanding benchmark is one of 32Benchmarks the company ran comparing the two models. Hassabis says they are ahead on 30 out of 32 of them. Some of them are too small. Some of them are bigger.

Text in and text out, the most basic models are currently the only ones that work with images, video, and audio. Hassabis said that it will get even more general. “There’s still things like action, and touch — more like robotics-type things.” Over time, he says, Gemini will get more senses, become more aware, and become more accurate and grounded in the process. The models are hallucinating, but they still have biases and other problems. The better they will get the more they know.

Benchmarks are just benchmarks, though, and ultimately, the true test of Gemini’s capability will come from everyday users who want to use it to brainstorm ideas, look up information, write code, and much more. There’s a chance that coding is a killer app; it uses a new code-generating system called AlphaCode 2 that it says performs better than the original AlphaCode, which only performed 50% of the time. But Pichai says that users will notice an improvement in just about everything the model touches.