MM1: A Novel Artificial Intelligence Model for Text, Images, and Chatbots (A Research Paper on Apple Engineers’ Website)

A research paper posted on Apple engineers’ website last Friday suggests that the company is making new investments in the field of artificial intelligence. MM1 is a new generative artificial intelligence model that works with text and images. The researchers show it answering questions about photos and displaying the kind of general knowledge skills shown by chatbots like ChatGPT. The model’s name is not explained but could stand for MultiModal 1.MM1 appears to be similar in design and sophistication to a variety of recent AI models from other tech giants, including Meta’s open source Llama 2 and Google’s Gemini. Work by Apple’s rivals and academics shows that models of this type can be used to power capable chatbots or build “agents” that can solve tasks by writing code and taking actions such as using computer interfaces or websites. That suggests that MM1 might be included in Apple’s products.

According to a professor at Carnegie Mellon, the fact that they’re doing this shows they can understand how to train and build models. There is a certain amount of expertise required.

MM1 is a relatively small model that is measured by a number of parameters, or internal variables that get adjusted as a model is trained. Kate Saenko, a professor at Boston University who specializes in computer vision and machine learning, believes that this will allow Apple to experiment with different training methods before reaching their full potential.

An example in the Apple research paper shows what happened when MM 1 was given a photo of a restaurant table and a beer bottle, as well as an image of the menu. The model correctly calculated the cost when it was asked how much someone would pay for all the beer on the table.

When ChatGPT launched in November 2022, it could only ingest and generate text, but more recently its creator OpenAI and others have worked to expand the underlying large language model technology to work with other kinds of data. In December of last year, when the model that now powers its answer to chatGputt was launched, the company boasted that it would be the beginning of a new direction in artificial intelligence. The next frontier for foundation models is MLLMs after the rise of LLMs.

It was not possible to train the leading models without using copyrighted materials told the UK parliament. There is a wave of lawsuits involving the use of online resources to train models in the field of Artificial Intelligence. Two announcements Wednesday offer evidence that large language models can in fact be trained without the permissionless use of copyrighted materials.

Fairly Trained offers a certification to companies willing to prove that they’ve trained their AI models on data that they either own, have licensed, or is in the public domain. When the nonprofit launched, some critics pointed out that it hadn’t yet identified a large language model that met those requirements.

Trained Fairly certified its first large language model. 273 Venture came up with KL3M, which was based on a trove of legal, financial, and regulatory documents.

The company’s cofounder Jillian Bommarito says the decision to train KL3M in this way stemmed from the company’s “risk-averse” clients like law firms. She says they need to know that output is not based on tainting data. “We’re not relying on fair use.” The clients didn’t want to get into lawsuits about intellectual property when using generative AI for legal documents and drafting contracts, but they were interested in it.

Bommarito says that 273 Ventures hadn’t worked on a large language model before but decided to train one as an experiment. She says that they did a test to see if it was possible. The company has created its own training data set, the Kelvin Legal DataPack, which includes thousands of legal documents reviewed to comply with copyright law.

Despite the small size of the data, the model performed better than expected as a result of how carefully the data had been researched. She says having clean, high-quality data may mean you don’t have to make a large model. It’s possible to Curate a dataset to make a special model for the task it’s designed for. 273 Ventures is now offering spots on a waitlist to clients who want to purchase access to this data.

The data was put together by the US Library of Congress and the National Library of France. Pierre-Carl Langlais, project coordinator for Common Corpus, calls it a “big enough corpus to train a state-of-the-art LLM.” In the lingo of big AI, the dataset contains 500 million tokens, OpenAI’s most capable model is widely believed to have been trained on several trillions.