CodeLama: a tool to generate and debug human-written code in the modern era of AI, GPT-4, and AlphaCode

Meta has released a tool called Code Llama, built on top of its Llama 2 large language model, to generate new code and debug human-written work, the company said.

Programmers are using LLMs to aid in a number of tasks, ranging from writing new software to debugging existing code, according to Meta. The goal is to make the developer’s work more efficient so they can concentrate on the most important aspects of their job.

Code generators help developers work. Copilot was powered by GPT-4 to quickly write and check code. Old code can also be updated by Copilot. CodeWhisperer is a service which writes, checks and updates code. And yes, Google also has a code-writing tool in AlphaCode, but that isn’t out yet.

Microsoft and Openai are accused of violating intellectual property law because their tool Copilot can reproduce licensed code.

The Copilot Code: A tool for AI coding research based on Meta LLMs, GPT, and the GitHub-Copilot plug-in

“It’s exciting that they’re releasing the weights to the community,” says Deepak Kumar, a postdoctoral researcher at Stanford who has studied AI coding, referring to the parameters of the neural network at the core of the model.

Kumar says the release of Meta’s regular language model Llama 2 led to the formation of communities dedicated to discussing how it behaves and how it can be modified. “It gives us a little bit more flexibility to play with what exactly is going on under the hood, compared to these closed-source models from Google or OpenAI.”

Talia Ringer, an assistant professor at the University of Illinois Urbana-Champaign who researches programming, says that Code Llama will be valuable for academic research. I already have students using Llama models for research, and I could see those students being extra excited about a code model given the nature of our work,” she says. The data used for training is also supposed to be released. “That’s often the missing piece for making sense of research on LLMs,” she says.

In May 2021, GitHub, a subsidiary of Microsoft, launched Copilot, a plug-in for coding programs that auto-completes sections of code based on the first line or a comment typed by the user. GPT is a large language model which is used by Copilot. That model is trained further using code that GitHub stores for developers, as well as, reportedly, by contractors who are paid to annotate their own code.

Masad says Meta may have limited the training data in order to avoid the consequences of a lawsuit being filed for using open source code. Copilot costs $10 per month for individuals and $19 per month, per user, for businesses.