Cohere tackles some generative AI challenges with Command R – TechTarget

author
3 minutes, 3 seconds Read
image

AI startup Cohere’s new large language model seeks to address some challenges enterprises face with generative AI technology.

Cohere on March 11 introduced Command R, a generative large language model (LLM) optimized for long context tasks such as retrieval augmented generation and using external APIs and tools.

Retrieval augmented generation (RAG) is a technique used to improve the accuracy and reliability of generative AI models.

The new LLM is available through Cohere’s hosted API. It will soon be available on major cloud platforms, according to the independent AI startup, which has a partnership with and investment from tech giant Oracle.

Large context window and accuracy

The LLM has a 128k context window and is available in ten major languages, including English, French, Spanish, Italian, German and Chinese. A context window is the textual or video amount that an LLM can process.

Command R comes as AI vendors such as Google are introducing models with context windows as large as 1 million tokens.

However, large context windows could lead to a higher probability of inaccurate outputs by the LLM.

Cohere’s Command R aims to address the challenge of factual inaccuracies surrounding RAG so that as enterprises input large documents, they’re likely to get accurate answers in what the LLM generates, Forrester Research analyst Rowan Curran said.

“The main thing that is approached with this release is being able to ensure that you can give very large amounts of information to these large context windows and be more assured that it’s going to remember everything within that context window when it produces the generation on the other side,” Curran said. “For the enterprise, having an offering that’s designed around what is emerging as the standard architecture for the current generation of large language model powered applications is an appealing thing.”

A holistic approach

Cohere also takes what he calls a holistic approach with Command R, Futurum Research analyst Paul Nashawaty said.

Usually, LLM providers introduce models using specific databases or data warehouses. However, Command R will be available across major cloud providers as well as on premises.

“That’s not just in one dataset, but it’s across all the datasets within your ecosystem,” Nashawaty said. “So that’s really huge.”

For developers, this allows them to connect to newer or existing data sources, he added. For organizations with existing datasets, Command R them to use their older as well as modernized environments.

Another differentiator is Command R’s pricing model, Nashawaty said. It costs $0.50 per million input tokens and $1.50 per million output tokens.

The pricing model allows for API consumption based on usage, which matters when connecting to the cloud, Nashawaty noted.

“The number of API calls can increase your cost,” he said. “By having fewer API calls, you can reduce your costs on the cloud.”

Some limitations

Command R’s language capabilities are a step in the right direction, but more can be done, Curran said, noting that the LLM is not available in any languages from India or Southeast Asia.

“That is one of the interesting linguistic gaps in a lot of these language models overall and it continues to persist here,” he said.

Moreover, since generative AI is still a new market, the promise of scalability with Command R will need to be vetted, Nashawaty said.

“They do tend to focus on scalability as a key differentiator,” he said. “I just question about the specific use cases. They can search millions or billions of documents, which is great. I’d like to see that in a real-world scenario.”

Command R’s model weights are now available on Hugging Face for research and evaluation.

Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.

This post was originally published on this site

Similar Posts