Nvidia CEO Jensen Huang in his usual leather jacket.
Getty
Nvidia announced new software on Tuesday that will help software makers prevent artificial intelligence models from misrepresenting facts, talking about harmful topics or exposing security holes.
The software, called NeMo Guardrails, is one example of how the AI industry is trying to solve the problem of “hallucinations” with the latest generation of large language models, a major sticking point for businesses.
Large language models such as GPT with Microsoft-support for OpenAI and LaMDA from Google, trained on terabytes of data to create programs that can spit out blocks of text that read as if they were written by a human. But they also tend to make things up, which practitioners often call “hallucinations.” Early applications of the technology, such as summarizing documents or answering basic questions, must minimize hallucinations to be useful.
Nvidia’s new software can do this by adding fences that prevent the software from looking at topics it shouldn’t. NeMo Guardrails can force the LLM chatbot to talk about a specific topic, prevent toxic content, and prevent LLM systems from executing malicious commands on the computer.
“You can write a script that says if someone talks about this topic, no matter what, respond that way,” said Jonathan Cohen, Nvidia’s vice president of applied research. “You don’t have to trust that the language model will follow a prompt or your instructions. In fact, the execution logic of the fencing system is hard-coded as to what will happen.”
The announcement also underscores Nvidia’s strategy to maintain leadership in the AI chip market by simultaneously developing critical machine learning software.
Nvidia provides the thousands of GPUs needed to train and deploy software like ChatGPT. According to analysts, Nvidia has more than 95% of the AI chip market, but competition is growing.
How it works
NeMo Guardrails are a layer of software that sits between the user and a large language model or other AI tools. It prevents bad results or bad prompts before the model spits them out.
Nvidia proposed a customer service chatbot as one use case. Developers could use Nvidia’s software to prevent the conversation from going off-topic or “off the rails,” increasing the potential for pointless or even toxic backlash.
“If you have a customer service chatbot designed to talk about your products, you probably don’t want it to answer questions about our competitors,” Nvidia’s Cohen said. “You want to monitor the conversation. And when that happens, you bring the conversation back to topics you enjoy more.”
Nvidia offered another example of a chatbot that answered internal corporate HR questions. In this example, Nvidia was able to add “safeguards” to prevent the ChatGPT-based bot from answering questions about the application company’s financial performance or accessing private data about other employees.
The software can also use the LLM to detect hallucinations by asking another LLM to check the response of the first LLM. It then returns “I don’t know” if the model does not come up with suitable answers.
Nvidia also said Monday that the fencing software helps with security and can force LLM models to interact only with whitelisted third-party software.
NeMo Guardrails is open source and is offered through Nvidia Services and can be used in commercial applications. Programmers will use the Golang programming language to write custom rules for the artificial intelligence model, Nvidia said.
Other AI companies, including Google and OpenAI, have used human feedback reinforcement to prevent harmful outcomes in LLM programs. This method uses human testers to generate data about which answers are acceptable and which are not, and then trains an artificial intelligence model using that data.
Nvidia is increasingly turning its attention to artificial intelligence, as it currently dominates the chips used to create this technology. Riding on the AI wave that has made it the biggest gainer in the S&P 500 so far in 2023, the stock is up 85% as of Monday.
