“Every time you query the model, the whole thing gets activated, so it’s wildly inefficient from a computational perspective,” she continues.
Consider the large language models (LLMs) that power many generative AI systems. They have been trained on massive amounts of written information, allowing them to generate text in response to almost any question.
“When you use Generative AI… it’s generating content from scratch, it’s essentially making up answers,” Dr. Luccioni adds. That implies the computer has to work quite hard.
A Generative AI system may consume around 33 times more energy than machines running task-specific software, according to news.
