It Can't Do That

I am being driven slowly insane by this continuing "surprise" about the fact that "AI" can't possibly do what people expect it to do because that is not what it is designed to do. It doesn't know anything! It doesn't reason! It really is just "spicy autocomplete."

Weeks earlier, the US Food and Drug Administration, the division of HHS that oversees vast portions of the American pharmaceutical and food system, had unveiled Elsa, an artificial intelligence tool intended to dramatically speed up drug and medical device approvals.

Yet behind the scenes, the agency’s slick AI project has been greeted with a shrug — or outright alarm.

Six current and former FDA officials who spoke on the condition of anonymity to discuss sensitive internal work told CNN that Elsa can be useful for generating meeting notes and summaries, or email and communique templates.

But it has also made up nonexistent studies, known as AI “hallucinating,” or misrepresented research, according to three current FDA employees and documents seen by CNN. This makes it unreliable for their most critical work, the employees said.

“Anything that you don’t have time to double-check is unreliable. It hallucinates confidently,” said one employee — a far cry from what has been publicly promised.