Hallucination

Hallucination is the failure mode where a large language model produces output that is factually incorrect, fabricated, or unsupported — but presented with the same confidence as accurate output.

LLMs are next-token predictors, not fact-retrievers. When asked about something they don’t know, they often invent something plausible-sounding rather than saying "I don’t know". Marketing-relevant hallucinations include made-up statistics, fabricated quotes, invented case studies, and confidently-wrong claims about competitors or product capabilities.

Mitigations include retrieval-augmented generation (ground the model in real brand documents rather than its training data), explicit "if you don’t know, say so" prompting, fact-checking judge models, and human review on any factual claim before publish. The risk is highest when AI is producing content the brand will be on the record for — pricing claims, product specs, customer quotes, statistics in thought-leadership posts.

A single hallucinated statistic in a published post is a brand-reputation issue and, in some industries, a compliance one. Tools that don’t protect against hallucination are passing the risk to the human reviewer.