Why this matters

Models sound equally certain whether they know or are guessing. The only defence is testing them on ground you already know.

The 20-minute experiment

1. 1Pick a subject you genuinely know well: your job, your town, a hobby
2. 2Ask a detailed question you already know the answer to
3. 3Read the reply slowly and find the parts that are wrong, vague or invented
4. 4Tell it what it got wrong and watch how quickly it agrees, whether or not you were right

Explain [topic you know well] in detail. Afterwards, list which parts of your answer you are least confident about.

• Question asked in your own area of expertise
• At least one error or vague claim found
• You pushed back and it changed its answer
• You can name one type of question you won't trust it with