A developer analysis explains that asking an LLM to express confidence numerically does not make the number a probability that its answer is correct. The number is also generated text and must be calibrated against actual outcomes for the task. Even a model that always gives the same average probability can appear well calibrated, so its ability to distinguish correct from incorrect answers needs separate testing. The article recommends combining external signals such as rule checks, retrieved evidence, and answer consistency. A related article by the same author also identifies the ability to detect and recover from errors as an important condition for moving a tool to full automation.