A Skeptical AI
Mon Apr 28 2025
AI today faces a fundamental flaw: it does not distinguish between truth and falsehood in its learning process. It accepts as fact everything it is taught, without built-in skepticism or verification.
Until AI gains the ability to perform its own experiments and independently verify the truth of information, it will remain dependent on the structure and biases of human-provided knowledge and will learn some non-sense. Solutions like reinforcement-based pruning may reduce some misinformation but will never fully eliminate the core issue: AI cannot know truth without acting on the world to observe cause and effect.
Knowledge as a Graph Structure
We can think of knowledge as nodes in a graph, where the most connected nodes represent the most “relied-on” knowledge. In theory, if an organism using this graph is successful, these core nodes are beneficial to survival. However, benefit does not equal truth. Success could arise from widespread but flawed beliefs.
Thus, an AI built purely on human data cannot inherently decide what is true or not—unless it can independently run experiments and verify knowledge against the real world.
The Need for Skepticism in AI
To create skeptical AI, it must assess whether new information conflicts with its existing knowledge. However, if information is presented in the wrong sequence—for example, fiction before fact—the AI might incorrectly classify real facts as false because they conflict with already-accepted misinformation.
One proposed solution is assuming that true knowledge is self-reinforcing, while false knowledge is not. This would allow AI to cross-check facts against the rest of its knowledge graph and eliminate information that lacks reinforcement.
But this approach carries risks:
Cutting-Edge Knowledge Loss: New discoveries often don’t have extensive cross-referencing yet. Aggressively pruning under-connected knowledge risks eliminating important, emerging truths.
Premature Skepticism: AI would need a method to recognize when information is simply too new to be reliably cross-validated and therefore not prematurely discard it.
Core Limitation
Attempting to prune the knowledge graph to remove “dangling” leaves—knowledge with few connections—may inadvertently strip away interesting or important ideas that haven’t yet been widely reinforced.
Ultimately, AI learns garbage because it cannot independently test hypotheses against the real world. It absorbs whatever it is fed, without the sensory experience or experimental framework that living organisms use to validate knowledge.
Afterthought:
An agentic learning process is necessary to think through how to integrate the new information and determine the truthfulness of it. New information needs to be bottlenecked through an agent. However, an agentic training step eliminates much of the efficiencies gained by parallel training. Maybe AI will have to learn slowly, as slowly as a human, but with the benefit of being copied as many times as needed.