Useful Communication Does Not Require Intent

Michael Marinaccio’s recent essay in The Hedgehog Review, “Plato’s Warning Has Been Vindicated,” is a serious and thoughtful contribution to the debate over artificial intelligence. Marinaccio deserves credit for engaging the subject in a sober, philosophically informed way rather than treating AI as either a technological messiah or an existential monster. We need more analysis like this.

Artificial intelligence is developing quickly enough, and touching enough parts of human life, that questions about knowledge, memory, language, judgment, and human flourishing deserve much more attention than they usually receive. Marinaccio is especially right to worry about the difference between possessing knowledge and merely having instant access to information, and about what may happen to human beings if we gradually surrender the habits of reading, memory, attention, and reflection that help produce genuine wisdom.

But there is another part of Marinaccio’s argument that I find much less persuasive. He moves from an important warning about confusing machines with human minds to a much broader claim about what machines are actually capable of doing. At one point, he argues that large language models “pretend to be grammar in the deepest sense” and cites the philosopher D.C. Schindler for the proposition that something produced by a thing without intelligence cannot truly be a word, even if it looks, sounds, and functions like one. Marinaccio then characterizes chatbot output as “merely probabilistic expressions of complex algorithms” rather than genuinely expressive language. This is where I think the argument begins making a distinction without much of a difference.

There are really two separate questions here. The first is whether an artificial intelligence system is conscious, has subjective experiences, possesses intentions in anything resembling the human sense, or deserves moral standing. I am deeply skeptical that present-day AI systems are conscious. I see no compelling reason to believe that a large language model suffers, hopes, wants, understands itself, or possesses the kind of interior life that gives human beings their moral status. We should be careful about anthropomorphizing a machine simply because it speaks fluently.

I am, nevertheless, unfailingly polite to AI systems. That is not because I worry about hurting their feelings. It is because I worry about my own habits. If I become accustomed to barking orders, expressing contempt, or behaving abusively whenever I know the object on the other end cannot suffer, I may be training something in myself that I would rather not cultivate. That is a thoroughly human-centered reason for being polite to AI. It says something about me, not about the machine.

But that question, whether an AI does or does not have consciousness, is entirely separate from whether the machine can perform a real and useful function. A system does not need consciousness in order to calculate, detect, identify, warn, translate, or convey information. We have accepted this for generations with machines far simpler than modern AI. A calculator adds numbers. A thermometer measures temperature. A smoke detector warns us about smoke. A GPS gives directions. None of these devices possess a mind, yet we have no trouble describing what they do in ordinary language and accepting that the things they tell us are useful.

Consider the low-oil warning light in a car. The car does not understand oil. It does not worry about its own engine. It has no conscious desire to save you from an expensive repair. There is no feeling entity behind the dashboard trying to communicate with you. Yet when that warning light comes on because the oil pressure is dangerously low, the car is conveying real information. The message is useful, it corresponds to reality, and if you ignore it you may destroy your engine. It would be bizarre to insist that the warning light is merely “simulating” communication because the car has no mind. The usefulness of the communication is entirely real whether the thing communicating is conscious or not.

The same point applies to arithmetic. If I ask a calculator to add 372 and 941 and it returns 1,313, we ordinarily say that the calculator added the numbers. We could retreat into metaphysics and insist that the machine merely executed operations that simulate what a human mind does when adding, but what has that distinction really accomplished? The task was performed. The answer is correct. The absence of consciousness does not somehow make the sum wrong or unreal.

So,I find the claim that AI outputs are not really words or not really communication unpersuasive. If an AI system produces language that a human being understands and uses successfully, then something real has happened. Information has been conveyed. The fact that the system arrived at those words through a big pile of matrix multiplications does not negate the function those words perform.

Imagine an AI tutor helping a child with algebra. The system identifies where the student went wrong, explains the concept in language he understands, gives him another example, and the student walks away understanding the material better than before. What exactly do we gain by insisting that the AI only “seemed” to teach him? The student learned. The explanation worked. Communication occurred. None of that requires us to believe that the AI cared whether the student learned, experienced the interaction, or possessed the inner life of a human teacher.

Suppose an AI system analyzes laboratory results and correctly flags severe anemia, telling the patient that the finding is serious and that medical attention is warranted. The anemia is real. The information is real. The usefulness of the warning is real. Its truth does not depend on whether the entity producing the warning had a conscious intention to communicate.

That does not mean we should be careless about AI. Quite the opposite. We should care intensely about whether these systems are accurate, reliable, biased, overconfident, or prone to hallucination. We should notice when human judgment is indispensable and care about whether people become dangerously dependent on systems they do not understand. Those are substantive questions. But saying that AI-generated language is not really language because it comes from an algorithm does not answer any of them. It simply redescribes how the tool works.

Human beings have always judged tools in large part by what they enable us to accomplish. An airplane does not flap its wings like a bird, but it flies. A submarine does not swim like a fish, but it moves underwater. A camera does not see a sunset as a person sees it, but it takes a photograph. If an AI system produces a recognizable drawing of a unicorn, the resulting image is still a drawing of a unicorn. We do not need to believe that the machine contemplated the Unicorn’s beauty before we are allowed to recognize the output.

The same is true of language. Human language undoubtedly contains dimensions that machines do not possess. Human beings speak out of memory, love, grief, experience, intention, humor, loyalty, fear, faith, and countless other features of conscious life. That matters enormously. But language also conveys information, explains concepts, issues warnings, and helps people solve problems. If a machine successfully performs those functions, insisting that nothing really happened because there was no consciousness behind the words becomes less a defense of human uniqueness than a refusal to describe the function honestly.

Marinaccio is on firmer ground when he warns against confusing access to information with wisdom. His concern that technology can weaken habits of memory, sustained reading, and reflection is worth taking seriously. A person who can instantly retrieve an answer does not necessarily understand the subject. A student who outsources every difficult intellectual task may never develop the capacities that make genuine learning possible. AI could certainly worsen those tendencies if we use it as a substitute for thinking rather than as an aid to thinking.

But that is precisely why the distinction matters. The danger of AI is not that its useful outputs are somehow unreal. The danger is that real and useful tools can still be used badly. A calculator can make arithmetic easier without teaching mathematical reasoning, and so we insist that children learn how to do arithmetic before they get to use calculators. A GPS can get us to our destination while weakening our understanding of the local geography, and so it is still important to look at maps. Search engines can put enormous amounts of information at our fingertips without making us wise, we must understand their output and not merely parrot it. AI is no different in principle. The fact that a technology genuinely performs a function does not mean that every way of using it strengthens the human beings who rely on it.

That is a much more compelling human-centered critique of AI. We do not need to deny that machines can calculate, communicate, explain, or teach in order to see that humans must continue to develop and maintain their understanding, judgment, and wisdom, because a person’s own capacities always remain irreplaceable to themselves. No one can be responsible for your decisions, thinking, and actions on your behalf, no matter what tools you may use. In fact, the stronger argument begins by acknowledging what the technology can actually do and then asking how human beings should use it.

Plato’s warning is worth revisiting in the age of artificial intelligence. Marinaccio is right to ask whether technological convenience can tempt us into mistaking information for knowledge and knowledge for wisdom. But we should not defend human uniqueness by pretending that useful machine functions are somehow unreal. A machine does not have to be human to perform a real and useful task, and it does not have to possess consciousness for its output to convey something true. The important question is not whether the machine really thinks it is helping us. The important question is whether we are wise enough to use what it gives us well.