Back

There are no slam dunks against LLM consciousness

I have mixed feelings about Rob Long’s article on LLM consciousness arguments:

Here’s a frustrating dialectic that I see (or imagine I see) sometimes:

LLM-Sneerer: “You think that LLMs could be conscious? Absurd! Don’t you know that LLMs are just predicting the next token? Plus, they make stuff up; their preferences are contradictory; their introspection is unreliable; and they’re confused about who they even are!”

LLM-Defender: “Aha, well…aren’t humans just predicting the next token? Plus, humans make stuff up, and humans have contradictory preferences; our introspection is unreliable, and we’re really confused about who we are too!”

I’m glad that the Defender is pushing back, but ultimately both of them are imposing an unproductive frame on LLM-human differences.

The third position Rob is trying to carve out is:

  1. Yes, LLMs are actually quite different from humans, and we should take those differences seriously in our theorizing. Differences in substrate, evolutionary history, embodiment, etc. might turn out to be really important!
  2. But just pointing out that LLMs are flawed or different in some way doesn’t give you grounds to dismiss the possibility of their consciousness out of hand—and most Sneerer arguments are remarkably shallow.

This is eminently sensible, it is the position that I endorse, and I think the consciousness debate would be much better off if people thought more like Rob Long. So why do I feel some hesitance about this article?

Coming from my personal interactions on substack, seeing many of these disagreements, I think Rob’s portrayal of the LLM Defender is a little ambiguous. Imagine following up on this:

Aren’t humans just predicting the next token? Plus, humans make stuff up, and humans have contradictory preferences; our introspection is unreliable, and we’re really confused about who we are too!

with this:

(A) Therefore, your arguments are much too crude. Since they apply to both humans and LLMs, you aren’t really drawing a distinction between the two of them. You need to be more specific with (i) what precisely is different between LLMs and humans, (ii) why this difference is likely to be load-bearing for consciousness, and (iii) why you think this is so likely it justifies dismissing LLM consciousness out of hand, despite widespread expert confusion about what consciousness even is.

versus this:

(B) Therefore, we don’t have any good reason to think LLMs aren’t conscious; we work basically the same way, so there are no consciousness-relevant differences.

I think some real-world Defenders are saying (A), and some are saying (B), and Rob’s imagined quote really doesn’t distinguish between the two. (B) is silly. But (A) is pretty sensible, and the Defender’s arguments are really important for establishing (A). (A) is not equating LLMs to humans, it’s trying to argue that Sneerer arguments prove too much.1

The basic principle which arguments of type (A) support probably deserves a name. In honor of “no free lunch” principles in computer science, I propose the “no slam dunks” conjecture: there are no “slam dunk” arguments for or against LLM consciousness.

By a “slam dunk,” I mean a compelling argument which requires minimal nuance/analysis of differences between humans and LLMs, but which is so effective as to dismiss some stance on consciousness out of hand. A “brick” is an uncompelling argument of the same type. Typically, consciousness bricks fail because they prove too much or fail to explain how some substantive observable difference entails a difference in consciousness.

“LLMs can’t feel anything, they’re just machines” is a brick. What’s a machine? What makes an LLM count as a machine and not a human? Why would being a machine disqualify an entity from consciousness? In order to provide an argument, you actually need to specify answers to these questions, which requires (gasp!) actually reading things about how computers and brains work and doing comparative analysis! The horror!

Or, to put it another way: if you want to say “LLMs can’t feel anything because they are machines,” you have to provide a reason why machines can’t feel anything but humans can. If you want to say “machines can’t feel anything because they are made of switches and wires”, you have to provide a reason why switches and wires can’t create feeling but neurons can. To do so, you actually have to look at properties which neurons have and switches and wires cannot, and give plausible reasons why those properties are necessary for consciousness. Now we’re firmly out of slam dunk territory; you actually have to do nuanced neurophilosophy to succeed here.

“LLMs are just token predictors” is also a brick, for reasons Rob explains quite well:

[E]ven granting that LLMs are doing next-token prediction, it doesn’t follow that LLMs are “just” doing that. As Grzankowski, Keeling, Shevlin and Street 2025 argue, that “just” can smuggle in a dubious argument: that the low-level, token-prediction explanation invalidates any other higher-level explanations of LLM behavior, like explaining their behavior with belief or intentions. But in general, low-level explanations don’t crowd out high-level explanations. Money is “just” a bunch of paper and metal and electronic records, but this low-level explanation doesn’t invalidate higher-level explanations, e.g. using prices to explain what people buy.

So again, if you want to say “LLMs are just next-token predictors, therefore they aren’t conscious,” you actually have to explain why the low-level description of next-token prediction rules out a high-level explanation in terms of thoughts, feelings, experiences, etc. Otherwise, I could just say “well, your brain is just computing the next neural response, therefore you aren’t conscious.” Again, maybe there is a detailed, successful argument here, but certainly no slam dunk.

Why do I believe there are no slam dunks? Several reasons:

  • I have seen many attempts at slam dunks for or against LLM consciousness. All of them have turned out to be bricks on closer inspection.
  • There appears to be widespread disagreement and confusion among expert philosophers about what consciousness is; there isn’t even an agreed-upon folk definition. Therefore, any slam dunk argument for/against LLM consciousness would need to either (a) succeed for a wide range of disparate theoretical positions, or (b) also be a slam dunk against almost all of those positions. This seems implausible.
  • If there were a slam dunk argument, then a professional philosopher of mind is more likely to discover it than a random substacker. But I have not seen any slam dunks from professional philosophers of mind. The closest thing to a general attempted slam dunk against machine consciousness is Searle’s Chinese room, to which there are many replies. Even if Searle’s argument is correct, it seems to require further argumentation to be convincing, and is therefore out of slam dunk territory.
  • Consciousness doesn’t seem to be the kind of thing where slam dunks would likely exist. For instance, either consciousness is fundamentally private, or it isn’t. If it’s fundamentally private, then how can you be confident LLMs are/aren’t conscious just by looking at their physical observables? It seems you could only gather abductive evidence. If it isn’t fundamentally private, then one would actually need to analyze how LLMs work in detail and compare them to how brains work in detail to see what it is that brains have and LLMs don’t (and why that thing would be consciousness-relevant). But then you’re no longer in slam dunk territory.

Am I beating a strawman here? Among most professional philosophers, yes. They are smart and well-read enough to realize that consciousness is difficult to study and requires precise argumentation, not flashy pro/anti-machine slogans. Even skeptical professionals like John Searle, Anil Seth, and Ned Block do their homework. I think Erik Hoel is wrong and deeply confused about consciousness, but he actually provides a substantial argument for his position.

But among all those who read and write substack phil-mind… sadly, no. I’ve seen loads of bricks presented as though they are devastating arguments which one would have to be an idiot not to see.

Now, I don’t think I’ll succeed at persuading these people that looking for slam dunks is useless. Convincing dogmatic die-hards is rarely the point of argument anyway. But I do think I stand a chance of convincing those who haven’t made up their mind yet not to waste time with snappy one-liners, chasing a kind of instant-gratification certainty which doesn’t exist.

  1. Man, how long has it been since I’ve written “it’s not X, it’s Y?” It makes me a little sad that I cringe at this now. ↩