LLM truthers keep trying different ways of saying "But how do we KNOW that using statistical regression to create a string of tokens that resemble an answer to a prompt is not the same thing as actual thought, though?"
And the answer is always: because it isn't.
There's a lot we don't know about human cognition and the stuff we do know is more complicated than how our own cognitive processes appear to us.
Like, it does involve predictive processes and some of it is essentially random. The same person can come to a different conclusion at different times.
If you ask a person what their favorite X is and they've never thought about it, the answer they come up with might not be the same answer they would have come up with had you asked the question differently, at a different time or in a different way.
