A challenge for AI sceptics
By AI skeptic here I mean someone who thinks the current wave of machine learning research is much less significant than the field of machine learning would want you to think. I’m aware that this is a vague definition, but AI skeptic is a broad category. Being an AI skeptic in this sense is orthogonal to being concerned about AI, though they are sometimes conflated in one way or another.
There’s a longstanding debate between different approaches to the mind, of course, and like any good experiment, the contemporary wave of results from machine learning has stepped on a lot of toes. Many people, it seems to me, feel disappointed by the success of the brute connectionist approach of contemporary machine learning (e.g. Gary Marcus, Steve Pinker). Machine learning models feel a lot like naive empiricism about the mind, they even have a bit of the flavor of behaviorist models, as were rejected so memorably by the field during the cognitive revolution (C.f. Chomsky). Other people do not like the success of contemporary machine learning models because they are not particularly ‘embodied’ or ‘enactivist’. In the specific area of language models, some people, (e.g. Emily Bender), do not like that LLM symbols aren’t grounded in extra-textual concepts in the way symbols used by humans are. Other people don’t like machine learning because it’s very engineering-based, and they prefer a theory-driven approach.
I have some sympathy for these critiques. It does seem like more explicit processing may help and it does seem like grounding the symbols of LLM’s may help. But I am sick of moving goalposts and imprecise head shaking and eye-rolling, and I think dismissing the capacities of these machines can leave us dangerously unprepared for the harm they might do. That’s why I’m issuing the following challenge, focused on LLMs because that’s where much of the debate is:
Give me a task, concretely defined and operationalized, that a very bright person can do but that an LLM derived from current approaches will never be able to do. The task must involve textual inputs and outputs only, and success or failure must not be a matter of opinion.
I want people to be more explicit about what they think the exact and unalterable failings of contemporary large language models are. Give me something exact that this current approach cannot do.
Of course, I’m not so unjust as to demand a task that is written out in advance, this could become part of an AI’s training data, and thus allow the AI to cheat but the task must be concretely specified. Also:
No cheating by creating tasks that definitionally require or advantage humans, e.g. “truthfully describe your experiences growing up as a child”.
No tasks that depend on intangibles inside the machine itself e.g. “say something with real feeling”.
No tasks that are beyond the reach of a very bright person e.g. “never hallucinate”.
There are of course tasks that current models can’t do. I want to know tasks that models built along the same lines as current models will never be able to do.
It’s not that I don’t believe such tasks exist, it’s just that I want to know exactly what they are.
If you want to give yourself an easy mode, I’ll also accept predictions of things that LLM’s along the lines of current models will not be able to do within five years. Also, if you feel some aspect of the rules I’ve outlined is unfair, free to make your case as to what a fairer operationalization would be.
Edit: Here are a few examples of bars one might propose. I am not suggesting any actual AI skeptic would endorse these, but they are examples of challenges that would meet my criteria.
1. Score 80%+ On a maths Olympiad from a year past its training data.
2. Write a literary novel that convinces at least 10/20 critics that it is a human-written literary novel in competition with a real novel. (The ideal form is probably to have in compete against 9 real novels, and convince critics it is in the top half, vis a vis quality, or likelihood of being human, or both).
3. Write a philosophy paper and get it published at a top or second tier journal, on the proviso it is allowed to lie about being a human.
We're going to play a series of games I just invented. You've never seen these games before. Some of them will be party games, some of them will model board games in text, some will be probabilistic, etc. I'm going to give you the rules, and then let's play! We're going to communicate by passing our moves back and forth, without commentary on the game state. Feel free to use a notepad or memory buffer.
If an LLM had never heard of the concept of chess, and you explained the rules and said:
"1. e4"
I think there's a good chance the LLM would lose to a _very bright_ human even in five years (or at least, make less illegal moves). And I wouldn't be surprised if after ten games, the human had improved more than the AI.
As I understand the way they work, it would be impossible for an LLM to come up with a genuinely original idea, or turn of phrase. If it's not in the training set, it doesn't exist.
I've had no success so far in getting ChatGPT to imitate my own style, but I imagine with a large enough training set it could produce something semi-convincing.