This is not a reply in the sense of a refutation- I doubt I will say anything here SA would disagree with. Rather, it is an extension or meditation. Scott Aaronson has written a prose poem called “We Are the God of the Gaps”. I quite like it, it occupies an odd space between a traditional prose poem and an essay. The thesis of the poem is that machines will gobble up all human tasks. Finally, the last task remaining, the one thing that a computer will never beat you at, will be predicting exactly what you will do in the very next moment:
“And the rising tide of the learning machines will flood them all,
Poker to poetry, physics to programming, painting to plumbing, which first and which last merely a technical puzzle,”
For Scott, this is a melancholy prospect, because it hits at his self-esteem:
From earliest childhood, those of us born good at math and such told ourselves a lie:
That while the tall, the beautiful, the strong, the socially adept might beat us in the external world of appearances,
Nevertheless, we beat them in the inner sanctum of truth, where it counts.
Turns out that anyplace you can beat or be beaten wasn’t the inner sanctum at all, but just another antechamber,
I get it. I do. Consider progress in chess. At some point, the best player in the world stopped being a human and became a computer. Humans briefly made themselves relevant through the creation of centaur chess, a computer, and a grandmaster working together to choose moves, often with the computer doing tactics and the human doing the positional play. Then, a few years after the invention of centaur chess, computers became so good that having a human second guess its moves only made it weaker.
Sometime in the next few years- decades at most, something similar will happen in my field of writing thoughtful essays. The best essayist in the world will be a computer. Superb human essayists will briefly hold onto relevance through collaborating with computers, and then they’ll be surpassed permanently. Similar things will happen in SA’s fields of mathematics and quantum computing.
But I’ll live. So long as humanity lives as a whole.
Let me confess to some vanities. I had the immense privilege of being able to make a bid for the status of a great philosopher. Of course, I never had a chance. I’ve easily met ten philosophers who are better than me. I’d be shocked if I made the top thousand in the world, pleasantly surprised if I made the top 10,000. Still, the world humored me in trying. I got to have my shot, like a very moderately gifted district track and field athlete who dreams of one day being an olympian. It was a joy to tilt at this windmill.
(To be doubly clear- what gave me the chance to take this shot was not any exceptional talent on my part, just a strange autistic narcissism that told me it was possible.)
Scott Aaronson did one better. He not only got to make an attempt at greatness, but he also succeeded. With an H-index of 51, he got to be one of the greatest quantum computer scientists of our age- and since quantum computing is so new, of any age.
Most people never get that. Most people never get to make a real attempt to be recognized as among the very best at something that matters. The number of people who do get to make such an attempt has been shrinking since the stone age. In a band of 10 to 30 people quite possibly most people got to be the best at something that mattered- singing, fire-starting, painting, good looks, tool making, shelter building, foraging, hunting, fighting, charm. Then as the world got bigger…
Scott’s melancholy, a melancholy I share, is an example of what I have previously called writerly bias. The tendency of people- especially readers, but to a degree humanity is a whole, to perceive the world disproportionately through the mentality of a writer, because the people who do the writing are all writers. They all have a writer’s problems, solutions, desires, background, etc. A lot of writers get to make an (almost always quixotic) attempt at greatness. Far more than the general population.
Thus a lot of humans are feeling the way Scott describes about AI right now, but that’s because this emotional complex is being pushed on them by writers. The bigger issue- rarely talked about- is what most of humanity has already lost- not just the status of being “the best" at anything, but even the chance to make an attempt at being “the best” at anything.
The vast majority of people are already living in a life-world in which attempting to be the best at something is unthinkable. That’s something that was taken from them perhaps sometime between 8000 BC and1 AD.
Scott and I’s shared melancholy is a melancholy not of the beginning of a process of human loss, but of the moment of its final consummation. The end of a march that began in deep time. A march towards fewer and fewer people, as a proportion, being the best, or among the best, or even making an attempt at it.
Maybe because of that quixotic quest, you (and Aaronson) have framed this as a loss of being "the best". But I have no need of being "the best", as indeed billions of other people do just fine without it and always have. (Hunter-gatherers did not dream of being "the best"; indeed in most societies that was expressly forbidden and severely punished!)
What I have need of is being USEFUL. Of having PURPOSE. And in our current state, humans are still useful and productive, and this helps us find purpose. But when there are machines that can do better than we can at literally everything, and those machines can be cheaply reproduced as many times as they are needed, we will reach a point where we are no longer useful, where anything we might do would only be a loss in efficiency and productivity.
If AI progresses as many experts believe it will, then all too soon we will reach a point where humans are no more useful or necessary than horses became after the invention of internal combustion. And what happened to horses when we stopped finding them useful? We made a world with a lot fewer horses, and the ones we kept around were simply there for our amusement.
I don't think I want a world where we become nothing more than pets for our robot overlords, even if it would mean a much higher objective standard of living than what we have today.
Very powerful. Thank you.