Regarding risks existential, bias, and domination, I believe 3 deserves not just more of our attention, but most of it. 1 is just too hard, we'll most likely have to get extremely lucky and end up in a universe where it just isn't that hard to make an aligned superintelligence. Next most likely thing that saves us is coordinating well enough to buy a whole lot of time, followed by incredible theoretical advances in a short amount of time. Neither of these is probably happening but I believe there's a non negligible chance of getting lucky. 2 could be bad but the magnitude of harm is dwarfed by the other two. 3, as pointed out in the essay, includes the possibility of near infinite negative utility, horrors not just unprecedented but beyond understanding. And while not easy it seems much easier to attack than alignment. Considering it's conditional on 1 being a non issue, it might be as easy as making sure the most powerful AIs we create are under some reasonable form of democratic control, or are independent of human control and non awful.
I think it's quite possible alignment will turn out to be a hard but solveable problem with failure modes below "destroys the world" largely because I think that there will be a lot of entities smart enough to do considerable damage, but not to destroy the world.
This is because I believe there's a large gap between "as smart as the smartest human" and "smart enough to trigger a fast takeoff singularity".
The solution to 3 is partly political- democratizing society.
yo philosophy bear , spd here, love your content and your way of writing, been subbed to u for a quite a while , i recently started a substack , so wud love your suggestions on how to start as beginners and on how to expand my newsletter.
I'm probably not the best person to ask about this because I have only 1000 or so subscribers and no paid subscribers, which is not much, but my main advice would be:
1. Of all the places I've shared my content, Reddit has been the best- you have to be careful though- you don't want to offend sensibilities by sharing in places it's not welcome. Post about other things from your account as well.
2. You should aim to write a lot, for many different reasons. It will in the long run it will increase your audience and make you a better writer, which will in turn increase your audience. Set a word target each day.
3. Try to read voraciously to match your writing. Book reviews and responses to books are always a good option when you have nothing else going.
Regarding risks existential, bias, and domination, I believe 3 deserves not just more of our attention, but most of it. 1 is just too hard, we'll most likely have to get extremely lucky and end up in a universe where it just isn't that hard to make an aligned superintelligence. Next most likely thing that saves us is coordinating well enough to buy a whole lot of time, followed by incredible theoretical advances in a short amount of time. Neither of these is probably happening but I believe there's a non negligible chance of getting lucky. 2 could be bad but the magnitude of harm is dwarfed by the other two. 3, as pointed out in the essay, includes the possibility of near infinite negative utility, horrors not just unprecedented but beyond understanding. And while not easy it seems much easier to attack than alignment. Considering it's conditional on 1 being a non issue, it might be as easy as making sure the most powerful AIs we create are under some reasonable form of democratic control, or are independent of human control and non awful.
I think it's quite possible alignment will turn out to be a hard but solveable problem with failure modes below "destroys the world" largely because I think that there will be a lot of entities smart enough to do considerable damage, but not to destroy the world.
This is because I believe there's a large gap between "as smart as the smartest human" and "smart enough to trigger a fast takeoff singularity".
The solution to 3 is partly political- democratizing society.
yo philosophy bear , spd here, love your content and your way of writing, been subbed to u for a quite a while , i recently started a substack , so wud love your suggestions on how to start as beginners and on how to expand my newsletter.
I'm probably not the best person to ask about this because I have only 1000 or so subscribers and no paid subscribers, which is not much, but my main advice would be:
1. Of all the places I've shared my content, Reddit has been the best- you have to be careful though- you don't want to offend sensibilities by sharing in places it's not welcome. Post about other things from your account as well.
2. You should aim to write a lot, for many different reasons. It will in the long run it will increase your audience and make you a better writer, which will in turn increase your audience. Set a word target each day.
3. Try to read voraciously to match your writing. Book reviews and responses to books are always a good option when you have nothing else going.
thank you very much
Minor correction: it's spelled Peter Thiel