I see a distinct risk of Goodharting here. At a small scale, one of the main contributors to altruism is empathy, so this would load heavily on empathy. But in the sort of large-scale, senior positions you discuss, empathy is significantly less valuable as a predictor of goodness.
There's another failure mode similar but not quite the same as (C) which is: Perhaps certain antisocial traits necessarily correspond to certain socially useful traits; e.g. a neutral "ambition" trait corresponding to antisocial competitiveness but also prosocial grit.
My main worry here is that it becomes yet another mechanism by which the effects of discrimination and mistreatment are used to justify continuing discrimination and mistreatment. It often really sucks to be around people who are themselves going through a tough time, as I'm sure you've noticed if you've had your packages stolen by homeless people, poured a seemingly endless amount of one-sided compassion and support into a stubbornly suicidal friend, or fostered a traumatized child. I certainly was able to observe my own prosociality score plummet over the course of several years of chronic illness. And something that I've noticed is that the antisocial ways in which people cope with hardship often form the backbone of arguments to continue or intensify the difficult conditions they're experiencing. So I would be terrified of a situation where "[group] is empirically less prosocial, therefore they deserve to have less [material resources / human dignity / autonomy over their own lives] than others" becomes a common refrain.
I think theres a kind of interaction between false positives, and C). There are many ways your brain could be set up to end up acting good or bad, and this will detect the ones which are 1) common and 2) the outcome not sensitive to small details. What Id expect the science to find is: i) normies, where prosociality is relatively predictable but also doesnt vary that much ii) various abnormalities, which could go badly, or ok, or maybe really well, details beyond detection iii) conditions that make almost invariably evil people. And I dont think there are a lot of 3s, far less than 10%, and theyre propably in prison already, because they cant even be superficially ok. So basically all the benefit would come from selecting out ii).
My suspicion is probably the biggest problem is C. The world is already full of bad actors, and the people best at dealing with them are...other bad actors! You see this in games as simple as the prisoner's dilemma. You can invent all kinds of thought experiments but IRL if you're too nice people tend to take advantage of you. Of course if you're too mean and don't disguise it well they turn on you, which is why we have all the shifting social equilbria we do.
I see a distinct risk of Goodharting here. At a small scale, one of the main contributors to altruism is empathy, so this would load heavily on empathy. But in the sort of large-scale, senior positions you discuss, empathy is significantly less valuable as a predictor of goodness.
@philosophybear I'm not sure whether you can delete replies, but you've got a spammer.
There's another failure mode similar but not quite the same as (C) which is: Perhaps certain antisocial traits necessarily correspond to certain socially useful traits; e.g. a neutral "ambition" trait corresponding to antisocial competitiveness but also prosocial grit.
My main worry here is that it becomes yet another mechanism by which the effects of discrimination and mistreatment are used to justify continuing discrimination and mistreatment. It often really sucks to be around people who are themselves going through a tough time, as I'm sure you've noticed if you've had your packages stolen by homeless people, poured a seemingly endless amount of one-sided compassion and support into a stubbornly suicidal friend, or fostered a traumatized child. I certainly was able to observe my own prosociality score plummet over the course of several years of chronic illness. And something that I've noticed is that the antisocial ways in which people cope with hardship often form the backbone of arguments to continue or intensify the difficult conditions they're experiencing. So I would be terrified of a situation where "[group] is empirically less prosocial, therefore they deserve to have less [material resources / human dignity / autonomy over their own lives] than others" becomes a common refrain.
I think theres a kind of interaction between false positives, and C). There are many ways your brain could be set up to end up acting good or bad, and this will detect the ones which are 1) common and 2) the outcome not sensitive to small details. What Id expect the science to find is: i) normies, where prosociality is relatively predictable but also doesnt vary that much ii) various abnormalities, which could go badly, or ok, or maybe really well, details beyond detection iii) conditions that make almost invariably evil people. And I dont think there are a lot of 3s, far less than 10%, and theyre propably in prison already, because they cant even be superficially ok. So basically all the benefit would come from selecting out ii).
I am like 90% sure I read The Truth Machine back in the 1990s, but cannot remember much of how the plot plays out.
https://coins.ha.com/information/ttm.s
My suspicion is probably the biggest problem is C. The world is already full of bad actors, and the people best at dealing with them are...other bad actors! You see this in games as simple as the prisoner's dilemma. You can invent all kinds of thought experiments but IRL if you're too nice people tend to take advantage of you. Of course if you're too mean and don't disguise it well they turn on you, which is why we have all the shifting social equilbria we do.