Okay, I decree that the dust specks have exactly zero knock-on effects; in each of the countless individual cases the total effect on utility at any timescale is the temporary discomfort alone. Assume a gazillion combo dust speck/amnesia events if you want. What makes Yudkowsky's argument interesting in the first place is that he (and presumably you) would still choose fifty years of torture at some number of these dust specks. You say that "there's a general important point here- even if higher order 'overall' functioning is what matters, and not moments of discomfort or even pain- and I am somewhat sympathetic to that view- moments of discomfort still negatively affect overall functioning with some probability, and so can still be traded off with them." But make that probability zero and nothing about your argument should change. There's still some number of dust specks that wins the utility comparison.
Later, you respond to a similar argument with "this cuts both ways. If we have a full assurance that the repairman will suffer zero overall consequences to his life- just 90 minutes of agony and then it’s done- no further ill effects whatsoever, once that assurance is given in full, I think it becomes a lot more tempting not to disrupt the broadcast." It doesn't have to cut both ways. Just construct the hypothetical like I did so that it only cuts one way, this doesn't change the utility math at all it just makes the necessary number a little bigger. By admitting that overall functioning plausibly matters and resorting to small probabilities of small impacts having large knock-on effects, you're undermining the original argument. If utility is a perfect common currency, truly meaningless tiny events with the measliest bit of negative utility and zero further negative consequences will eventually outweigh Hell itself if the numbers add up. And that's why this argument is fundamentally wrong. Yes, utility exists, but there must be some lexical priority consideration as well.
Sure, if you go that way I just say you shouldn't maximize utility, you should maximize something more like "overall functioning" or eudaimonia, which is consistent with the kind of (approximate) utilitarian that I am, focused on something more like a eudaimonic or objective list theory of the good. I think I state above that I am sympathetic to such approaches to utilitarianism, and I previously covered my view here:
The main purpose of this article is to argue utility does exist, but I agree, in the limit, there is an argument that utilitarianism should prefer something other than utility as its yardstick of wellbeing. However, as far as arguments that prove you should diverge from classic utilitarianism to something more like Eudaimonia utilitarianism, I consider this one not especially strong, since unlike in, say, the "tiling the universe" case, the intuition is far from overwhelming, and it never really applies in the real world, as there's always the possibility of making a functional difference with a small negative intervention here.
For me, I don't believe in utility for a few reasons, including:
(1) I don't think preferences are transitive (which is necessary for there to be a common currency)
It is easy to construct hypothetical and arguably realistic scenarios where people have logical rules in applying preferences that are not transitive, and I find it psychologically plausible that people could (unconsciously) apply these rules. Basically, all you need to do is add in more than one dimension/quality that is relevant to the person's preference, which sounds super plausible to me.
(2) I think how desirable a specific state of affairs is, is a relative judgement that depends on a number of factors including the identity of the evaluator, the time and context of the evaluation, the time and context of the state of affairs being rated, etc. So I don't think there is an eternal, single quantity to how desirable a state of affairs is. For an example, think of that famous story with the person breaking their leg and then not being drafted and then...
I got Bing to elaborate a bit more on what I mean by point 1:
Sure, here is how I would write out this example:
Item Quality 1 Quality 2
A 10 1
B 9 5
C NA 3
The logical rule for preferences is:
If an item has a rating for quality 1, then prefer the item with the higher rating for quality 1.
If an item does not have a rating for quality 1, then prefer the item with the higher rating for quality 2.
This rule results in intransitivity because:
A is preferred to B by rule 1 (10 > 9).
B is preferred to C by rule 2 (5 vs 3).
C is preferred to A by rule 2 (3 > 1).
An example of a situation where these preferences and rules might be plausible is:
The items are different types of books.
Quality 1 is the soothingness of the voice of the narrator in an audiobook (rated from 1 to 10, with 10 being the most soothing).
Quality 2 is the excitingness of the plot of the book (rated from 1 to 5, with 5 being the most exciting).
The person prefers more soothing voices to less soothing voices to the exclusion of all else when applicable, but otherwise has a preference for more exciting plots. Item 1 and 2 could be audiobooks, while item 3 is a regular book.
Okay, I decree that the dust specks have exactly zero knock-on effects; in each of the countless individual cases the total effect on utility at any timescale is the temporary discomfort alone. Assume a gazillion combo dust speck/amnesia events if you want. What makes Yudkowsky's argument interesting in the first place is that he (and presumably you) would still choose fifty years of torture at some number of these dust specks. You say that "there's a general important point here- even if higher order 'overall' functioning is what matters, and not moments of discomfort or even pain- and I am somewhat sympathetic to that view- moments of discomfort still negatively affect overall functioning with some probability, and so can still be traded off with them." But make that probability zero and nothing about your argument should change. There's still some number of dust specks that wins the utility comparison.
Later, you respond to a similar argument with "this cuts both ways. If we have a full assurance that the repairman will suffer zero overall consequences to his life- just 90 minutes of agony and then it’s done- no further ill effects whatsoever, once that assurance is given in full, I think it becomes a lot more tempting not to disrupt the broadcast." It doesn't have to cut both ways. Just construct the hypothetical like I did so that it only cuts one way, this doesn't change the utility math at all it just makes the necessary number a little bigger. By admitting that overall functioning plausibly matters and resorting to small probabilities of small impacts having large knock-on effects, you're undermining the original argument. If utility is a perfect common currency, truly meaningless tiny events with the measliest bit of negative utility and zero further negative consequences will eventually outweigh Hell itself if the numbers add up. And that's why this argument is fundamentally wrong. Yes, utility exists, but there must be some lexical priority consideration as well.
Sure, if you go that way I just say you shouldn't maximize utility, you should maximize something more like "overall functioning" or eudaimonia, which is consistent with the kind of (approximate) utilitarian that I am, focused on something more like a eudaimonic or objective list theory of the good. I think I state above that I am sympathetic to such approaches to utilitarianism, and I previously covered my view here:
https://substack.com/inbox/post/128428596
The main purpose of this article is to argue utility does exist, but I agree, in the limit, there is an argument that utilitarianism should prefer something other than utility as its yardstick of wellbeing. However, as far as arguments that prove you should diverge from classic utilitarianism to something more like Eudaimonia utilitarianism, I consider this one not especially strong, since unlike in, say, the "tiling the universe" case, the intuition is far from overwhelming, and it never really applies in the real world, as there's always the possibility of making a functional difference with a small negative intervention here.
Yep I’ve read your blog for a while and think we have very similar views on ethics
For me, I don't believe in utility for a few reasons, including:
(1) I don't think preferences are transitive (which is necessary for there to be a common currency)
It is easy to construct hypothetical and arguably realistic scenarios where people have logical rules in applying preferences that are not transitive, and I find it psychologically plausible that people could (unconsciously) apply these rules. Basically, all you need to do is add in more than one dimension/quality that is relevant to the person's preference, which sounds super plausible to me.
(2) I think how desirable a specific state of affairs is, is a relative judgement that depends on a number of factors including the identity of the evaluator, the time and context of the evaluation, the time and context of the state of affairs being rated, etc. So I don't think there is an eternal, single quantity to how desirable a state of affairs is. For an example, think of that famous story with the person breaking their leg and then not being drafted and then...
I got Bing to elaborate a bit more on what I mean by point 1:
Sure, here is how I would write out this example:
Item Quality 1 Quality 2
A 10 1
B 9 5
C NA 3
The logical rule for preferences is:
If an item has a rating for quality 1, then prefer the item with the higher rating for quality 1.
If an item does not have a rating for quality 1, then prefer the item with the higher rating for quality 2.
This rule results in intransitivity because:
A is preferred to B by rule 1 (10 > 9).
B is preferred to C by rule 2 (5 vs 3).
C is preferred to A by rule 2 (3 > 1).
An example of a situation where these preferences and rules might be plausible is:
The items are different types of books.
Quality 1 is the soothingness of the voice of the narrator in an audiobook (rated from 1 to 10, with 10 being the most soothing).
Quality 2 is the excitingness of the plot of the book (rated from 1 to 5, with 5 being the most exciting).
The person prefers more soothing voices to less soothing voices to the exclusion of all else when applicable, but otherwise has a preference for more exciting plots. Item 1 and 2 could be audiobooks, while item 3 is a regular book.