A Dynamic WAR on EDT

games and decisions unpublished

A common argument for favoring Evidential Decision Theory (EDT) over Causal Decision Theory (CDT) is that EDT has predictably higher expected returns in Newcomb Problems. But this doesn’t show much. For almost any pair of theories you can come up with cases where one does, on average, better than the other. Here I describe three cases involving dynamic choice where EDT predictably does worse than CDT.

Brian Weatherson http://brian.weatherson.org (University of Michigan)https://umich.edu
11-15-2021

Here is a common form of argument against causal decision theory, as described by Ahmed and Price (2012). (I’ve slightly changed some of the wording, but otherwise this argument is taken directly from page 16 of their paper.)

  1. In Newcomb problems, the average returns to one-boxing exceed that to two-boxing.
  2. Everyone can see that (1) is true.
  3. Therefore one-boxing foreseeably does better than two-boxing. (by 1, 2)
  4. Therefore Causal Decision Theory (CDT) is committed to the foreseeably worse option for anyone facing Newcomb’s problem.

Here’s what they, and many other proponents of Evidential Decision Theory (EDT) say follows from 4.

The point of the argument is that if everyone knows that the CDT-irrational strategy will in fact do better on average than the CDT-rational strategy, then it’s rational to play the CDT-irrational strategy. (Ahmed and Price 2012, 17)

This is what Lewis (1981) called the “Why Ain’cha Rich” argument, and what following Bales (2018) I’ll call the WAR argument. I’m going to argue this last step of the WAR argument doesn’t follow. Or, at the very least, that proponents of EDT cannot coherently say that it follows. For there are several cases where EDT foreseeably does worse than CDT. This paper will go over three of them.

Example One - Split Newcomb

This game takes place over three rounds. I’m calling the examples games from now on because, following Harper (1986), I take it is best to think of the original Newcomb Problem as a game. The demon is an actor who considers what the other player is trying to do, and has incentives that explain their decision given their beliefs about the other player. In this case the incentive is to correctly predict what the other player does. That sounds like the demon is a player in a traditional game-theoretic setup. So I’ll make it explicit that that’s what the demon is. With that said, here is the three stage game.

At stage one, the human player chooses In or Out. If they choose Out, player gets 5 and demon gets 1. If they choose In, we move onto stage two.

At stage two, demon chooses Left or Right, and this choice is announced.

At stage three demon and the player simultaneously choose either Up or Down. Demon is very good at predicting what player’s choices will be, and indeed at stage two they were already very good at making such a prediction. And demon wants to use these predictive powers to get as high a payoff as possible, and this is common knowledge.

If demon chose Left at stage two, here is the payoff table at stage three. Player chooses the row, demon chooses the column, so between them they determine a cell, and in that cell I’ll list player’s payoff followed by demon’s payoff.

Up Down
Up (2, 1) (4, 0)
Down (1, 0) (3, 3)

If demon chose Right at stage two, here is the payoff table at stage three.

Up Down
Up (12, 4) (14, 0)
Down (11, 0) (13, 2)

In both games, the CDTer will choose Up, and the EDTer will choose Down. These are both fairly straightforward Newcomb Problems from players perspective, after all. In both games Up causally dominates Down, but Down will get a higher return if you assume, as we did assume, that demon mostly makes correct predictions.

So at stage two, demon will know that if the person facing them is an EDTer, they will get a return of 3 from Left and 2 from Right. (They’ll end up in the Down-Down cell either way.) So they will rationally choose Left. On the other hand, if the person facing them is a CDTer, they will get a return of 1 from Left and 4 from Right. (They’ll end up in the Up-Up cell either way.) So they will rationally choose Right. And everything in this paragraph can be deduced by a rational player at stage 1.

So at stage one, a CDTer will know that if they play In, they expect to get 12 (the game will go Right then Up-Up), and if they play Out, they know they’ll get 5. So they’ll play In. But an EDTer will know that if they play In, they expect to get 4 (the game will go Left then Down-Down), and if they play Out, they know they’ll get 5. So they’ll play Out.

The result of all this is that the CDTer will get 12, and the EDTer will get 5. So the CDTer will predictably do better than the EDTer. Indeed, the EDTer will voluntarily choose at stage one to take a lower payout than the CDTer ends up with. This seems bad for EDT, at least if we think that predictably ending up with a lower outcome is bad.

Now you might object that this is because at stage two the demon chooses to treat the EDTer differently to how they treat the CDTer. I don’t really agree for two reasons, though I’m not sure either of these reasons work. (Hence the second and third examples that are about to come.) One is that the demon isn’t trying to harm the EDTer; they are just trying to maximise their return. It so happens that EDT is such an impermissive theory that it doesn’t allow for any flexibility, and the demon, knowing this, is forced to take choices that are bad for EDT (and it turns out for the demon). But this isn’t the demon’s fault; it’s the fault of EDT being so impermissive. The other reason is that the demon does not in fact make any choices that hurt the EDTer. The EDTer should expect that the demon will in fact make such choices, in response to their theory, but that’s not quite the same thing. The only player who moves at all in the EDT version of the game is the player. So it’s a little hard to say this is just a case where the EDTer is harmed by the demon’s malicious choices.

I think those responses work, but I’m not completely sure that they do. So let’s look at a second example, one where the demon doesn’t have these variable payouts.

Example Two - Coins and Signals

This example is a version of a signaling game of the kind introduced by Lewis (1969). And in particular it’s a version of the broadly adversarial kinds of signaling games that are central to the plot of Cho and Kreps (1987). Again, it will involve three stages.

At the first stage a fair coin is flipped, and the result shown to the human player, but not the demon.

At the second stage, the human will choose Up or Down, and the choice will be publicly announced.

At the third stage, the demon will try to guess what the coin showed. The demon knows the payoff table I’m about to show you, and is arbitrarily good at predicting the human’s choice dispositions. That is, the demon can make accurate predictions of the form “If Heads, the human will make this choice, and if Tails, they will make that choice.”

The payoffs to each player are a function of what happens at each of the three steps, and are given by the following table.

Coin Human Demon Human Payoff Demon Payoff
H U H 40 1
H U T 400 0
H D H 0 1
H D T 0 0
T U H 40 0
T U T 28 1
T D H 28 0
T D T 36 1

The demon’s payoffs are just as you’d expect - they get rewarded iff they figure out how the coin landed. The human’s payoffs are more complicated, but the big thing to note is they get the biggest rewards if they manage to play Up while the demon makes an incorrect prediction.

One last thing about the demon before we analyse the game. If the demon predicts the human will do one thing if Heads and another if Tails, they will use the information from the human’s choice to make their guess about how the coin landed. But if they predict the human will say the same thing whether the coin landed Heads or Tails, they won’t know how the coin landed, and will flip their own coin to make a guess. So in that case it will be 50/50 whether the demon says Heads or Tails.

Onto the analysis. It should be fairly clear that if the coin lands Heads, the human should say Up. The worst possible return from Up is 40, the best possible return from Down is 0. This argument can be made a bit more rigorous, but I’ll leave that as an exercise for the reader, and just assume that that’s what both a CDTer and an EDTer would do, and hence what the demon would predict that they will do.

So what happens if the coin lands Tails? Given the demon will predict Up if Heads, we can work out the value of Up and Down if Tails to the EDTer. If they play Up, the demon will predict that, and hence the demon will flip a coin to choose Heads or Tails. So they have a 50/50 shot at getting either 40 or 28, and so their expected return is 34. If they play Down, the demon will predict that, and hence the demon will say Tails, and they will get a return of 36. Since 36 > 34, they will play Down if Tails.

That’s the unique solution to the game for the EDTer. They play Up if Heads, Down if Tails. The demon can figure out that they’ll do this, so will correctly guess what the coin showed. And they will get 40 if the coin landed Heads, and 36 if it landed Tails, for an expected return of 38.

What should the CDTer do? Well it turns out that there are multiple solutions that are consistent with the general spirit of CDT. It’s consistent with CDT to do exactly what the EDTer does. But it’s also consistent to say Up no matter what. Let’s go over why this is consistent. The question is whether the CDTer can endorse their decision to play Up no matter what given each way the coin could land. They can clearly endorse it if the coin lands Heads; in that case Up strictly dominates Down, and the CDTer likes strictly dominant choices. What if the coin lands Tails? Well they think they’ll play Up. So they think the demon will flip a coin to guess in this situation. So they think the expected return of Up is 34 (like the EDTer thinks), and the expected return of Down is 32. The key difference here is that when working out the expected return of a non-chosen option, the CDTer does not change the expected behavior of the demon, while the EDTer does. (That’s what is needed to validate dominance reasoning.) So this CDTer will think that even if the coin lands Tails, they would do worse on average if they switched to playing Down if Tails. So it follows that they can consistently play Up either way.

And if they do play this, the rewards are handsome. The demon won’t have any information about the coin, so the demon will flip their own coin. So lines 1, 2, 5 and 6 of the table are all equally likely to appear. So this CDTer is equally likely to get a return of 40, 400, 40 or 28, for an overall expected return of 127. And this is much higher than the 38 the EDTer is expected to receive. By changing the payout on line 2, we can make the gap in expected returns be arbitrarily large.

Now you might object that while the CDTer can do better, it doesn’t follow from CDT alone that they will (on average) do better. After all, it is also consistent with CDT that the CDTer makes exactly the same plays as the EDTer. I don’t think that matters much. The point of the WAR is to refute a theory, and if the EDTer does foreseeably worse than one kind of CDTer, that should be enough to refute them. But just in case you think this objection is stronger, we’ll include one last example.

Example Three - Coins and Newcomb

This is just like Example Two, with one twist. If the game goes Tails, Down, Tails, then we don’t immediately end the game and make payouts to the players. Instead we play another game, with a familiar structure. As always, demon is really good at predicting human’s play, and the human payouts are listed first in every cell.

Up Down
Up (20, 1) (40, 0)
Down (16, 0) (36, 1)

The EDTer will think they’ll get 36 from this game, so the example will be just like Example Two. And the EDTer will play Up if Heads, Down if Tails, for an expected return of 38.

But the CDTer will think that if the game gets to this stage, they’ll get 20. So now they think that in the original game, Up dominates Down no matter whether the coin lands Heads or Tails. So every CDTer will play Up no matter what, and get an expected return of 127.

Why The Examples Matter

I’m putting forward these examples not as part of a direct assault on EDT, but as a war on WAR. The point of the first example is that any theory whatsoever is subject to a WAR argument. That’s because for any theory whatsoever, you can construct pairs of choices like Left and Right, where the theory says to take choices that lead the demon to preferring to go Left. So for any theory whatsoever, or at least any theory that is consequentialist in the sense popularised by Hammond (1988), there is an example where the theory leads to worse returns. So any consequentialist theory is subject to an objection by WAR. It’s the paradigm of an over-generating objection.

There is perhaps something a bit interesting about the second example, though it isn’t a problem especially for EDT. What makes the second example work is that the human player is in a situation that rewards unpredictability, but EDT is decisive, and hence predictable. By ‘decisive’ here, I mean it gives a clear verdict in cases that don’t involve ties. Of course, lots of theories are decisive, and hence the behavior of a chooser following such a theory is predictable. Sometimes it is thought to be a virtue of a decision theory that it is decisive, and gives clear advice about what to do in any situation. I’m not so sure. After all, for any such theory there will be a version of example two that can be constructed so as to raise a ‘problem’ for any such theory. Is this a problem for decisive theories, or a problem for the kind of WAR argument that we see in example two? Given how bad the other WAR arguments are, and given how popular decisive theories are, we should probably say this is a problem for WAR.

While this isn’t a direct assault on EDT, it is meant to raise doubts about it. If you think WAR is the intuitively strongest argument for EDT, and it turns out to be a bad argument, that’s not good news for EDT. But whether there is a way to rescue the motivations for EDT is a question for a different, and longer, paper. What I mean to have shown here is that WAR arguments are bad. They are arguments that every theory is bad, and arguments against every theory are successful arguments against no theory.

Ahmed, Arif, and Huw Price. 2012. “Arntzenius on ‘Why Ain’cha Rich?’.” Erkenntnis 77 (1): 15–30. https://doi.org/10.1007/s10670-011-9355-2.
Bales, Adam. 2018. “Richness and Rationality: Causal Decision Theory and the WAR Argument.” Synthese 195 (1): 259–67. https://doi.org/10.1007/s11229-016-1214-x.
Cho, In-Koo, and David M. Kreps. 1987. “Signalling Games and Stable Equilibria.” The Quarterly Journal of Economics 102 (2): 179–221. https://doi.org/10.2307/1885060.
Hammond, Peter J. 1988. “Consequentialist Foundations for Expected Utility.” Theory and Decision 25: 25–78. https://doi.org/10.1007/BF00129168.
Harper, William. 1986. “Mixed Strategies and Ratifiability in Causal Decision Theory.” Erkenntnis 24 (1): 25–36. https://doi.org/10.1007/BF00183199.
Lewis, David. 1969. Convention: A Philosophical Study. Cambridge: Harvard University Press.
———. 1981. “Why Ain’cha Rich?” Noûs 15 (3): 377–80. https://doi.org/10.2307/2215439.

References