Mixing Expert Opinion

1 Introduction

As Elkin and Pettigrew (2025) point out, there are many reasons that we might want to pool probability functions. One reason they discuss, and which has been discussed elsewhere in the recent literature, is when there is a clash between two experts each of whom our protagonist is disposed to defer to.¹ So imagine that our protagonist, let’s call them Quinn, is disposed to defer to both Ava and Ben with respect to p in the sense that for any x, Quinn’s credence in p conditional on one of Ava and Ben having credence x in p and nothing else, is x. If Quinn just knows what one expert says, they’ll follow that expert. What should they do, however, if Ava and Ben disagree?

¹ Important recent papers on this include Gallow (2018), Zhang (2026), Dellsén (2020) and Wright (2025). The first two provide formal results which, I think, give us independent reason to be sceptical of the Unanimity principle discussed below. The second two make important points about the relationships between experts, a point that will become important in Section 4.

² I’m using a special case of the geometric averaging formula here. The general version is discussed in Pettigrew and Weisberg (forthcoming).

Traditionally there are two kinds of answer to this. One approach says that Quinn’s credence should be a fixed function of Ava and Ben’s. Call this the formulaic approach. There are two versions of this approach that are particularly prominent in the recent philosophical literature. The linear averaging version says that if Ava’s credence in y, and Ben’s is z, then Quinn’s credence should be \(\frac{y + z}{2}\). The (simplest form of) the geometric version says that instead Quinn’s credence should be \(\frac{\sqrt{yz}}{\sqrt{yz}\sqrt{(1-y)(1-z)}}\).² I’m mostly going to set these aside however, because this isn’t a paper about formulaic approaches.

Instead I’ll discuss the other approach, the supra-Bayesian approach. It says that Quinn should conditionalise on Ava and Ben’s credences. The problem traditionally with this approach was that it seems computationally infeasible. The problem is not the update, that’s just reading a number off a table, the problem is the antecedent demand it puts on Quinn to have conditional credences for everything that Ava and Ben might announce.³

³ For a useful overview of work on this in the twentieth century, see Givens and Roback (1999). In the literature they discuss, the standard taxonomy is a three-way distinction between the linear, logarithmic and supra-Bayesian approaches. In the classification I’m using, the first two are both versions of the formulaic approach. The binary division I’m using is taken from Baccelli and Stewart (2021), though I’ve tinkered with the terminology. What I’m calling a formulaic approach, they call a ‘pooling’ approach. I prefer to use ‘pooling’ for the general question of how to merge attitudes, which allows for the supra-Bayesian answer.

In general, that’s a huge problem. But there are some special cases where it can be solved. And in some of those cases, it’s possible to make some very simple generalisations about the approximate value of the solutions. Those generalisations are, I hope, interesting in their own right, and a useful constraint on what kind of formulae we might use if we are thinking of the formulaic approach as a non-ideal approximation to the ideal supra-Bayesian approach.

The cases I’ll discuss all have the following form. Quinn regards Ava and Ben as experts because all three of them have the same prior credences, Ava and Ben are better informed, and Quinn knows that Ava and Ben update by conditionalisation.⁴ Moreover, while Quinn doesn’t know exactly what evidence Ava and Ben have, they do know the experiment that each performed to acquire that evidence, so they can form probabilistic beliefs about the results of those experiments. As a result, Quinn’s own posterior credences can be a weighted average of what their credences would be if they had performed the experiments Ava and Ben actually performed, with the weights given by the probabilities the experiment turned out this or that way.

⁴ In Ned Hall’s terminology, Ava and Ben are database experts not analysis experts. See Hall (2004) for the distinction, and Bokros (2020) and Lane (2025) for more recent discussions about whether complete deference is ever warranted outside this special case.

⁵ This view is in turn motivated by the model of testimony as evidence sharing that Frank Jackson (1987) developed.

This paper is part of a broader project of arguing that ideally, this special case isn’t so special. That is, I think in general the ideal receiver of expert opinion updates by conditionalisation not just on what the experts say, but on hypotheses about what evidence the experts got that led them to say that.⁵ If that broader project succeeds, cases like these are really fundamental. But even if the project fails, it should be common ground that in the special case where the experts are database experts, and the receiver knows what experiments they performed, it gives the correct answers. And that’s a useful guide to, and constraint on, the correct approach.

The first point that I’ll make is that how Quinn should update when Ava and Ben disagree depends in part on the relationship between their evidence. That will be shown by working through some cases where changing that relationship changes how Quinn should merge the opinions.

The second point I’ll make is that in some realistic cases, what Ben Levinstein (2015) calls Thrasymachus’ Principle is correct. Quinn often doesn’t go too far wrong if they defer to whichever of Ava and Ben makes the strongest, ie the most surprising, claim.

2 Setup

We’re assuming that Quinn regards Ava and Ben as experts about p in the following sense.

1. If Quinn learns that Ava’s credence in p is x, and nothing else, they will change their credence in p to x.

2. If Quinn learns that Ben’s credence in p is x, and nothing else, they will change their credence in p to x.

Given that, what is the answer to this question.

3. If Quinn learns that Ava’s credence in p is y, and Ben’s credence in p is z, and nothing else, what should their credence in p become?

The supra-Bayesian says that this case, like every other case, ideally calls for conditionalisation. Formally, this means that (1) and (2) are equivalent to (4) and (5).

4. ∀x: Cr_Q(p | Cr_A(p) = x) = x
5. ∀x: Cr_Q(p | Cr_B(p) = x) = x

Where Cr_Q, Cr_A and Cr_B are Quinn, Ava and Ben’s credence functions respectively. Then (3) gets rephrased as a request for the value of

6. Cr_Q(p | Cr_A(p) = y ∧ Cr_A(p) = z)

That’s good as far as it goes, but it raises two natural questions. First, what reasonable credal functions make (4) and (5) true, and what do they tend to say about (6)? Second, given the massive work it would be for Quinn to have these conditional plans for what to do whatever Ava and Ben say, are there heuristics for approximating the value of (6) realistic cases? We’ll make some progress on both questions.

Two quick notes. First, I’m only going to look at cases where the experts are treated symmetrically. That’s a restriction, but it’s a useful one for letting us see the range of cases. Second, I’m going to be agreeing with Easwaran et al. (2016) a lot, especially in the first half of the paper. I’m ultimately going to consider some different kinds of cases to what they consider - but that’s a difference in focus, not a difference in conclusions. (They look at a bunch of kinds of cases that I won’t consider as well; it’s not like I’m going strictly beyond their work.) This paper is intended as a complement to theirs, not at all a substitute. But I think it’s a valuable complement, because I’ll show how some very realistic cases require a generalisation of their model, and make some suggestions for what that generalisation should look like.

3 Case One: Conditionally Independent Evidence

In our first case, the experts’ evidence is as independent as possible. Here’s the example we’ll use to illustrate that. Carmen has an urn with 50 marbles, 25 black and 25 white. She draws one at random and marks it with invisible ink. She has a scanner that can detect which marble is marked, but no one else can distinguish it from the other marbles. Let p be the proposition that the marked marble is white - that’s what we’ll focus on from now on.

After selecting one marble to be marked, she puts together a jar containing the marked marble and 9 other marbles drawn at random from the urn. (I’ll use ‘urn’ for where Carmen keeps all the unused marbles, and ‘jar’ for what she constructs to show the experts.) She shows that to one of the experts, let’s say Ava. She gets to inspect the jar, i.e., count how many marbles in it are white and black. She then reports to Quinn, but crucially not to Ben, her credence in p.

In this example, the next thing that happens is that Carmen takes the jar back, removes the 9 unmarked marbles, puts them back in the urn, and draws a new set of 9 marbles. (That set may overlap with the first set of course.) She puts these 9 in the jar, along with the marked marble, and shows the jar to Ben. He examines the jar, and reports to Quinn his credence in p.

Now in this case we can work out precisely how Quinn should update on these two pieces of information. When one expert reports a credence of x in p, Quinn can infer that they saw 10x white marbles. After all, what the expert knows is just that the marked marble is equally likely to be any of the marbles in the jar they see. So given Cr_A(p) = y and Cr_B(p) = z, Quinn can infer how many white marbles were in each jar. And they can work out the probability of each of those jars turning up given p and given ¬p. And that’s enough to plug into Bayes’s Theorem to work out a posterior probability for p. When you do that, you get the following result.

7. Cr_Q(p | Cr_A(p) = y ∧ Cr_B(p) = z) = yz/(yz + (1-y)(1-z))

I’m not going to work through the derivation of this, because it’s a straightforward consequence of something I will derive below. If you do want to check it for yourself, the key input is that the probability of drawing x white balls in t draws without replacement from an urn with w white balls and b black balls is

\[ \frac{\binom{w}{x} \binom{b}{t-x}}{\binom{w+b}{t}} \]

More importantly, (7) looks just like a special case of the central formula (Upco) that Easwaran et al. (2016) use. And that’s not surprising, since this case uses the same conditional independence assumption that they make through much of their paper. To say that A and B are conditionally independent given C is just to say that Pr(A ∧ B | C) = Pr(A | C)Pr(B | C). In this case, any pair of claims about how many white balls are in the jars shown to Ava and to Ben are conditionally independent, both conditional on p and on ¬p.

Equation (7) violates what Baccelli and Stewart (2021) call Unanimity. This principle requires that Cr_Q(p | Cr_A(p) = y ∧ Cr_A(p) = y) = y. If (7) is true then Unanimity is violated in every case except where y equals 0, 0.5 or 1. But this is bad news for Unanimity, because the case for (7) in this case seems very strong. Quinn really knows how many white marbles were in each jar, and it’s just a bit of algebra to get from there to (7) via conditionalisation. And it’s very plausible that conditionalisation is the right way to update in this case, when Quinn has so much background knowledge about how Ava and Ben got their credences. So any principle incompatible with (7) is false.

It turns out that varying how many marbles are in the urn Carmen starts with does not change (7). But changing the ratio of white marbles to black marbles in the urn does change the formula. If the proportion of the initial urn that is white is r, then the general result is:

8. Cr_Q(p | Cr_A(p) = y ∧ Cr_A(p) = z) = (yz(1-r))/(yz(1-r) + (1-y)(1-z)r)

Again, this isn’t a new result; Easwaran et al. (2016, 27) derive an even more general formula from which this falls out as a special case. But my way of deriving it is just different enough to be worth including.

Let A_x be the disjunction of all possible evidence propositions that would lead Ava to have credence x in p. In this case A_x is a simple proposition that there are 10x white marbles in the jar, but we don’t need to assume that A_x will be anything like that simple. Everything that follows about A_x also holds for B_x, the disjunction of all possible evidence propositions that would lead Ava to have credence x in p, but I won’t repeat the derivations. Since Quinn defers to Ava, i.e., (4) is true, we have the following proof. (All credences are Quinn’s, so I’ll drop the subscripts.)

\[ \begin{aligned} Cr(p | A_x) &= x \\ Cr(p \wedge A_x) &= x \cdot Cr(A_x) \\ &= x (Cr(p \wedge A_x) + Cr(\neg p \wedge A_x)) \\ (1-x)Cr(p \wedge A_x) &= x \cdot Cr(\neg p \wedge A_x) \\ Cr(\neg p \wedge A_x) &= \frac{1-x}{x} Cr(p \wedge A_x) \\ Cr(A_x | \neg p) &= \frac{(1-x)Cr(p)}{x\cdot Cr(\neg p)}Cr(A_x | p) \end{aligned} \]

So we know the ratio of Cr(A_x | p) to Cr(A_x | ¬p). That will become useful in what follows. Assuming evidentialism, what matters for (6) is working out the value of Cr(p | A_y ∧ B_z). But we now know enough to do that.

\[ Cr(p | A_y ∧ B_z) = \frac{Cr(p ∧ A_y ∧ B_z)}{Cr(A_y ∧ B_z)} \]

Using the general fact that X is equivalent to (p ∧ X) ∨ (¬p ∧ X), and that Quinn’s credences are probabilistic, so their credence in an exclusive disjunction equals the sum of the credence in the disjuncts, we know this equals.

\[ \frac{Cr(p ∧ A_y ∧ B_z)}{Cr(p ∧ A_y ∧ B_z) + Cr(\neg p ∧ A_y ∧ B_z)} \]

Since Cr(p ∧ X) = Cr(X | p)Cr(p), we can rewrite this as:

\[ \frac{Cr(A_y ∧ B_z | p) Cr(p)}{Cr(A_y ∧ B_z | p)Cr(p) + Cr(A_y ∧ B_z | \neg p)Cr(\neg p)} \]

And since A_y and B_z are independent given both p and ¬p, this becomes:

\[ \frac{Cr(A_y| p) Cr(B_z | p) Cr(p)}{Cr(A_y| p) Cr(B_z | p) Cr(p) + Cr(A_y| \neg p) Cr(B_z | \neg p) Cr(\neg p)} \]

If we assume the initial value of Cr(p) = r, and use the earlier derived fact that Cr(A_x | ¬p) = ((1-x)r)/(x(1-r)Cr(p)) this becomes:

\[ \frac{Cr(A_y| p) Cr(B_z | p)r}{Cr(A_y| p) Cr(B_z | p)r + \frac{(1-y)r}{y(1-r)} Cr(A_y| p) \frac{(1-z)r}{z(1-r)} Cr(B_z | p) (1-r)} \]

Now we can finally eliminate Cr(A_y | p)Cr(B_z | p)r from the top and bottom, so this becomes:

\[ \frac{1}{1 + \frac{(1-y)(1-z)r}{yz(1-r)}} \]

Or in other words:

\[ \frac{yz(1-r)}{yz(1-r) + (1-y)(1-z)r} \]

And that’s the completely general result when the evidence the experts have is conditionally independent of both p and ¬p, and Quinn starts with credence r in p. It turns out we don’t need the extra assumption that each expert has the same starting credence, because the way the case is set up, the expert’s prior credences (and hence prior evidence) is screened off by the new evidence. In practice, that kind of complete screening is rare, so it’s safest to use this formula when all three of them start with credence r.

For instance, imagine that Quinn, Ava, and Ben are detectives investigating a murder, and they all think Dallas is the most likely suspect. They are, however, far from sure. Each of them has 0.6 credence that Dallas is guilty. Ava and Ben go back to scour different parts of the evidence to see what they may have missed. Quinn trusts each of them to extract correct information and update on it appropriately, and so satisfies both (4) and (5). Ava goes over the forensic reports more carefully, and comes away with credence 0.8 that Dallas is guilty. Ben goes over the witness statements more carefully, and also comes away with credence 0.8 that Dallas is guilty. It’s plausible that Ava and Ben’s credences, while sensitive to Dallas’s guilt and hence not independent, are conditionally independent, conditional on whether Dallas is guilty. If so, Quinn’s credence on learning both Ava and Ben’s new credences should rise, according to this formula, to 32/35, i.e., a little over 0.9. That violates Unanimity, but sounds fairly plausible. The fact that Ava raised her credence is evidence that Dallas is guilty. The fact that Ben raised his credence, after looking at different evidence, is further evidence that Dallas is guilty. So it’s plausible that Quinn should raise their credence more when learning both these things than they would when learning just one of them. Since just learning one would raise Quinn’s credence to 0.8, learning both should raise it further still.

Still, cases like this are fairly special. It’s rare for a non-expert to think that two different experts have no common cause for their credences other than the truth or falsity of the target claim. The next model will help with more realistic case.

4 Case Two: Common Marbles

In our second case, Carmen once again has an urn with 50 marbles, 25 black and 25 white. She draws one at random and marks it with invisible ink. She can tell which one is marked, but no one else can. And p is still the proposition that the marked marble is white - that’s what we’ll focus on from now on. After selecting the marble to be marked, she puts together a jar containing the marked marble and 9 other marbles drawn at random from the urn. She shows that to one of the experts, let’s say Ava. She gets to inspect the jar, i.e., count how many marbles in it are white and black. She then reports to Quinn, but crucially not to Ben, her credence in p.

So far, it’s just like the last case. But what happens next is (possibly) different. In this case, Carmen removes m unmarked marbles from the jar, puts them back in the urn, and draws a new set of m marbles to put in the jar. It’s all random, so this could include some of the marbles she just removed. She shows the jar to Ben, he inspects it, and reports his credence in p to Quinn. And, crucially, Quinn knows m, the number of marbles that are in common between the jars. So m is a measure of the independence of the experts’ opinions.

Once again, we can work out precisely what Quinn’s credence should be given m, and the two credences. Unfortunately, it’s a long formula that doesn’t seem to reduce nicely. But if you’ve got a machine that’s good at calculating hypergeometric distributions, and you dear reader are probably reading this paper on a machine that’s good at calculating hypergeometric distributions, it’s not that hard to calculate the values by brute force. I won’t list all the values, there are several hundred of them, but I’ll present them graphically in Figure 1. (Note I’ll leave off the case where one or other expert announces a credence of 0 or 1; in that case Quinn knows whether p is true, so the question of how to merge the credences is easy.)

Figure 1: The result of merging two somewhat connected opinions.

Here is how to read the graph in Figure 1. Each row corresponds to a particular credence announced by Ava; the credence is shown on the right. Each column corresponds to a particular credence announced by Ben; that credence is shown on the top. The x-axis of the individual graphs shows the value for m, the number of marbles removed. And the y-axis shows Quinn’s final credence in p. There are more dots on some graphs than others because some combinations of Ava credence, Ben credence and m are impossible. The announced credences can’t, by the rules of the game, differ by more than 0.1m.

One notable feature of that graph is that as m gets larger, the final credence tends to move away from 0.5; it tends to get more opinionated. Another notable feature, though probably not one you can see in this resolution, is that this move towards greater opinionation happens in a surprisingly linear fashion. To a first approximation, Quinn’s credence moves away from 0.5 roughly the same amount for each addition to m, at least holding y and z fixed.

It’s not perfectly linear, but it’s surprisingly close to being linear. Figure 2 shows the result of zooming in on a part of the graph to see this more vividly.

Figure 2: The bottom right corner of Figure 1

The curve in the bottom right panel of Figure 2 is not really linear; it definitely curves downwards. But as you move your eye upwards and leftwards in the table, the curves look much much straighter. The panel where they both announce 0.7 is really remarkably straight. If we focus on the middle of Figure 1, this is even more striking. (I’ve left off the cases where Ben announces a credence under 0.5 because those graphs are just mirror images of graphs already shown.)

Figure 3: A detail of the middle of Figure 1

This matters because the big problem in practice with the supra-Bayesian approach is that it is impractical, especially in contrast with the formulaic approaches. If we’re prepared to tolerate a little imprecision, it turns out that in cases like this one, there is a simple and close approximation to the supra-Bayesian answer.

Intuitively, the approximation works like this. Let g be the geometric average of Ava and Ben’s opinions, as discussed in Section 1. That is, if Ava’s credence in p is y and Ben’s is z, the geometric average of their opinions assigns credence \(\frac{\sqrt{yz}}{\sqrt{yz}\sqrt{(1-y)(1-z)}}\) to p. Let h be the credence we worked out in Section 3 Quinn would have if they were certain that Ava and Ben’s credences were conditionally independent. Let k be the proportion of evidence that they share; in this case the proportion of non-marked marbles that were by design put into both of their jars. Then roughly, Quinn’s credence in p should be kg + (1—k)h. That is, it’s a weighted average of the geometric mean and the value that Easwaran et al. (2016) showed was correct in the conditional independence case, with the weights given by how much evidence Ava and Ben share. Spelling all that out, the approximate credence Quinn should have is this:

\[ k\frac{\sqrt{yz}}{\sqrt{yz} + \sqrt{(1-y)(1-z)}} + (1-k)\frac{yz}{yz + (1-y)(1-z)} \]

Let’s check visually how this does against the exact calculations. In the graphs that follow, starting with Figure 4, I’ll use circles for the ideally calculated posterior credences, and triangles for the estimates made using this formula.

Figure 4: A detail from Figure 1 with estimated probabilities shown.

That looks pretty good. There is a tiny bit of separation in the bottom right panel of Figure 4, but otherwise the estimate tracks the calculated credences pretty closely. Figure 5 shows the middle of the graph.

Figure 5: The middle of Figure 1 with estimated probabilities shown.

And all through Figure 5 the dots are overlapping. That’s close enough. So at least in this special case, the supra-Bayesian can produce an estimate that is very close to the ideally calculated credence. The formula for it is not as trivial as adding up and dividing by two, but it’s not that complicated as these things go. So if Quinn’s situation resembles this common marbles case, we don’t need to use an all-purpose formula as an approximation device.

To be sure, the simplifications here are dire. Here are six ways we might want to generalise the model.

1. Have the prior probabilities of p and ¬p vary.
2. Have more colors for the marbles, and have each expert announce credences over all the colors.
3. Have the person doing the merger be uncertain about k.
4. Have the experts sample the jars they are given, not inspect them fully.
5. Have more than two experts.
6. Allow that some experts are more informed than others.

The first two points are not that hard. I could produce a string of graphs for different priors over the colors, or for more colors, and the typical story is not that different to what we’ve seen so far. It just gets messy because we have more degrees of freedom than is consistent with a concise graphical display.

The next two points are harder. It’s not that they are harder to come up with the ideal value. For any prior over k, or sampling technique that’s available to the expert, it’s pretty easy to write code to come up with the optimal calculated credences. It’s rather that the number of degrees of freedom are so great that it gets a little harder to eyeball how good any given approximation is. The big point is that the posterior distribution of k will usually be different to the prior. In extreme cases, the announced expert credences might rule out some hypotheses about k. So it won’t just be a matter of calculating the values of the above formula for each value of k, and averaging them out using the prior probabilities of k. There is a lot of possible future research here.

Having more than two experts raises both computational questions, like what we’ve just discussed, and conceptual questions. The number of variables we need to specify to say how connected the experts are roughly doubles every time one adds an expert. But the point is not just that the computations of the ideal supra-Bayesian credence require an exponentially increasing number of inputs as the number of experts rises. It’s that even thinking about how to approximate this ideal calculation, we need a good way to conceptualise this space whose dimensionality rises exponentially with the number of experts in a way that lets us even think about what a good approximation would look like. I don’t have an answer to this; it too feels like a question for future research.

That said, there are real-life cases where even this simplified model looks like it is useful. Imagine that Quinn, Ava and Ben are part of an American academic department trying to make a new hire.⁶ They each have credence 0.6 that Elsa is the best person for the job. Quinn learns that each of Ava and Ben have talked more with Elsa and come away with credence 0.8 that she’s the best person for the job, and Quinn is sure that they would have the same credence had they sat in on either interview. What should they do with this pair of facts?

⁶ I’m specifying it’s American to allow for the possibility of informal interviews like the ones I’ll describe to matter to the verdict.

The model in this section suggests the following procedure.

1. If Quinn thinks Ava and Ben asked the exact same questions and got the exact same answers, Quinn’s credence should be 0.8.
2. If Quinn thinks they asked completely separate questions, and whether Elsa did well on one interview is independent of how well she did on the other, conditional on her suitability for the job, Quinn’s credence should be 32/35, as in the previous section.
3. If Quinn thinks the proportion of the two interviews that overlapped, i.e., Ava and Ben asked questions that tested the same skills, is k, Quinn’s credence should be 0.8k + 32k/35. That is, it should be a weighted average of the first two, with the weight being the overlap in the evidence.

Point 3 is not mathematically precise. But it’s really close; close enough for practical purposes. In any realistic case, the error bars in the estimate of k will be many times larger than the difference between this linear estimate and the mathematically precise answer. So this is a useful estimate in realistic cases.

The last example I’ll go on to discuss considers what happens if we do not assume the experts are roughly as well informed as each other.

5 Case Three: Differentially Informed Experts

In our last case, one expert is better informed than the other. Carmen first fills the jar with the marked marble and 19 randomly chosen unmarked marbles. She flips a coin to decide which expert to show this jar to. They inspect the jar, and record their credence in p to the nearest 0.1. (We’ll come back very soon to why this is rounded. If their credence is half-way between two multiples of 0.1, as it often will be, they flip a fair coin to decide whether to round up or down.) Carmen then removes 10 unmarked marbles from the jar, chosen at random, and then shows it to the other expert. They inspect it, and come up with a new credence in p. Then both these recorded numbers are reported to Quinn, without any indication about who saw the larger jar and who saw the smaller one.

There is a weird thing in this setup in that one of the experts reports something other than their precise credence. The reason I set up the example this way is to make it impossible for the recipient of the expert opinion to infer who saw the smaller jar. If they both reported their actual credence, it would be possible for the recipient to be told one of them has credence 0.75 in p and the other has credence 0.6. And then it would be obvious that the hearer should have credence 0.75 in p, since that’s the credence of the more informed person. So I made the first person round to the nearest 0.1 to make it harder to make such inferences.

Given all that setup we can work out what Quinn’s credence in p should be given the two announcements, and I’ve shown the values in Table 1. (I’m rounding to three decimal places to save space. I’m leaving off the cases where one or other party announces an extremal credence - the hearer agrees with those credences, at least to three decimal places. And the ‘NA’ values are where it is impossible given the setup for those to be the announced values.)

Table 1: The posterior probability after hearing two differentially informed experts.

Ava/Ben	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
0.1	0.100	0.103	0.100	0.100	0.100	0.100	NA	NA	NA
0.2	0.103	0.200	0.208	0.203	0.202	0.200	NA	NA	NA
0.3	0.100	0.208	0.300	0.320	0.325	0.348	0.500	NA	NA
0.4	0.100	0.203	0.320	0.400	0.439	0.500	0.652	0.800	0.900
0.5	0.100	0.202	0.325	0.439	0.500	0.561	0.675	0.798	0.900
0.6	0.100	0.200	0.348	0.500	0.561	0.600	0.680	0.797	0.900
0.7	NA	NA	0.500	0.652	0.675	0.680	0.700	0.792	0.900
0.8	NA	NA	NA	0.800	0.798	0.797	0.792	0.800	0.897
0.9	NA	NA	NA	0.900	0.900	0.900	0.900	0.897	0.900

A striking thing about Table 1 is how close it comes to verifying a strong form of what Levinstein (2015) calls Thrasymachus’ Principle. The hearer defers to the expert with the strongest view, i.e., the view that’s furthest from the prior. In contemporary terms, the hearer listens to the expert with the hottest take. It isn’t an unvarnished form of that. When one says 0.5 and the other says 0.6 you end up with 0.561, not 0.6. But that’s in large part because there’s a good chance that the person who said 0.6 was merely rounding up as the result of a coin flip. In general, the rule in this case is find the expert credence that is furthest from the prior, and adopt it.

There is a reason that a case like this should follow Thrasymachus’ Principle. If the experts are rational, hotter takes should correspond to stronger evidence. While it isn’t impossible for the person with more evidence to have in a sense weaker evidence, the extra evidence may be full of defeaters for the first obtained evidence, it is pretty unlikely. In general, if someone is worthy of deference, and they have a strong view, they have strong evidence. If someone else has a weaker view, i.e., a view closer to the prior, the best explanation is that they simply don’t have the evidence that the person with stronger view does.

As with the model in Section 3, this seems like it could reasonably model several cases involving detection. Go back to Ava and Ben doing another run through the evidence while Quinn is busy. In this case, they look at the same documents, but it’s possible that one of them will notice something in the documents the other will not. In these cases we often call things like forensic reports and witness statements the ‘evidence’, but here it’s important if we are sympathetic to something like Williamson’s E=K picture to distinguish the documents themselves from what the detectives take from the documents (Williamson 2000). In the latter sense, one of Ava and Ben might get more evidence from the same document because they observed something in them the other did not.⁷ That’s a fairly plausible explanation of why they might come away from the same documents with different credences, because one of them acquired evidence the other did not. And in those cases, Quinn should defer to the one with more surprising credences.

⁷ Compare the discussion in “A Scandal in Bohemia” (Conan Doyle [1891] 2015) about how many steps are in their lodgings. Holmes and Watson had access to the same staircase, but Holmes took in more evidence than Watson did. In that case it’s not surprising which one got more evidence, but the same thing can happen in more symmetric relationships.

In general, any kind of averaging in cases like these will not be respecting the evidence Quinn has. If one of Ava and Ben has a very surprising credence, that’s evidence that they have more evidence, and hence are more worthy of deference. Quinn should use that higher-order evidence in coming to a composite view. They shouldn’t average the opinions in any interesting sense. Table 1 shows the optimal response by supra-Bayesian lights. And the simple approximation is, “When one expert has clearly stronger views, listen to them. Otherwise take the geometric mean.”

6 Conclusions

There are many arguments that, ideally, we should be Bayesians. But when it comes to merging expert opinions, the thoroughly Bayesian approach is practically infeasible. One response to this is to move to some completely different approach, such as using linear or geometric means. This paper is a start at showing a different response to this problem, one that develops relatively simple approximations to the complex supra-Bayesian answers. The thought is that by having a family of approximations to hand, and having some of those approximations include variables like the amount of overlap between the experts’ evidence in Section 4, we can produce practical results that are both more plausible, and more theoretically sound, than any one formula.

There are three attitudes one could take towards the urns and jars and marbles examples I’ve used in this paper. I’ll describe them in decreasing order of philosophical commitment.

One is that these kinds of examples are basically the right way to think about basically all problems of practical uncertainty. This is, I think, a plausible view, but I haven’t come close to defending it here.

A second is that there are many real life cases that are usefully modelled by these kinds of examples. That’s what I have argued for here. The point of my cases involving detectives and search committees was to argue that sometimes, realistic cases are similar enough in structure to urns and marbles that we can model them that way. When we do that modelling we get mathematical results which, despite conflicting with principles like Unanimity, seem fairly intuitive in the particular cases.

A third, and much weaker, position is that the kind of probabilistic reasoning involving urns and marbles is a good way to think about urns and marbles, but possibly not much else. If that’s right, then the examples here would be less interesting than I’ve suggested. But note that even on that weak view, we still have reason to reject Unanimity, and we still have reason to reject the global application of any kind of averaging. Even if I’m wrong about the positive application of these models to real-life cases, the argument against unrestricted applications of averaging rules still goes through.

The models here all do have one important idealisation in common. I’ve assumed that the experts do not know each others’ credences. What should we say about cases where they do? This is an old and, to my mind, open question. For reasons that trace back to Aumann (1976), in anything like the kind of model I’ve used here, if the experts know each other’s credences, they have to agree. And someone who knows both credences should agree with them. But the real world obviously contains experts who do agree to disagree. What to say about those cases is the biggest open questions around here, and I’m not sure whether this approach can help. Gallow (2018) ends his paper by raising doubts about whether it is rational to be disposed to defer to two different experts. I’m not worried about that in general; I’ve described three very different kinds of cases where it is rational. But I suspect one could not be rationally disposed to defer to two experts who one knows are themselves disposed to agree to disagree. That, however, is a story for another paper. This paper has described a number of cases where the hearer knows something the experts doesn’t know: namely what other experts think. And it has described both precise and approximate answers for what to do in those interesting cases.

References

Aumann, Robert J. 1976. “Agreeing to Disagree.” The Annals of Statistics 4 (6): 1236–39. https://doi.org/10.1214/aos/1176343654.

Baccelli, Jean, and Rush T. Stewart. 2021. “Support for Geometric Pooling.” Review of Symbolic Logic 16 (1): 298–337. https://doi.org/doi:10.1017/S1755020320000416.

Bokros, Sofia Ellinor. 2020. “A Deference Model of Epistemic Authority.” Synthese 198 (12): 12041–69. https://doi.org/10.1007/s11229-020-02849-z.

Conan Doyle, Arthur. (1891) 2015. The Adventures of Sherlock Holmes. Project Gutenberg.

Dellsén, Finnur. 2020. “The Epistemic Value of Expert Autonomy.” Philosophy and Phenomenological Research 100 (2): 344–61. https://doi.org/10.1111/phpr.12550.

Easwaran, Kenny, Luke Fenton-Glynn, Christopher Hitchcock, and Joel D. Velasco. 2016. “Updating on the Credences of Others: Disagreement, Agreement, and Synergy.” Philosophers’ Imprint 16 (11): 1–39.

Elkin, Lee, and Richard Pettigrew. 2025. Opinion Pooling. Cambridge University Press. https://doi.org/10.1017/9781009315203.

Gallow, J. Dmitri. 2018. “No One Can Serve Two Epistemic Masters.” Philosophical Studies 175 (10): 2389–98. https://doi.org/10.1007/s11098-017-0964-8.

Givens, Geof H., and Paul J. Roback. 1999. “Logarithmic Pooling of Priors Linked by a Deterministic Simulation Model.” Journal of Computational and Graphical Statistics 8 (3): 452–78. https://doi.org/10.2307/1390869.

Hall, Ned. 2004. “Two Mistakes about Credence and Chance.” Australasian Journal of Philosophy 82 (1): 93–111. https://doi.org/10.1080/713659806.

Jackson, Frank. 1987. Conditionals. Blackwell.

Lane, Devin. 2025. “Should You Defer to Individual Experts?” Philosophy and Phenomenological Research 111 (1): 216–34. https://doi.org/10.1111/phpr.70025.

Levinstein, Benjamin Anders. 2015. “With All Due Respect: The Macro-Epistemology of Disagreement.” Philosophers’ Imprint 15 (13): 1–20.

Pettigrew, Richard, and Jonathan Weisberg. Forthcoming. “Geometric Pooling: A User’s Guide.” British Journal for the Philosophy of Science. https://doi.org/10.1086/727000.

Williamson, Timothy. 2000. Knowledge and its Limits. Oxford University Press.

Wright, Jack. 2025. “The Value of Independence Between Experts: Epistemic Autonomy and Different Perspectives.” Episteme 22 (4): 818–34. https://doi.org/10.1017/epi.2024.20.

Zhang, Snow. 2026. “Coherent Combination of Multiple Experts’ Opinions: Another Impossibility Result.” Theory and Decision 100 (2): 451–78. https://doi.org/10.1007/s11238-025-10092-6.