Inference runs every rule backwards. Probability is what makes that legitimate.

Sherlock Holmes called his method deduction. It was nothing of the kind.

Deduction runs from a rule and a premise to a conclusion that cannot fail. Soft ground takes an impression; the ground was soft; there is an impression. Holmes works the other direction entirely. He finds the impression and names the man who made it. That move carries no guarantee whatsoever, and in a first-year logic course it has a name: the fallacy of affirming the consequent.

Which would be a pedantic complaint, except that the fallacy is now industrialised. A spam filter sees the word "invoice" and infers intent. A fraud model sees a card-not-present transaction at 3 a.m. and infers theft. A retrieval system sees vector proximity and infers relevance. Each one observes an effect and reaches for the cause, which is the invalid direction, running some billions of times a second across every production system in the field.

So either the entire discipline rests on a logical error, or something in the reversal survives that logic has no vocabulary for. The second is closer to the truth. What survives is not a truth value but a weight, and everything from precision-recall tradeoffs to Bayes' theorem is machinery for computing it.

The argument runs in five moves. Propositions and connectives give us a language for reasoning in absolutes. Implication shows us where that language leaks. Translating implication into set theory shows us the shape of the leak. Counting the leak gives us two distinct kinds of being wrong. And attaching a number to those sets, on two axioms and one ratio, rebuilds the inference that logic had to abandon.

Thinking in absolutes

Start with how knowledge gets stored. At the base level it sits in propositions: declarations that are either true or false, with no third option and no partial credit. Write true as 1 and false as 0 and the whole apparatus becomes arithmetic.

"There are infinitely many prime numbers" is a proposition, and it is true. "1 + 1 = 3" is a proposition, and it is false. "I have the flu" is a proposition whose value you may not know, which is a different situation from the value not existing. Logic assumes the value is there, fixed, waiting to be discovered.

We abbreviate them as variables. Let A = "it is raining outside" and B = "the sidewalk is wet". Working out the truth value of a proposition from evidence and reasoning is called inference, and stated that way it sounds like bookkeeping.

Logic assumes the truth value exists and is merely unknown to you. Everything difficult about inference follows from that assumption being an idealisation.

Four connectives, and one that carries the weight

Single propositions run out of expressive power fast. Real reasoning chains them: if it is raining outside, then the sidewalk is wet. That sentence is not a new atomic fact but a compound built from A and B with a connective.

Four connectives do most of the work. Negation, ¬, flips a value. Conjunction, ∧, is true when both parts are true. Disjunction, ∨, is true when at least one part is. Implication, →, formalises deducing a conclusion from a premise, and its definition is worth stating precisely: A → B is true when B is true, and also when A and B are both false.

Read that definition twice. Three of the four rows come out true, and the implication fails only when a true premise produces a false conclusion. Rain with a dry sidewalk breaks the rule. Nothing else does.

Connectives are defined by nothing more than the truth values they produce, which is why a truth table is a complete definition rather than an illustration of one. Set the inputs, read the output.

Truth tables: the complete definition of each connective

A	¬A
1	0
0	1

A	B	A ∧ B	A ∨ B	A → B
1	1	1	1	1
1	0	0	1	0
0	1	0	1	1
0	0	0	0	1

Set the two statements yourself and watch the four compounds respond.

Connective explorer

A it is raining outside

true false

B the sidewalk is wet

true false

¬A

–

negation

A ∧ B

–

conjunction

A ∨ B

–

disjunction

A → B

–

implication

Where implication leaks

That last observation deserves a section of its own, because it is the hinge of the whole argument. When A is false, A → B is true regardless of B. Logicians call this vacuous truth. "If I am the King of France, the sidewalk is wet" is a true statement, and it is true because there is no King of France.

The consequence is severe. A true implication constrains only the rows where its premise holds. It tells you what follows from rain and it tells you nothing whatsoever about what a wet sidewalk implies. Reading it in reverse, from evidence to cause, is the fallacy of affirming the consequent, and it is invalid no matter how strong the original rule.

This matters because true implications are where knowledge actually lives. "If a system is closed, its entropy cannot decrease." "If ABC is a right triangle, then A² + B² = C²." Reasoning like Sherlock Holmes means chaining statements of that form until the conclusion falls out. The catch is that each link runs one way only. A wet sidewalk might mean rain, or it might mean somebody upended a barrel of water, and the implication has no opinion on which.

Implication is containment

Every connective has a twin in set theory, and the translation is where the picture starts to do work. Take the set of all mornings and, inside it, the mornings that satisfy each proposition. Conjunction becomes intersection: the mornings in both sets. Disjunction becomes union: the mornings in either. Implication becomes the subset relation.

That last one deserves slow reading. Saying "if it rains, the sidewalk is wet" is saying that every rainy morning is also a wet-sidewalk morning, which is to say the rain set sits entirely inside the wet set. Not overlapping it. Contained by it.

Three geometries, three logical situations

The geometry now explains the fallacy without any appeal to truth tables. Reading the implication backwards means claiming the wet set sits inside the rain set. Nothing about the original containment grants that, and the two claims hold together only when the sets coincide.

Invalid is not the same as useless, though, and this is where the argument turns quantitative. Suppose you track 1,000 mornings. It rains on 100 of them and rain always wets the sidewalk, so the containment is exact. On the 900 dry mornings, sprinklers run 90 times and the street cleaner passes 45 times, each wetting the sidewalk with no rain involved.

So 235 mornings show a wet sidewalk and only 100 involved rain. Your evidence, resting on a rule that is exceptionless, supports "it rained" about 42.6% of the time. Below a coin flip. The rule never failed once. The reverse reading was simply never entitled to the rule's certainty.

Certainty about a rule does not transfer to certainty about a cause. What survives the reversal is a weight, not a verdict.

Two circles, four regions

The nested picture was generous in one respect: it granted the rule perfect reach. Every rainy morning produced observable wetness, so the hypothesis stayed inside the evidence and one error type was impossible by construction.

Relax that. Say five of the hundred rainy mornings were brief pre-dawn showers that dried before you looked, or fell while you were under an awning. The rain happened. The evidence did not survive to be observed. Those five mornings now sit outside the evidence set, and the containment breaks.

The moment it breaks, the geometry moves from the left panel above to the middle one, and four regions appear where the binary frame offered two.

The geometry of inference

False positive

The sidewalk is wet and it did not rain. Evidence present, hypothesis absent. The sprinklers and the street cleaner live here, and so does every spam filter that eats an invoice.

135 of 1,000 mornings

Select a region to read what it costs.

The overlap is the true positive: 95 mornings where it rained and the sidewalk was wet, evidence and hypothesis agreeing. The blue crescent is the false positive, 135 mornings of evidence with no rain behind it. The yellow crescent is the false negative, the 5 rainy mornings your evidence never reached.

Those two crescents are not variations of a single mistake. They sit on opposite sides of the picture and they come from opposite causes. The blue one grows when the world offers alternative explanations for your evidence. The yellow one grows when your observation fails to catch the cause. Nothing you do to one automatically improves the other.

Why is the sidewalk wet? Weights over 230 wet mornings

Explanation	Mechanism	Weight
Rain	Falls on 100 mornings, observable on 95 of them	41.3%
Sprinklers	Timed, run on 90 dry mornings, independent of weather	39.1%
Street cleaning	Scheduled route wets 45 dry mornings	19.6%

A fraud model that flags every transaction has no false negatives and an unusable pile of false positives. A model that flags nothing inverts the problem exactly. Between those two poles sits a decision, and the decision is about which of the two crescents you can afford to live with.

Two axioms, and logic becomes probability

Here is the useful accident of the set translation. We do not need a new language to handle weights, because the sets are already drawn. All that is missing is a number attached to each one.

Fix the sample space first. Call it Ω: every morning in the record, the universe of things that could happen. An event is any subset of Ω, which means rain is an event, a wet sidewalk is an event, and so is their intersection. A probability is a function P that hands every event a non-negative number. That requirement is usually listed as the first axiom and it is the least interesting of them. Two conditions do the real work.

The axioms of probability

1) P(Ω) = 1

2) P(∪_k=1^∞ A_k) = ∞∑k=1 P(A_k) whenever the events are pairwise disjoint: A_i ∩ A_j = ∅

The first axiom says something happens. The whole space gets weight 1, which makes every event's weight a share of a fixed budget and makes two weights comparable. It is also why the explanations in the table above summed to exactly 100%. That was not tidiness; it was the axiom.

The second axiom is where the care lives. Weights add across events, but only when those events cannot occur together. Rain, sprinklers, and street cleaning added cleanly in that table because the model was built so that sprinklers run on dry mornings only. Nothing in the real world enforces that. Sprinklers on timers run in the rain, and the moment two explanations can co-occur, adding their weights counts the overlap twice.

Which is exactly the intersection from the set translation, and it is what the correction subtracts:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Disjointness is therefore a modelling assumption, not a mathematical convenience, and it is one of the quieter ways a well-specified model goes wrong. Double-counted evidence looks like confidence.

What happened to logic

Nothing was thrown away. Restrict P so that it only ever returns 0 or 1 and propositional logic comes straight back: every proposition gets a truth value, conjunction is intersection, disjunction is union. Truth is the special case where the weight has collapsed to an endpoint.

Implication survives too, in a weaker but more useful form. Containment plus additivity forces monotonicity: if A sits inside B, then P(A) is never greater than P(B). A valid rule no longer certifies its conclusion, but it does bound it. Rain on 10% of mornings and wet sidewalks on 23.5% of them is consistent with the rule; rain at 30% of mornings would not be.

A true implication used to say the conclusion must hold. Now it says the premise is never more probable than the conclusion. Weaker, and still enough to reason with.

What implication becomes

Two of the three connectives came across the bridge intact. Intersection is the joint occurrence of two events, both happening together, which is conjunction. Union is either one occurring, which is disjunction. The axioms then tell us how to weigh both.

Conjunction and disjunction, weighed

P(A ∩ B)  both happen

P(A ∪ B)  at least one happens

Implication is the one left stranded. In classical logic, events matter in the context of other events, and implication supplied that context: given the premise, take the conclusion. Probability needs its own version of the same move, and the question it has to answer is narrower than the one logic asked. Not "does B follow from A" but "how likely is B, given that A has been observed".

The construction is almost crude. If A has occurred, then everything outside A is off the table. Discard it. What remains is a smaller universe, A itself, and the only part of B that still exists inside it is the intersection. The sample space has shrunk, so the weights need renormalising — divide by the weight of the new universe.

Conditioning: shrink the universe, then renormalise

P(B | A) = P(A ∩ B) P(A)

Run the sidewalk numbers through it. Rain happens on 10% of mornings, wet sidewalks on 23%, and both together on 9.5%. Condition on the evidence and you get P(rain | wet) = 0.095 / 0.230, which is 41.3%. That is the number we reached earlier by counting mornings, arrived at now by formula.

Condition the other way and the same intersection gives P(wet | rain) = 0.095 / 0.100, or 95%.

Those two numbers have names. The first is precision: of everything flagged, how much was real. The second is recall: of everything real, how much got flagged. They differ by a factor of 2.3 while sharing a numerator, and the entire difference is the denominator — what you chose to treat as the given.

Cause and effect share a numerator. The only thing that distinguishes them is what you divide by.

This is where implication reappears, no longer as a guarantee but as one end of a range. Two extreme geometries make the point faster than any algebra.

The two endpoints of conditioning

Take the right panel first. A sits entirely inside B, so the intersection is all of A, the ratio is A over A, and P(B | A) = 1. That is modus ponens, restated as arithmetic. The exceptionless rule from earlier said exactly this: rain nested inside wet gave P(wet | rain) = 0.100 / 0.100.

The left panel is the mirror image. A and B share nothing, the intersection is empty, and the ratio is 0. Observing A does not leave you uninformed about B; it rules B out, which is itself an implication — A implies not-B. In the sidewalk model, street cleaning was defined to happen only on dry mornings, so P(cleaning | rain) = 0 by construction.

So the two logical verdicts sit at the two endpoints of a single scale. Certainty is 1, refutation is 0, and neither is a special kind of claim. They are the values the ratio takes when the geometry is extreme enough to permit them.

Logic gave us the endpoints and nothing in between. Conditional probability fills the interval, and every model worth deploying lives in the interval.

Go back to the three geometries from earlier and the picture is complete. Nesting gives 1. Disjointness gives 0. Partial overlap, which is what real evidence almost always looks like, gives 0.413 — a number logic has no way to express and no vocabulary to reason about.

Everything in between

The interesting relations are the ones near an endpoint without reaching it. Slide B until it barely clips A and the intersection thins to a sliver: P(B | A) ≈ 0. Low plausibility. Observing A makes B unlikely without making it impossible.

The approximation sign carries more weight than it looks. Zero says the geometry forbids it, and no volume of observation will change that. Three percent says the geometry permits it and it happens rarely, which at scale means it happens constantly. Card fraud sits at well under one percent of transactions. Every case a fraud model exists to catch lives in a sliver, and a model that rounds slivers down to zero achieves excellent accuracy by never being useful.

One degree of freedom, the whole range of plausibility

Separation between the two events

P(B | A)

0.02

P(A | B)

0.01

Low plausibility. A barely touches B. Observing A leaves B unlikely and not impossible.

Push the slider to full containment and the asymmetry becomes impossible to miss. P(B | A) reads 1.00, exact certainty, while P(A | B) reads 0.62. One rule, one geometry, and the confidence you are entitled to depends entirely on which direction you read it. Holmes at the footprint is standing on the 0.62 side and reporting the 1.00.

Probabilistic modus ponens

The far end of the slider is the case worth naming, because it is the one that licenses action. When almost all of A lies inside B, P(B | A) sits just under 1 and the sliver poking out is the set of exceptions. High plausibility. Not a guarantee, and close enough to act on.

That gives us a replacement for the inference rule logic used to supply:

The rule, restated for a world without certainty

1. P(B | A) ≈ 1
2. A is observed
3. B is probable

Fill it in with a diagnosis and it reads the way clinicians actually work. Headache, muscle fatigue, and a sore throat make flu highly probable given those symptoms; the symptoms are present; flu is the leading call. This is the structure behind most decisions anyone makes with incomplete information, and it took the whole apparatus of the last five sections to justify it.

One difference from the deductive version deserves attention, because it is the source of considerable trouble in practice. Deduction is monotonic: add premises and a valid conclusion stays valid forever. The probabilistic version is not. Learn that the patient's whole household has strep and P(flu | symptoms) drops, though nothing in the original inference was wrong. Conclusions reached this way are provisional by construction, and a system built to act on them needs somewhere to put the retraction.

Deduction accumulates. Inference revises. Every model in production is running the second kind of reasoning while its users assume the first.

The inference problem

Now the hard part, which the last section quietly assumed away. Line 1 of that argument requires a number. Where does P(flu | headache, fatigue, sore throat) come from?

Not from the definition. The ratio needs P(symptoms ∩ flu) and P(symptoms), and nobody hands those over. You have a finite pile of past cases, a world that keeps changing, and a symptom combination that may appear in your records a dozen times or never. Estimating that one number from data is the founding question of mathematical statistics, and later of machine learning. Everything else is technique.

So strip the problem to its bones. Forget symptoms and diseases, which come loaded with biology and judgement, and use the oldest abstraction in probability: balls and urns. Reducing a question to coloured balls in containers is not a simplification for the reader's benefit. It is how the mathematics gets done.

Two urns

Two urns sit in front of you, each holding fifty balls. Urn 1 has 40 red and 10 white. Urn 2 has 25 red and 25 white.

The toy problem

urn 1 · 40 red, 10 white · P(red | urn 1) = 0.80 urn 2 · 25 red, 25 white · P(red | urn 2) = 0.50

Read forward and there is no difficulty at all. Name the urn and the probability of drawing red follows by counting: 0.80 from the first, 0.50 from the second. Both numbers are exact, and neither required any estimation.

Now reverse it, which is the direction that actually matters. Someone draws a ball from an urn you did not see them choose. The ball is red. Which urn?

P(urn 1 | red) = ?

Intuition offers something before any arithmetic. Urn 1 is the richer source of red, so if she chose blind, the answer has to come out above 50%. That much is free, and it is worth having as a check on whatever the algebra produces. What intuition will not tell you is the distance above 50%, and the distance is the entire practical question — 52% and 95% both satisfy the intuition and would justify very different decisions.

So use the definition. It is right there, and it looks like enough.

The numerator is P(urn 1 ∩ red), the weight of one particular urn and a red ball occurring together. Nothing in the setup measures that. You never watched a long run of draws and tallied how often both happened; you were told two urn compositions, and those are conditionals pointing the wrong way — P(red | urn), the direction that was never in doubt.

The denominator is no better. P(red) is the weight of a red ball across every route by which one could arrive, and the routes are precisely what the curtain hides.

The definition is correct and both of its inputs are out of reach. That is the actual obstacle, and it is the same obstacle as the flu case in more honest clothing.

Reversing the implication

Here logic and probability part company for good, and the reason is almost embarrassingly small.

Sets do not care about order. A ∩ B and B ∩ A are the same region of the same picture, the crimson lens from every diagram in this article, which has no direction and no preferred reading. So the weight of that region can be factored two ways, and both must give the same answer.

That is the whole trick. Write the definition for the direction you want, then re-factor its numerator using the direction you can actually compute.

P(B | A) = P(B ∩ A) P(A) = P(A | B) · P(B) P(A)

The letters have been standing in for something particular since the first diagram in this article. A is the evidence, the thing you observe: a red ball, a wet sidewalk, a symptom, a flagged transaction. B is the hypothesis, the thing you want to know and cannot see: urn 1, rain, flu, fraud.

Which is a reversal of the roles they held in the opening sections, where A was the premise and B what followed from it. The swap is not sloppiness. It is the direction of reasoning turning around: deduction starts from the cause and derives the effect, inference starts from the effect and reaches for the cause. Same two events, opposite ends of the argument.

Written in those terms the theorem stops looking like algebra and starts looking like a job description, and each of its four pieces has a standard name.

Each term earns its place. The likelihood asks how readily this hypothesis would produce what you saw. The prior asks how seriously the hypothesis deserved to be taken before you saw anything. The marginal asks how ordinary the evidence is, and it is the term that punishes clues which every hypothesis would have produced anyway. The posterior is the answer, and it is the only one of the four that could not have been written down before the evidence arrived.

Run that update repeatedly as evidence arrives, each posterior becoming the next prior, and the practice has a name: Bayesian inference. Two of its four terms are also where it gets into trouble. The prior has to come from somewhere other than the data, which is the standing objection to the whole approach. The marginal, as the next section shows, is frequently impossible to compute at all.

Something worth noticing falls out of this. Precision is P(hypothesis | evidence) and recall is P(evidence | hypothesis), so the theorem is the exchange rate between the two numbers every model gets evaluated on. The sidewalk figures demonstrate it: recall of 0.95, times a prior of 0.10, divided by 0.23, returns 0.413 — the precision computed by counting mornings, six sections ago.

Read what that says against the article's opening. Logic held that a true implication licenses nothing in reverse, and it was right, because logic had no means of paying for the trip back. Probability does. The asymmetry has not disappeared (P(A | B) and P(B | A) still differ, exactly as the slider showed), but the gap between them is now a computation rather than a wall, and the toll is two extra quantities.

Logic could not run an implication backwards because it had nothing to pay with. Probability can, and the currency is the prior.

This is precisely what the urn problem needs, and the quantities it asks for are the ones already sitting in plain view. Both urn compositions are exact fractions, obtained by counting and nothing else.

P(red | urn 1) = 45 P(red | urn 2) = 12

The first goes into the numerator of the reversal, since urn 1 is the hypothesis on trial. The second is needed too, though it is easy to overlook: without knowing how readily the rival hypothesis produces red, there is no way to work out how often red appears at all, and that total is the denominator.

Red is more likely from urn 1, so urn 1 is the better guess. By how much is a different question, and the honest answer is that the information given is not sufficient to say. Both urns produce red balls. The evidence narrows nothing to certainty, and to turn "better guess" into a number you need one more ingredient that has not appeared anywhere in this article yet.

The missing ingredient

Add one sentence to the setup and the problem becomes solvable: she picks the urn at random, each equally likely. That sentence is not a detail about the puzzle. It is the whole of what the evidence cannot tell you, stated in advance — the weight each hypothesis carried before any ball was drawn. Call it the prior.

With it, every term is available. The likelihood was 0.80 by counting. The prior is 0.50 by assumption. Only the denominator takes any work, and the second axiom supplies it: a red ball arrives by one of two disjoint routes, so their weights add. Half the time she reaches into urn 1 and draws red at 0.80; half the time urn 2 at 0.50. That gives P(red) = 0.5 × 0.80 + 0.5 × 0.50 = 0.65.

Substitute into the reversal and the numerator is 0.80 × 0.50 = 0.40.

P(urn 1 | red) = 0.400.65 = 8/13 ≈ 0.615

Check it by counting instead. Across both urns there are 65 red balls, 40 of them in urn 1, giving 40/65 — the same fraction. The shortcut works here because the urns hold equal numbers of balls and were equally likely to be chosen. Change either condition and only the ratio survives.

The intuition was right, then, and not by much of a margin. It promised something above 50% and delivered 61.5%, which sits closer to the guess than to certainty.

That gap between what the calculation felt like it should give and what it actually gave is where the useful lessons hide. Two of them are worth pulling out.

What the number is telling you

Belief started at 0.50 and ended at 0.615. The evidence was worth about eleven and a half points, which is a thin return on what felt like a strong clue. Red is 60% more likely from urn 1 than urn 2, and yet the conclusion barely moves.

The reason is the thing every practitioner eventually learns the hard way. Evidence discriminates only to the extent that the competing hypotheses differ in producing it. Urn 2 makes red balls half the time, so a red ball is close to unremarkable, and unremarkable evidence cannot carry much inferential weight no matter how it feels. To move belief decisively you need evidence that one hypothesis produces and the other almost never does.

And the prior keeps its grip throughout. Hold the red ball fixed, vary only what you assumed before the draw, and the answer travels most of the unit interval.

Same red ball, five different starting assumptions

Prior on urn 1	What it encodes	Posterior
0.10	She almost always reaches for urn 2	0.151
0.30	A mild preference for urn 2	0.407
0.50	No preference, the stated problem	0.615
0.70	A mild preference for urn 1	0.789
0.90	She almost always reaches for urn 1	0.935

Put the urn numbers into those four slots and the whole calculation fits on one line. Prior 0.50, likelihood 0.80, marginal 0.65, posterior 0.615. The marginal is the only one that took any assembling, and its job is to keep the posteriors across all hypotheses summing to 1 — urn 1 at 0.615 and urn 2 at 0.385.

The evidence supplies the likelihood. It cannot supply the prior, and the prior is often doing more of the work.

The intractability of evidence

The marginal gave no trouble in the urn problem. Two hypotheses, two routes to a red ball, add the weights: 0.65. Done in a line.

The reason it was a sum at all is worth seeing rather than deriving. The hypotheses carve up the sample space between them: every draw comes from urn 1 or urn 2, never both and never neither. The evidence is a region laid across that partition, and the dividing line cuts it into pieces.

Why the marginal is a sum

Each piece is a joint weight, prior times likelihood, so adding them gives P(evidence) = Σ P(evidence | hypothesis) × P(hypothesis) across the partition. With two urns that is two terms.

Two hypotheses is a luxury of toy problems. Real inference partitions the space into as many cells as there are candidate explanations, and the evidence lies across an arbitrary number of them.

The same picture with six hypotheses

Six hypotheses, five of them crossed, so the marginal is a five-term sum. The strip left untouched deserves a moment: its likelihood is zero, which is the refuted case from earlier arriving as a consequence rather than an assumption. Evidence rules hypotheses out by failing to overlap them.

Now let the hypothesis stop being a label. If what you are inferring is a multi-parameter model rather than a choice between two urns, the partition is no longer a handful of strips but a continuum, and the sum becomes an integral over every parameter at once.

A model with twenty parameters gives a twenty-dimensional integral. A few hundred parameters puts the calculation beyond any method available, and production models carry parameters by the million. The phenomenon has a name: the intractability of evidence, and it is not a computational inconvenience to be engineered around but a hard wall.

Then comes the part that rescues the whole enterprise, and it costs nothing. Look at what the marginal depends on: the evidence, and nothing else. No hypothesis appears anywhere in it, so it is the same number for every candidate you might be weighing. A quantity that is identical across all your options cannot change which option wins, and can be dropped.

P(B | A) ∝ P(A | B) · P(B) posterior ∝ likelihood × prior

Score the urns this way and the arithmetic gets easier than it already was. Urn 1 gets 0.80 × 0.50 = 0.40, urn 2 gets 0.50 × 0.50 = 0.25, and urn 1 leads by a factor of 1.6. The ranking is settled and the intractable term was never touched.

If you want calibrated numbers rather than a ranking, and the hypothesis space is small enough to enumerate, add the scores and divide: 0.40 and 0.25 over 0.65, returning 0.615 and 0.385. This is not a trick specific to urns. A classifier's softmax is exactly this operation — unnormalised scores over hypotheses, divided by their sum, because the sum is the only route to a marginal anyone can afford.

The hardest term in the formula is also the one you can most often ignore. What you cannot ignore is what happens when you forget you ignored it.

Which is the failure mode worth carrying away. Dropping the marginal costs you nothing when you are ranking and everything when you are deciding. A fraud queue sorted by unnormalised score works fine. The same score compared against a fixed threshold, or multiplied by the cost of a false positive, is meaningless — the number is not a probability, it only orders like one. Systems that rank well and calibrate badly fail precisely at the point where the decision gets made, and the symptom is a threshold that has to be re-tuned every time anything upstream shifts.

Making the decision

All of which leaves a practical question unanswered. You have several hypotheses and a way to score each one. Something still has to be chosen.

The obvious rule is to take the hypothesis with the largest posterior, and since the marginal is a constant shared by all of them, that means taking the largest product of likelihood and prior. The rule has a name.

For the urns it returns 0.40 against 0.25, so MAP picks urn 1.

One word in that rule carries a great deal of weight, and it is easy to lose. The rule maximises the posterior, not the likelihood. Drop the prior and keep only the likelihood and you have a different estimator, maximum likelihood, which answers a different question: not "which hypothesis is most plausible given what I saw" but "which hypothesis would have produced what I saw most readily".

In the urn problem as stated, both rules agree, which is exactly why the distinction slips past unnoticed. Change one number and they part company.

Where the two rules disagree

Prior on urn 1	Maximum likelihood	Maximum a posteriori	Posterior for urn 1
0.50	urn 1	urn 1	0.615
0.39	urn 1	too close to call	0.506
0.10	urn 1	urn 2	0.151

Suppose your friend reaches for urn 1 only one time in ten. Red is still likelier from urn 1, so maximum likelihood still says urn 1, and it says so with the same confidence as before. MAP scores 0.10 × 0.80 = 0.08 against 0.90 × 0.50 = 0.45 and concludes urn 2, by a margin of better than five to one. The evidence pointed one way and the base rate pointed harder the other way.

The crossover sits at a prior of 5/13, about 0.385. Below it MAP and maximum likelihood give opposite answers on identical evidence.

Maximum likelihood asks which cause explains the evidence best. Maximum a posteriori asks which cause is most probably there. Only the second is the question anyone actually has.

This is the base-rate error with a formal name attached. Rare diseases, fraud, security incidents, anywhere the interesting hypothesis is also the uncommon one: these are precisely where maximising likelihood picks the dramatic explanation and gets it wrong most of the time. Maximum likelihood is MAP with the prior quietly set to uniform, which is a real assumption masquerading as neutrality.

What argmax throws away

One more caution, because MAP is a point estimate and points discard everything around them. Posteriors of 0.615 and 0.385 return urn 1. So do posteriors of 0.99 and 0.01. The decision is identical and the situation is not remotely the same, and the first case is close to a coin flip dressed up as an answer.

Take the argmax and you throw away the distribution the entire article was built to construct. Sometimes that is the right trade, because a system has to act and an action is a single choice. But the number that got discarded is the one that tells you how much to bet, when to escalate to a human, and whether the answer is worth having at all.

The fallacy, repaired

Which brings the argument back to where it started, with Holmes standing over a footprint.

The invalid move has a name and a shape. Take the rule "flu produces headache, sore throat and fatigue", which runs in that direction and is close to certain. Then observe the symptoms and conclude flu. Writing H for the hypothesis and E for the evidence, the reasoning is this:

The move logic refuses

1. H → E
2. E is observed
3. H

Now translate each line. The rule H → E becomes P(E | H) = 1, the containment case from the two extremes. Line 2 is the same in either language. And line 3, which logic cannot license at all, becomes something logic had no way to express.

The move probability permits

1. P(E | H) = 1
2. E is observed
3. H is more probable than it was

The algebra takes one line, and the picture makes it obvious.

P(E | H) = 1 ⇓ P(H | E) = P(H) P(E)  > P(H)

If P(E | H) = 1 then H sits inside E, so the intersection is all of H, so P(H | E) = P(H) / P(E). And since P(E) is at most 1, that quotient is at least P(H). Observing the evidence cannot lower your belief in the hypothesis, and it fails to raise it only when the evidence was certain to appear anyway.

The exceptionless sidewalk gives the numbers. Rain on 10% of mornings, wet pavement on 23.5%, and the rule never failing. A wet sidewalk takes belief in rain from 0.10 to 0.426, a fourfold move on an inference that classical logic dismisses as a fallacy.

The multiplier is 1/P(E), which is worth staring at. It depends only on how rare the evidence is. Evidence that shows up 80% of the time multiplies belief by 1.25 and tells you almost nothing. Evidence that shows up 12% of the time multiplies it by 8.3. This is the same lesson the red ball taught, now in closed form: what makes a clue valuable is not how reliably the cause produces it but how seldom anything else does.

Logic was right that the inference proves nothing. It was wrong to conclude that nothing follows. What follows is a factor of 1/P(evidence), and Holmes has been collecting it all along.

Where that leaves us

The same shape has appeared throughout: an effect observed, a cause inferred, a rule read against its direction. The sidewalk supports rain at 41.3%. The red ball supports urn 1 at 61.5%. Every time, what comes out is a weight rather than a verdict.

That is the trade the discipline made. Logic offered certainty on the condition that you only ever reason forward, from causes you already know to effects you can derive. Prediction requires the other direction, so certainty had to go, and what replaced it is a number that depends on something the data cannot hand you.

Which raises the question this article cannot answer. In the urn problem the prior was given, one sentence, exact. Nothing in production works that way. A fraud model's prior is a base rate that drifts every quarter; a diagnostic's prior depends on which clinic the patient walked into; a retrieval system's prior is whatever the index happens to contain this week. The machinery is sound and the input is a judgement call.

So: what is the base rate for the thing your model predicts, and when did you last check whether it had moved?

Where to go next

Three books, in ascending order of how much they will demand of you.