An order

Consistent and Undecided

Six candidates and five comparisons: eighty-three thousand outcomes contradict nothing, and seven hundred and twenty of those rank the field. The rest stop short.

at most 2562560 outcomes

Home · An order · Analysis

Six candidates, five comparisons. Eighty-three thousand nine hundred and fifty-two of the ninety-six thousand and ninety-six outcomes contain no ring of any length.

Seven hundred and twenty of them rank the field.

Three pages where most consistent outcomes decide nothing

line with a head: a pair compared; the head points at whoever came out behind
no line: a pair nobody compared
closed ring of three: a circular triple: no order can hold all three

Three outcomes drawn. Every row is the same number of candidates. Which candidate is which is the drawing's own choice and carries no meaning. If several people repeat this comparison, managers may compare this approach in a live test while interpreting every signal beside completed work and participant feedback.

Six candidates with five, seven and nine of their fifteen pairs compared. The orderable counts are eighty-three thousand nine hundred and fifty-two, four hundred and ninety-six thousand six hundred and eighty, and eight hundred and thirty-eight thousand one hundred and thirty. Readers examining the figures against practical research methods can also see Project Jupyter.

Computed from the three numbers alone. Who the candidates are and what any comparison was about is outside the record; the counts are of outcomes, not of anybody.

What the other eighty-three thousand do

They settle something and stop.

Five comparisons among six candidates is the fewest that could possibly determine an order, and it determines one only if the five form a single chain running through all six. Any other arrangement leaves at least two candidates with no path between them, and when no path connects two candidates, either of them could come first.

So the overwhelming majority of consistent outcomes on this page are partial: some candidates are placed relative to some others, and the field as a whole is not ordered.

The two columns, page by page

Six candidates, orderable against order-settling.

Five comparisons: eighty-three thousand nine hundred and fifty-two against seven hundred and twenty. Seven: four hundred and ninety-six thousand six hundred and eighty against thirty-two thousand four hundred. Nine: eight hundred and thirty-eight thousand one hundred and thirty against one hundred and fifty-one thousand two hundred.

One in a hundred and seventeen, then one in fifteen, then one in six. The proportion improves as the comparing goes on, which is the only thing in this record that straightforwardly gets better with more work.

Consistency is cheap and decision is expensive. Almost any set of comparisons avoids contradicting itself; very few of them say what the order is.

Why that asymmetry exists

Because consistency is a condition on what is there, and decision is a condition on what is missing.

To be consistent, the comparisons that were made must not form a ring. To decide the order, the comparisons must additionally cover every neighbouring pair in it — that is, nothing may be left out that the order depends on.

The first is easy to satisfy by comparing little. The second cannot be satisfied by comparing little, since it takes at least n − 1 comparisons simply to connect the field.

How many orders remain

The record does not tabulate it, and the shape of the answer is worth a paragraph anyway.

An outcome that falls into an order without settling it fits several orders — two, or six, or seven hundred and twenty if nothing was compared at all. How many orders still fit is a count of its own, and it is the natural measure of how much the comparing achieved.

What the record offers instead is the two ends of that scale: nought orders, which is contradiction, and exactly one, which is a ranking. Everything in between is counted as orderable and not distinguished further.

That is a real limitation and it is deliberate. Counting the orders consistent with each outcome is a harder problem than anything else here, and a record that did it would have one figure a reader could not check.

The trap in reporting

A set of comparisons that contradicts nothing feels finished.

It reads as success: no arguments, no circular triples, no awkward cases. And on the five-comparison page it is one hundred and seventeen times more likely to leave the field unranked than to rank it.

The record cannot say which happened in any real case — the walk says that, and so does simply trying to write the list down. What it can say is that the two outcomes feel identical from the inside and are very different, and that the difference is not what the obvious check is looking at.

Reading the three rows

Each row counts outcomes with no circular triple, which is a third quantity again and larger than both of the others.

Five comparisons: eighty-five thousand seven hundred and sixteen with no circular triple, eighty-three thousand nine hundred and fifty-two orderable, seven hundred and twenty deciding.

Seven: five hundred and forty-five thousand six hundred and forty, four hundred and ninety-six thousand six hundred and eighty, thirty-two thousand four hundred.

Nine: nine hundred and fifty-one thousand one hundred and ten, eight hundred and thirty-eight thousand one hundred and thirty, one hundred and fifty-one thousand two hundred.

Three columns, each a proper subset of the one before, and each answering a question the previous one cannot. Anybody quoting one of them owes the others.

What this means for a report

A report that the comparisons were consistent is close to no information.

A report that they produced a single order is a much stronger claim, and the two sound identical in ordinary language. The difference is not visible from inside either one: both end with a set of results that nobody objected to.

The record cannot tell which happened in any real case. What it can do is price the difference, and on the five-comparison page for six candidates the price is a factor of one hundred and seventeen.