Skip to content
Two at a Time

Fifty notes on comparing candidates two at a time

Every pair

One Figure, Several Meanings

Twenty-four is a count of outcomes, a count of circular ones and a count of orders in this record. Twenty-one figures here mean more than one thing.

at most 8064 outcomes

Home · Every pair · Caution

Somebody reports that twenty-four outcomes were of the kind they were looking for. What has been established?

Very little. Across this record, twenty-four is the number of outcomes with no circular triple when four candidates are fully compared; the number with two circular triples on the same page; and the number of orders four candidates can be put in. For repeated pairwise evaluations, operational workload insights can add project and time context without turning one activity signal into a verdict about quality.

Two rows reporting twenty-four, and one that does not

OUTCOMESHAPEOUTCOMESWITH cSHAREABCD4 / 6 / 0candidates / pairs / circles6424about 1 in 3ABCD4 / 6 / 2candidates / pairs / circles6424about 1 in 3ABCDE5 / 5 / 1candidates / pairs / circles80641560about 1 in 5
  • line with a head: a pair compared; the head points at whoever came out behind
  • no line: a pair nobody compared
  • closed ring of three: a circular triple: no order can hold all three

Three outcomes drawn. Rows differ in how many candidates. Which candidate is which is the drawing's own choice and carries no meaning. Readers examining the figures against practical research methods can also see Interaction Design Foundation.

The first two rows are the same page and report the same two figures while describing opposite outcomes. The third is shown for contrast.

Computed from the three numbers alone. Who the candidates are and what any comparison was about is outside the record; the counts are of outcomes, not of anybody.

Twenty-one figures mean more than one thing

The record carries three columns of counts — outcomes, outcomes with a given number of circular triples, and outcomes that fall into an order. Twenty-one values appear in more than one of those columns.

One, two, six, twelve, twenty, thirty, sixty, one hundred and twenty, one hundred and eighty: each is a count of three different kinds of thing somewhere here. A figure quoted without saying what it counts has not said much.

The two rows that read alike

Four candidates with every pair compared. Twenty-four outcomes contain no circular triple; twenty-four contain two.

Every printed figure is identical for those two shapes: sixty-four outcomes on the page, twenty-four of the kind being counted, a share of three in eight, twenty-four that fall into an order.

They are opposite situations. In the first, the comparisons rank cleanly and the order can be read off. In the second, there are two separate three-way circles in a field of four, which is as tangled as four candidates get.

Nothing in the figures separates them. The shape does — four, six, nought against four, six, two — and that is why the shape is printed on every row.

Two counts agreeing have agreed about a small integer. They have not agreed about anything that happened.

Where the two twenty-fours come from

Two arguments with nothing in common. That is the reason to call the agreement an accident and not a rule.

The left-hand twenty-four counts outcomes that fall into an order. With every pair compared, an outcome ranks cleanly exactly when it agrees with one of the orders, and four candidates have four factorial orders. Twenty-four.

The right-hand twenty-four counts outcomes containing two circular triples. Four candidates have four triples; getting exactly two of them to close requires a particular arrangement, and counting those arrangements gives twenty-four by a route with no factorial in it.

How much a pair of figures narrows things

More than one figure, and not all the way.

Taking the number of outcomes on the page together with the number of the kind being counted gives one hundred and twenty distinct classes for the record's one hundred and twenty-eight shapes. Eight shapes go missing in the collisions: the five pages where nothing was compared, which all report one of one; two shapes reporting twelve of twelve; and three further pairs on the fully compared pages of four and five candidates.

So two figures are nearly enough, which is exactly what makes the habit of printing three numbers hard to keep. A convention that is unnecessary ninety-four times in a hundred gets dropped, and then the ninety-fifth case arrives without warning.

What a figure does narrow

Something, and it is worth being precise rather than dismissive.

A circular-triple count on its own narrows one hundred and twenty-eight shapes to one of about a hundred classes — the record carries one hundred and two distinct values in the circular column, and the busiest of them covers five shapes.

The number of outcomes is coarser: page totals repeat across the table, since a page is fixed by two numbers and different pairs of numbers can give the same product.

So a count of the kind being looked for is nearly identifying, a page total is not, and the pair of them together is better than either. The shape beats all of them, because the shape is what produced them.

The third row

Five candidates, five pairs compared, exactly one circular triple: one thousand five hundred and sixty outcomes of eight thousand and sixty-four.

Nothing else in the record reports that pair of figures. The shape is named exactly by them, and so are one hundred and nineteen others.

That is a property of where the table stops rather than of the arithmetic. Widen the table and more figures would collide; narrow it and fewer would. Safest to assume any figure has company elsewhere, since most of them do.

What follows for the rows

Three integers get printed beside every figure on this site, including the many rows where nobody needs them.

A pair of figures pins down one hundred and twenty of the hundred and twenty-eight shapes, and that is precisely the difficulty: a safeguard that earns its keep eight times in a hundred and twenty-eight gets quietly abandoned, and the one case it existed for turns up unannounced.

So the shape is printed whether or not it is needed. The eight occasions when it is the only distinguishing mark on the row are what the other hundred and twenty pay for.

Near this one