In applied statistics we routinely compare two groups with the intention of ordering them (by location, scale, number of modes, whatever). Especially for ordinal data, it is attractive to come up with tests that do not simply reduce each sample to a single number. As political scientists have known for centuries, it can then be hard to ensure that the pairwise comparisons are consistent with any ordering of all possible samples or all possible underlying distributions. In fact, it is essentially impossible: without strong restrictions on the possible probability distributions, 'truly' ordinal tests will have the 'rock-paper-scissors' problem; non-transitivity. I will discuss both why this happens mathematically and why the inability to find truly ordinal non-parametric tests is a feature, not a bug.