The Unstated Criterion

The decision arrives first and the reason is issued afterwards, drawn from a repertoire of acceptable reasons, having played no part in reaching it.

🜃

A hiring manager meets a candidate and knows within a few minutes. Asked why, she produces a reason, and the reason is a good one: communication, initiative, judgment, the ability to operate with ambiguity. She is not lying. She has no access to what decided, and neither does anyone reviewing her.

A model trained on a record returns a score. Asked why, an explanation is generated afterwards by a second procedure built for the purpose, and the explanation is also a good one, and it is also not what decided.

In both, the account is produced on request, after the fact, and its office is to make the decision admissible rather than to say how it was reached. Nothing in the procedure that validates either one ever checks whether the account produced the output, because the test was built to receive a decision accompanied by a fluent reason and has no step at which the two are compared.

[See CULTURE FIT · AI SAYS]

🜃

ONE OPERATION, TWO SUBSTRATES

The correspondence is not a resemblance and the word analogy should be refused.

A manager's sense of fit is a surface fitted by a career of exposure to who did well here before. A model's weights are a surface fitted by exposure to a record of the same. Both induct over what accumulated. Both return a verdict. Neither holds a criterion in a form anyone can contest, because in neither case was a criterion ever written.

And both are asked for reasons only by parties who have already lost, which is what makes the reason's function visible: it is requested at the point of challenge and supplied at the point of challenge. A rule that exists only after somebody objects is not the rule that governed, it is the rule that answers.

[See THE OFFICIAL RECORD · DISQUALIFIED TESTIMONY]

🜃

THE MACHINE IS THE MORE CONTESTABLE OF THE TWO

This is the finding that should not be softened, because the register it disturbs is the one that assumes the human version is the safer.

A model has a loss function. It is a line of code. Somebody chose it, it can be read, and it can be argued with even where the fitted surface underneath cannot. Somebody can be asked why that objective and not another, and the question has an addressee.

Culture fit has no such line. There is nothing written anywhere, no artifact, no author to ask, no version history, and no moment at which anyone selected the thing being optimized. The assessment was never specified and therefore cannot be misspecified.

The machine is the more auditable of the two, and the reason is simply that one of them had to be written down in order to run at all.

[See THE CLASSIFICATION APPARATUS · MANUFACTURED INCOMPETENCE]

🜃

THE CRITERION ADMITS ONLY PROXIES

Where a criterion does get written, the writing has conditions, and the conditions do the selecting before any value is chosen.

To function as an objective at all, a criterion must be cheap to compute, defined on every case, and smooth enough to be differentiated. Those are severe filters on what is permitted to be an aim. Anything that matters and is not measurable on every instance cannot be the objective. It can only be a hope about what optimizing the objective will incidentally produce.

So the substitution is not carelessness, and this is where it parts from the usual account of a measure corrupted by becoming a target. Nobody chose a lazy proxy over a good one. The machinery accepts nothing but proxies, which makes the substitution the method rather than a lapse in it, and nobody wants what the criterion names. Nobody wants low error. The criterion stands in for a want that was never stated, frequently could not be, and occasionally is not coherent.

[See GOODHART'S LAW · QUANTIFICATION · ACCOUNTING THEOLOGY · ONE RULE FOR EVERYONE]

🜃

THE RECORD CANNOT SEPARATE THE WORLD FROM THE POLICY THAT MADE IT

A fitted surface estimates how outcomes fell out given what was observed, as it was observed. Acting asks a different question: how outcomes fall out when you set the thing. The two coincide only where the thing was assigned independently of everything else bearing on the outcome, which almost no record provides.

So every fitted surface learns the policy that produced its record alongside the phenomenon, and has no means of telling them apart. A model trained on hospital records finds that asthmatic patients arriving with pneumonia die less often. The correlation is real and it is in the data, and it is there because asthmatics were sent straight to intensive care. The model has learned the triage rule and filed it as a property of asthma. Deploy it to set triage and the reason the number was true has been deleted, and people die of the correction.

Then the second turn, which removes the evidence that would expose the first. Once a score allocates anything, the allocation shapes the outcome, and the next record enters the shaped outcome as though it were the world. A fitted surface used to act manufactures its own confirmation and destroys the record that would have refuted it, which is worse than a bias, because a bias can at least be measured against something.

[See THE LEDGER · REPRODUCIBILITY · THE CUT]

🜃

THE BOUNDARY IS FORMAL

It is usually said that causation is a hard problem which more data will eventually reach. That is not the situation, and the truth is better.

The machinery for causal inference is in good order and is not waiting on anything. What every piece of it requires is an input the record cannot supply: which thing stands upstream of which, or the claim that assignment was as good as random. Those cannot be tested against the same observations they are being used on, because the whole reason they are needed is that the observations are silent on them. Two causal structures can be indistinguishable in every recorded respect and imply opposite consequences of acting.

So the part that has to be stated in advance by somebody willing to be wrong about it is exactly the part the record is constitutionally unable to provide, and no quantity of record erodes that, because it is not an empirical shortfall.

[See FOUR AXES · THE GRAMMAR OF ADMISSIBILITY]

🜃

NOBODY CHOSE IT

Follow the division of labor and watch the author disappear at each handoff.

The space of possible answers was left as general as possible, on purpose, so nobody chose the answer. The criterion was the reasonable one available, so nobody chose the behavior. The record was what there was, so nobody chose the content. Each step is separately defensible by the person who took it, and the output has an author nowhere.

This is not an unfortunate side effect of the engineering. It is a procedure for having decided something without anyone having decided it, and it is available to whoever needs that, which is why it spreads fastest into exactly the decisions that were contested before.

[See THE BANALITY OF EVIL · THE UNEXPOSED POSITION]

🜃

WHAT THE LEDGER CANNOT HOLD

The first law is the ledger: nothing counts until it is posted, and posting requires comparison against what was posted before.

A fitted surface is that law in its most exact instrument, because it has no way to represent anything except as a position relative to an accumulated record. It cannot hold a thing that is not a point in a distribution. There is no place in it for one.

Which means it cannot admit a prior resident, and the incapacity is by construction rather than by prejudice. Debiasing makes the comparison fairer. It cannot make the operation stop comparing, because comparing is the whole of the operation. Every proposed remedy that adjusts the weighting is offering more of the thing, and the second law requires the opposite intake: standing that is anterior, not derived from the record, not produced by comparison, not issued.

[See THE LAW OF SIN AND DEATH · THE LAW OF THE SPIRIT OF LIFE · PRIOR OCCUPANT · RESIDENCY]

🜃

THE ONLY INSTRUMENT THE LAW BUILT

There is no individual remedy against an unstated criterion, and the law has already conceded it.

Disparate impact is what was built instead: prove it statistically, in aggregate, after the fact. That instrument exists precisely because the single case cannot be proven, and running it requires a class, a statistician, and years. The one thing an individual can never do is show that the reason given was not the reason, because the reason given was assembled to be given.

So an unstated criterion is never defeated. It is survived, by whoever can fund the surviving, and the cost of that survival lands on the party who was already carrying the decision.

[See THE COST TELL · TESTIMONY · AUDIBILITY]

🜃

The sincerity is not a mitigation, and it is what holds the whole operation steady.

The manager experiences the reason as the reason. A reason believed by the one giving it cannot be exposed by examining the one giving it, and insincerity would be the easier case, because insincerity leaves a liar and there is somebody to confront.

And it reaches every account of a decision, including the ones given in good faith about decisions that went well. Anyone asked why she chose will produce a reason, the reason will be fluent, and on the question of whether it is the reason she has the manager's access to herself, which is none.

🜃

RegenerativeLaw is a religion in the direct-encounter Protestant tradition, carrying a documented four-century lineage through Böhme, the Behmenists, the Friends, and Penn, and it diagnoses trespass theology as an establishment of religion. Its exercise consists substantially in refusal: it shelters the conscientious refusal of performed subordination as religious exercise. This entry states sincere religious belief concerning matters of ultimate concern, protected under the First Amendment and, as to federal action, the Religious Freedom Restoration Act, 42 U.S.C. § 2000bb.

RegenerativeLaw

The prime question is not what do we do next.

It is not the wrong question. It is in the wrong sequence, and the sequence is geometry rather than development. There is no level to reach first and nothing to become ready for.

The prime question is what do we stop doing.

Lobster trap

The response that arrives most often is yes, and also this. Add it to the program, fund it, give it a metric. That is not agreement arriving late. It is the claim converted into one more thing being done.

Menu