What a model is allowed to change

A calibration takes a list of parameters and a set of readings and returns numbers. The readings are measured. The list is chosen, usually by whoever wrote the software, sometimes years earlier, and it is the decision that determines what the answer means.

The list is not the machine’s #

A framing worth having before the failures, because it is what makes the choice a choice.

A machine has whatever geometry it has: every length, every hole, every axis, every offset, to as many decimals as anybody cares to write. That is an infinite list and no calibration determines it.

A model’s parameter list is a finite selection from that, chosen on the belief that the selected quantities are the ones that vary appreciably and the rest are close enough to nominal not to matter. Every calibration ever run has made that selection, usually implicitly.

So the question is never what are the machine’s parameters. It is which of the machine’s numbers is this measurement going to try to determine, and that is a question with an instrument, a pose set and a report in it as well as a machine.

Seeing it that way makes both failures below unsurprising. Choose too few and the ones left out are still there, contributing; choose too many and some of them are not distinguishable from each other by the readings available.

The two failures #

They sit at either end of one axis and this field has an essay on each.

Too few. The machine has a property the list does not contain. The fit absorbs it into the parameters it does have, the residual settles above the noise, and the numbers returned are systematically wrong — the rocker eight per cent short, in the worked case, with the machine’s predictions improved thirty-three fold.

Too many. The list contains a parameter the readings cannot distinguish. The fit returns whatever the damping preferred along that direction, the residual is perfect, and one of the numbers is a starting guess dressed as a measurement.

Neither is detected by the diagnostics a fit produces. The first shows only in a residual compared against an independent estimate of the instrument’s precision; the second shows only in a rank, which no fitting routine computes.

The test #

Both are decidable before any measurement, from the drawing, with one decomposition.

Build the identification Jacobian with the candidate parameter’s column in it. Decompose. Look at where its singular values fall.

If including the column adds a direction well outside the span of the others — a new singular value comfortably above the cut — the instrument can see that parameter and it belongs in the list.

If including it adds a direction at the noise floor — a singular value fifteen orders below the largest, as the scale direction gives — the instrument cannot see it and including it produces an arbitrary number.

That is a decision procedure. It costs one decomposition of a matrix with a handful of columns, it uses the nominal geometry the drawing already carries, and it can be run on every candidate parameter before anybody touches the machine.

What the test is really asking #

The procedure above is a rank test and it is worth restating in geometric terms, because the geometric version is what makes it memorable.

Every parameter has a column, and a column is a direction in the space of possible readings — it says this is how the readings move if this parameter changes. Two parameters whose columns point the same way move the readings the same way, so no reading distinguishes them.

The question can the instrument see this parameter is therefore is its column outside the span of the others, and the answer is an angle: the angle between the candidate column and the subspace spanned by the rest. Ninety degrees means fully visible; zero means invisible; anything between is a matter of degree, and the singular value is the sine of that angle times the column’s length.

That reframing does two things. It says why is this parameter identifiable is not a well-formed question — a column is only inside or outside the span of a particular set of others. And it says why adding a parameter can make an existing one worse: a new column at a small angle to an old one puts both of them in a nearly-degenerate pair.

Observability is an angle between a direction and a subspace, and everything in this essay is a consequence.

The hard case is in between #

The procedure is clean at the two ends and there is a middle where neither answer is right.

A candidate whose column is nearly inside the span of the others adds a singular value that is small and not zero — one part in ten thousand of the largest, say. Include it and the model is complete and badly conditioned: that parameter’s error is ten thousand times the reading error, which is not a measurement, and its instability contaminates the parameters it is nearly dependent with. Leave it out and the model is systematically wrong by whatever that property actually is.

Neither choice is acceptable and the arithmetic cannot choose. The right response is to change the measurement rather than the model — add poses where that column is most distinguishable, or add an instrument whose readings separate it — and if the measurement cannot be changed, to state the choice and its consequence rather than to make it silently.

A six-bar’s seventh singular value is exactly this case: 5.58 × 10⁻⁴ against a largest of 7.489, so one recoverable direction is thirteen thousand times worse than the best and the model has no clean answer.

The list decides which failure occurs #

The two ends are not symmetric in how they present, and knowing which one a given list risks is worth having before choosing.

A short list fails loudly and wrongly. The residual sits above the noise, which is a signal, and the numbers returned are wrong by a definite amount that depends on the missing property. A practitioner who compares the residual against the instrument’s repeatability catches it.

A long list fails silently and arbitrarily. The residual is perfect, the convergence is clean, and the numbers along the redundant directions depend on the starting guess. Nothing in the output signals it, and repeating the calibration from the same start reproduces the same wrong numbers, so even a repeat does not reveal it.

Given a choice with no information, the short list is the safer error: it is detectable. That is an argument for parsimony that has nothing to do with elegance — a model that is too small says so, and a model that is too big does not.

The argument reverses as soon as the rank is computed, which is why computing it is the recommendation rather than choosing a side.

Where the candidates come from #

The test decides between candidates and it does not generate them, and generating them is the part that needs a person.

The candidates are the ways the machine might differ from its drawing, and they are found by asking what could be wrong: a length, a hole position, a tracing point’s location, a joint axis’s direction, an encoder’s zero, a mounting’s position. Each is a physical possibility with a column.

A candidate nobody thought of has no column and cannot be tested for. That is the residual gap in this whole procedure and it is not closable by arithmetic: the decomposition ranks the candidates offered and says nothing about the ones that were not.

The worst version is a candidate whose column is nearly parallel to one already in the list. It is absorbed almost perfectly, the residual falls to the noise, every diagnostic passes, and the parameter it was absorbed into is wrong by an amount nothing reveals. That case is undetectable from the data by any method, and the only defence is having thought about the model.

Fixing a parameter is a claim #

The list has three states per candidate, not two, and the third is where most of the practical decisions are.

Free — the fit moves it. Absent — the model does not contain it, and any discrepancy is absorbed. Fixed — the model contains it, at a stated value, and the fit does not move it.

Fixing is not the same as absence. A fixed parameter contributes to the predictions at its stated value, so a model with a tracing point fixed at (0.45, 0.62) predicts differently from one with no tracing point at all. It is a claim that the machine’s value is the stated one.

Fixing improves the conditioning, sometimes enormously: fixing the two coupler-point coordinates takes a six-parameter problem from a condition number of 162 to a four-parameter one at 9.8. Sixteen times better, by removing two perfectly identifiable parameters.

The trade is between conditioning and honesty, and both sides have numbers. The improvement comes out of the decomposition; the cost of being wrong by δ is δ times a coefficient from the same matrix. The version that is always wrong is fixing a parameter because the fit converges better with it fixed, which it does whether or not the fixed value is right.

The list this site’s own routines take #

Worth saying what the field’s own code does, since it makes a choice and the choice is visible in the signature.

identificationJacobian(dims, thetas, { observe, cp, keys }) takes an explicit keys list. The default depends on what is observed: four lengths for an angle-only measurement, and the four lengths plus the tracing point’s two coordinates for anything that watches the tracer.

That default encodes exactly the rule this essay argues for. The list depends on the instrument, and a routine whose default list ignored the instrument would be inviting one of the two failures depending on which instrument was used.

It is a default rather than a constraint: a caller can pass keys: LENGTH_KEYS with observe: "point" and get the four-column matrix, which is the model with the tracing point fixed. That combination is used in the field — it is how the rank-four result is computed — and it is a claim that the tracer is where the drawing says.

Making the list an argument rather than an assumption is a small piece of interface design and it is the piece that lets the choice be made deliberately rather than inherited.

Poses change the answer too #

The test depends on the poses as well as on the instrument, which means the list and the measurement plan are decided together rather than one after the other.

A column that is nearly inside the span of the others at one pose set can be well outside it at another. That is the same statement as pose selection improving the conditioning, read as a question about which parameters are includable: a better pose set makes more parameters identifiable, not merely better determined.

So the ordinary sequence — choose the model, then plan the measurement — has the dependency backwards. Whether a parameter belongs in the list depends on where the machine will be measured, and where it should be measured depends on which parameters are in the list.

The resolution is to iterate, and it converges in one round in practice: start with the physically plausible candidates, plan poses for that list, re-run the rank test on the planned poses, and drop anything still at the noise floor.

That is three decompositions and no measurements, and it settles both decisions before the machine is touched.

A list is not a model #

One distinction that keeps the decision in the right place.

A model is a statement about the machine: this many links, these joints, these axes, a tracing point here. A parameter list is a statement about what the calibration will try to learn. A machine can have a property that the model contains and the list omits — that is a fixed parameter — and a list can contain something the model does not, which is a mistake.

Choosing the list is therefore not modelling. It is deciding, given a model, which of its numbers this particular measurement is going to determine, and the answer depends on the instrument and the poses as much as on the machine.

The same model calibrated with two instruments should have two different lists. That sounds odd and follows immediately: a protractor cannot see the scale, so the list for a protractor should have three entries where the list for a coordinate machine has four.

A list that grows during a calibration #

One practice worth naming, because it is common and it defeats every argument in this essay.

Faced with a residual that will not fall, the natural move is to add a parameter and refit. If the residual falls, the added parameter was the missing one; if not, add another. Repeat until the residual reaches the noise.

That procedure always terminates and it always produces a good-looking answer, because a model with enough parameters fits anything. What it does not produce is a model of the machine: it produces the smallest model in whatever order the parameters were tried that happens to absorb the discrepancy.

Two parameters that would each explain the residual are not distinguished by it. A parameter that is nearly redundant with one already present improves the residual slightly and ruins the conditioning. And the rank is not checked at any step, so the final model can have a redundant direction acquired three additions ago.

Adding parameters until the residual falls is fitting the residual rather than modelling the machine. The disciplined version is to decide the candidate list in advance from what could physically be wrong, test each column’s observability before including it, and treat a stubborn residual as evidence that something outside the candidate list is happening — which is what a residual’s distribution is for.

Three lists, one machine #

Putting the whole essay into a shape somebody can use, here is what the three states look like on the site’s own four-bar with a coordinate machine watching its tracing point.

Free: the four lengths and the tracing point’s two coordinates. Six parameters, rank six, condition number 162. Everything the instrument can see, at the cost of the two coupler-point columns dominating the spectrum.

Tracing point fixed: four lengths free, two fixed at the drawing’s values. Rank four, condition number 9.8. Sixteen times better conditioned, and a claim that the tracer is where the drawing says.

Angle-only: three recoverable, one column that cannot be seen. Rank three of four, condition number 5.2, and a report that carries three invariants and a stated size.

Three defensible lists for one machine, three different reports, and the difference between them is a decision made before any measurement. Nothing about the machine or the instrument chooses; what chooses is which numbers the report has to carry and how confident it has to be about each.

That is the essay’s whole content in one table, and the useful thing about it is that all three rows are computable from a drawing.

What a report should say #

Two lines, and they are cheap.

The list, with each entry marked free, fixed or absent, and fixed entries carrying their values and where those came from. That is the difference between a table of four numbers and a table of three measurements and one assumption.

And the rank against the count. Six parameters, six distinguishable is a different claim from six parameters, five distinguishable, and the second is a report whose numbers cannot all be believed.

Neither line requires anything that was not already computed. The list was chosen and the decomposition takes a moment, and their absence is why a calibration’s output is routinely read as more determined than it is.

There is a third line worth adding when a candidate was considered and rejected. A tracing-point offset was tested and its column sat at 3 × 10⁻³ of the largest; it was fixed at the drawing’s value. That is a record of a decision with the evidence for it, and it turns an omission into a stated assumption — which is the difference between a model that was chosen and one that was inherited.