Six things a measurement cannot tell you

Six things that get said about measuring a mechanism, each of them reasonable, each of them true of some other kind of measurement, and each answered here with a number from the machine’s own identification Jacobian.

One: a small residual means a good answer #

It means the model fits the data. Whether the answer is determined is a different question with a different instrument.

Take this four-bar built two per cent long on the crank, three per cent long on the coupler and one per cent short on the rocker. Read its output angle at thirty positions and fit from the nominal. The residual reaches 1.8 × 10⁻¹⁶ radians — the solver’s own floor — and the four lengths that come back are

3.96916    1.01214    3.57720    2.94710

against the machine’s 4, 1.02, 3.605, 2.97. Every one of them is the truth multiplied by 0.9922894, to fifteen figures.

The convergence is clean too: six steps, monotone, with the last two falling faster than the ones before, which is what a Gauss-Newton descent does when the Jacobian has full rank on the directions it is allowed to move in. Every ordinary diagnostic passes with distinction.

The shape is exact and the size is the starting guess, and the residual cannot say so because the objective is flat in that direction. The instrument that can is the rank of the identification Jacobian, which no fitting routine computes.

It means the model fits the data. Whether the answer is determined is a different question with a different instrument.

3.96916    1.01214    3.57720    2.94710

against the machine’s 4, 1.02, 3.605, 2.97. Every one of them is the truth multiplied by 0.9922894, to fifteen figures.

Two: a full rank means one answer #

It means the answer is locally determined. It says nothing about another answer somewhere else.

Three cognate four-bars trace one coupler curve. Their cranks are 1.600, 2.214 and 2.558 — a spread of sixty per cent — and the identification Jacobian at each of the three is of full rank with an unremarkable condition number.

A derivative describes a neighbourhood. An alternative that is not in the neighbourhood leaves no trace on any derivative, so the rank, the null space, the condition number and all five observability indices pass at every one of the three.

The same is true of the smaller instance: one set of four lengths and two assemblies. Three measured angle pairs return this machine’s lengths to fourteen figures, and the linkage assembled the other way misses every reading by up to 268°. The parameters are right; the machine is not; and no rank anywhere in the calculation is deficient.

The instrument for a discrete alternative is a multi-start search, which samples, or an algebraic construction, which is complete and exists only sometimes.

It means the answer is locally determined. It says nothing about another answer somewhere else.

The instrument for a discrete alternative is a multi-start search, which samples, or an algebraic construction, which is complete and exists only sometimes.

Three: an improvement proves the parameters #

It proves the predictions.

Give a four-bar’s tracing point an offset of 0.198 units and fit a model that has only the four lengths. The fit reduces the prediction error over the measured half-turn by a factor of thirty-three and over the unmeasured half by twenty-one. Every practical test says it worked.

The rocker it returns is 2.7526 for a machine whose rocker is 3.0000 — eight per cent short.

The improvement is real, repeatable and durable: refit the same machine a month later and the same numbers come back, and a second machine of the same design improves by the same order. As a device for making a machine predict better the calibration works exactly as advertised.

A predictive calibration and a metrological one are different products of one procedure, and only the first is tested by the improvement. The second is tested by the residual, which here settles at 4.2 × 10⁻³ instead of at the instrument’s floor and is the only signal there is.

It proves the predictions.

The rocker it returns is 2.7526 for a machine whose rocker is 3.0000 — eight per cent short.

Three and a half: a better instrument fixes it #

A corollary of the third that is worth its own number, because it is the response the third invites.

Faced with a fit that improves the machine and returns a wrong parameter, the natural move is to measure better. A more accurate instrument, more poses, more repeats.

None of it helps. The error is systematic: it comes from the model rather than from the data, so it is reproduced identically by every reading and averages away at exactly zero rate. Ten thousand readings of a model missing a parameter converge to the same wrong answer with a very small standard error.

What a better instrument does buy is visibility. The residual that will not fall is only a signal when it can be distinguished from the instrument’s own noise, so an instrument good to 10⁻⁵ makes a 4.2 × 10⁻³ residual unmistakable and one good to 10⁻² hides it entirely.

A better instrument makes the problem detectable and not smaller, which is an unusual property for accuracy to have and is worth expecting whenever a residual sits above the noise.

Four: a classification is a result #

It is a result and it is not a measurement until it carries a margin.

This four-bar is a crank rocker. Sorted, its lengths are 1, 3, 3.5 and 4, so s + l = 5 against p + q = 6.5 and the margin is 1.5 units — a hundred and eighty times a ±0.01 tolerance, which is comfortable.

A four-bar at 4, 1, 3.4, 2.55 is also a crank rocker, with a margin of 0.05. One and a quarter tolerance bands from being a different kind of machine.

Both report the same label and one of them is a design decision. A count or a class with no margin is a claim with no evidence about how nearly it is false, and grashof already returns the margin.

Four and a half: a tighter tolerance protects the class #

The corollary of the fourth, and it is the reasonable-sounding response that does not work as expected.

Given a four-bar with a margin of 0.05 units, the instinct is to tighten the tolerance on the four lengths until the margin is safe. That works, and the arithmetic of how much is not what a reader expects.

The margin is s + l − p − q, and its sensitivity to each of the four lengths is ±1. So a ±δ tolerance on each of the four gives the margin a worst-case band of 4δ, and a margin of 0.05 needs δ under 0.0125 to be certain of the class.

That is a tolerance ten times tighter than the ±0.01 the machine’s output band was computed with, on all four lengths, to protect a classification rather than an accuracy.

The margin’s sensitivity to every length is one, which is larger than any of the output angle’s sensitivities — 0.29, 0.34, 0.37, 0.15 at a crank angle of one radian. So the classification is the more tolerance-critical quantity on a machine with a small margin, and no ordinary tolerance study looks at it.

Five: a plan’s score is the measurement’s score #

The score was computed on the plan and the measurement is a different set of rows.

A plan of eight chosen poses on this four-bar achieves an observability of 0.3017. Execute the same eight on a double rocker at 4, 3.2, 1.4, 3.0 — a machine that reaches only part of a turn — and five of the eight do not exist: the matrix has three rows, and its observability is 0.0268.

A factor of eleven, and nothing in the pipeline notices, because the number was computed before the measurement and never recomputed after. A pose the machine refuses is dropped and counted, and the count is the diagnostic.

The repair is one more decomposition: score the matrix that was built, not the plan that was drawn.

Six: enough data settles it #

Not if the model is missing something, and not if the errors are systematic.

A systematic reading error — a protractor with a zero offset, a machine that was warm for the first ten poses — does not average away. Repeating the measurement a thousand times reproduces it a thousand times, the residual settles at whatever the systematic error is, and the answer is displaced by an amount that does not shrink with n.

That is the easier half of this claim’s failure and the harder half is the model.

A property the machine has and the model does not has no column, so no rank sees it, no condition number sees it, and no search over the parameter space finds it — the search is over a space that does not contain it.

What is left is the residual, and the residual’s power depends on how nearly the missing property resembles a parameter already present. A missing property whose effect is nearly parallel to an existing column is absorbed almost perfectly: the residual falls to the noise, every diagnostic passes, and the parameter it was absorbed into is wrong by an amount nothing reveals.

There is no bound on how wrong. The absorption is exact in the limit of parallel columns, so an arbitrarily large error in the machine can produce an arbitrarily small residual. That is the field’s third limit and it is not detectable from data by any method.

Six and a half: a self-calibration needs nothing from outside #

The sixth has a variant that deserves its own line, because it is believed by people who would not fall for any of the others and because the counter-example is the most extreme on this site.

The claim. A machine with redundant sensing can determine its own geometry with no external reference at all. It has more readings than freedoms, the surplus over-determines the parameters, and an instrument in the room would only be measuring what the machine already knows.

Why it is nearly right. The counting is correct and the equations are real. A platform with every joint read genuinely produces three independent equations per pose, over-determines fifteen parameters at twenty-five poses, and converges.

What it misses. Whether the answer has a scale in it depends on whether any reading has a scale in it, and joint encoders do not. Every equation is then a difference of lengths on both sides, so multiplying the whole machine by a constant multiplies every residual by the same constant — and with any noise at all, the fit is rewarded for shrinking. Run on a real platform, the unanchored fit returns a machine 0.24% of the true size, in two steps, at a residual of 1.5 × 10⁻¹⁰ against the right answer’s 1.05 × 10⁻⁴.

So the machine did calibrate itself, completely and confidently, to a geometry that is not there. The repair is one length measured once with a rule, which recovers exactly one parameter and is the difference between a commissioning procedure and a fiction.

The general form is a question to ask of any objective before its residual is trusted: multiply every parameter by a constant and see whether the residual moves. If it scales, the objective has a free ride to zero and the fit will take it, and no amount of data prevents it — this is the one item on the list that more poses actively make worse, since every extra equation is another one the collapsed machine satisfies exactly.

Why each one is nearly right #

Worth spending a paragraph each on what makes the six persuasive, because none of them is a careless mistake and all of them are true somewhere.

A small residual does mean a good answer — in a problem with no flat directions, which is most problems outside kinematics. Fit a straight line to noisy points and the residual is a complete diagnostic. It stops being one the moment the model has a redundancy, and a mechanism’s model very often does.

A full rank does mean one answer — locally, which is what rank means, and locally is enough when the objective is convex. It is not convex here.

An improvement does prove the parameters — when the model is complete, because then the only way to improve is to move towards the truth.

A classification is certain — as a statement about the parameters it was computed from, which are themselves uncertain.

A plan’s score is the measurement’s score — when the plan is executed as written, which it is except when the machine declines to visit a pose.

And enough data does settle it — everything the model can express.

Each is a statement with a hidden precondition, and in each case the precondition is satisfied often enough that the statement becomes a habit. The habit is what fails, on the cases where the precondition does not hold, and those cases are ordinary rather than exotic on this site’s machines.

What the six have in common #

Each takes a diagnostic that is genuinely good at one thing and reads it as good at another.

A residual tests the model against the data and is read as testing the answer against the machine. A rank tests local determination and is read as testing uniqueness. An improvement tests prediction and is read as testing parameters. A classification is exact and is read as certain. A plan’s score is a prediction and is read as a result. And a large data set beats noise and is read as beating everything.

Every one of them is a true statement about a narrow question generalised to a wide one, which is the same shape as the six things a network is not and as every other essay in this field.

A seventh that is not on the list #

One claim that sounds like it belongs and does not, because it is simply true and worth defending against the general scepticism the six produce.

A measurement of a mechanism’s angles recovers everything about its proportions. That is not a misconception; it is the sharpest positive result in this field. Grashof’s class, the transmission angle through the turn, the velocity ratio, where the dead centres fall, the coupler curve’s shape up to similarity — all of it comes back from the cheapest instrument there is, exactly.

An essay listing six failures leaves an impression that measurement establishes little, and the opposite is closer to the truth. What a well-planned identification establishes is large, exact and computable in advance; what the six are about is the gap between what it establishes and what its output looks like it establishes.

The apparatus is strong and its reporting is weak, and every one of the six is a reporting failure rather than a measurement one. That is why the closing recommendation is four lines to print rather than anything to do differently at the bench.

What to print instead #

Four items, all of which cost one decomposition between them, and each of which answers one of the six.

The residual, distributed over the poses, against the instrument’s independently measured repeatability. That settles one and three and gives the only available handle on six.

The singular values of the identification Jacobian at the answer, computed on the rows that were actually built. That settles one and five, and it says which parameters are determined and how well.

The margin of every count and class reported. That settles four, and the numbers already exist inside the routines that produce the counts.

And the result of a multi-start check. Twenty fits from scattered starts, and how many distinct answers came back. That settles two, imperfectly, and is the only instrument available for it.

None of that is new work. It is the output a fitting routine already has, plus one decomposition, plus twenty seconds of repeats — and its absence is why the six get said.

There is a shorter version for anybody who will print one thing. Print the singular values. A four-parameter problem has four of them and they fit on a line; they settle the rank, the conditioning, which parameters are weak and whether the plan achieved what it promised. They do not settle uniqueness and they do not settle the model, and those are the two limits the field’s own boundary essay names as the ones no measurement reaches.

Four numbers, one line, and four of the six disposed of.