MULTIPLE REGRESSION CAN BE OBTAINED BY SEQUENTIAL MATCHING

Returning to the setting of the question, we have one target yy and two matchers x1x1 and x2x2. We seek numbers b1b1 and b2b2 for which yy is approximated as closely as possible by b1x1+b2x2b1x1+b2x2, again in the least-distance sense. Arbitrarily beginning with x1x1, Mosteller & Tukey match the remaining variables x2x2 and yy to x1x1. Write the residuals for these matches as x2⋅1x2⋅1 and y⋅1y⋅1, respectively: the ⋅1⋅1indicates that x1x1 has been "taken out of" the variable.

We can write

y=λ1x1+y⋅1 and x2=λ2x1+x2⋅1.y=λ1x1+y⋅1 and x2=λ2x1+x2⋅1.

Having taken x1x1 out of x2x2 and yy, we proceed to match the target residuals y⋅1y⋅1 to the matcher residuals x2⋅1x2⋅1. The final residuals are y⋅12y⋅12. Algebraically, we have written

y⋅1y=λ3x2⋅1+y⋅12; whence=λ1x1+y⋅1=λ1x1+λ3x2⋅1+y⋅12=λ1x1+λ3(x2−λ2x1)+y⋅12=(λ1−λ3λ2)x1+λ3x2+y⋅12.y⋅1=λ3x2⋅1+y⋅12; whencey=λ1x1+y⋅1=λ1x1+λ3x2⋅1+y⋅12=λ1x1+λ3(x2−λ2x1)+y⋅12=(λ1−λ3λ2)x1+λ3x2+y⋅12.

This shows that the λ3λ3 in the last step is the coefficient of x2x2 in a matching of x1x1 and x2x2 to yy.

We could just as well have proceeded by first taking x2x2 out of x1x1 and yy, producing x1⋅2x1⋅2 and y⋅2y⋅2, and then taking x1⋅2x1⋅2 out of y⋅2y⋅2, yielding a different set of residuals y⋅21y⋅21. This time, the coefficient of x1x1 found in the last step--let's call it μ3μ3--is the coefficient of x1x1 in a matching of x1x1 and x2x2 to yy.

Finally, for comparison, we might run a multiple (ordinary least squares regression) of yy against x1x1and x2x2. Let those residuals be y⋅lmy⋅lm. It turns out that the coefficients in this multiple regression are precisely the coefficients μ3μ3 and λ3λ3 found previously and that all three sets of residuals, y⋅12y⋅12, y⋅21y⋅21, and y⋅lmy⋅lm, are identical.