Model-validation through diagnostic residuals | Combine

Model-validation through diagnostic residuals

Models are widely used within the field of engineering, which is highly understandable as models enable prediction of a systems behavior based on the initial state of the system. Within the field of diagnostics however, the models are used in the reversed manner, estimating the state based on the observed system behavior.

The simplest diagnostic example would basically consist of two sensors y and z, which are measuring the same unknown quantity x. When considering that the sensor-values could include errors, f1 and f2, the resulting system become:

y = x + f1                           (eq. 1)
z = x + f2                           (eq. 2)

As x is the only unknown variable, this system of equations is overdetermined.
This enables the construction of a residual, that is a connection between known quantities that are equal to zero in a fault free scenario. Residuals are usually denoted with r, which in this case results in the following residual:

r = z – y = f2 – f1

The residual r has the possibility of detecting the physical faults f1 and f2, but there is no way to determine which of the faults that has caused r to deviate from zero. The ability to pinpoint which fault has caused the deviation is known as the isolability of the system. By adding a third equation to the system, full isolability is achieved.

u = x + f3                           (eq. 3)

r = z- y = f2 –f1
r1 = y – u = f1 – f3
r2 = z – u = f2 – f3

It is however possible to create residuals through which all 3 faults are detectable by combining all three equations, e.g.

r3 = z – 0.5y -0.5u.

This residual does not contribute any additional information compared to the information already given by r, r1 and r2, which follows from which equations that were used to create each residual.

{E1, E2} resulting in r                   (set 1)
{E1, E3] resulting in r1                 (set 2)
{E2, E3} resulting in r2                 (set 3)
{E1, E2, E3} resulting in r3           (set 4)

What differs the top 3 sets from the bottom one is that the top three are what is called Minimal Structurally Overdetermined sets of equations, also known as MSOs. The minimal part corresponds to an MSO not being a subset to any other overdetermined set of equations. Set 2 is a subset of set 4 for instance, but not vice versa. The structural nature part of MSOs enables analysis of very complex systems as it only takes the existence of unknown variables and faults into account and not in what way these are included into the equation. For example, equation 1 would structurally be summarized as x and f1 exist. For a system of equations, this can be plotted by using a matrix, where each row corresponds to an equation, and each column represents existence or non-existence of faults or unknown variables. This is called the Dulmage-Mendelsohn decomposition.

For additional information regarding computing MSOs, see Fault Diagnosis Toolbox on github. One interesting application of residuals is model validation. This application is possible due to the fact that if a model is correct, the residual value is likely to be low and vice versa.  If a model has a low accuracy, it is often of interest to pinpoint the low accuracy to a particular subpart of the model, if it is possible. This can be achieved by letting the faults {f1, f2, f3} represent model equation errors {fe1, fe2, fe3} and then generate residuals based on MSOs.

By using as few equations as possible in each residual, maximum isolability regarding model inaccuracy can be achieved. One method used to convert residual-values to one metric (in order to compare the validity of different model equations) is to compute the mean-values for all residuals sensitive to a specific fault fex, and then multiply these means together to a single value R_fex. The absolute value of R_fex doesn’t provide much information, but by comparing R_fex to values generated through residuals that are sensitive to other faults (fey, fez,..) an indication of model accuracy is achieved.
R_fe1 > R_fe2 -> equation 1 is likely of lower accuracy then equation 2.

For further information and examples on bigger models see (Karin Lockowandt, 2017, p.30).

Contact us