Smoothing

Smoothing (Not available in Mnova Lite):

The signal to noise ratio (S/N) of a spectrum can be enhanced by smoothing (or filtering) techniques. This technique is used to reduce the effect of noise on a spectrum and possibly reveal signals embedded in, or distorted by, noise. Noise contains rapid fluctuations which are are generally faster than the rates of change of genuine signals. Smoothing applies a low-pass filter to the spectral data to remove these rapid fluctuations while having minimal effect on signals.

Mnova incorporates some of the more used signal smoothing algorithms.

This command is available by following the menu 'Processing/More Processing/Smoothing'. Then, a dialog box will open:

The user can select the smoothing algorithm (Moving Average Filter, Whittaker Smoother, Savitzky-Golay or Wavelets) in the Method scroll bar. The user can also apply the smoothing to All Dimensions (by ticking the applicable box).

'Moving average algorithm' : The simpler software technique for smoothing signals consisting of equidistant points is the moving average. An array of raw (noisy) data (y1, y2, …, yN) can be converted to a new array of smoothed data. The "smoothed point" (yk)s is the average of an odd number of consecutive 2n+1 (n = 1, 2, 3, ..) points of the raw data yk-n, yk-n+1, …, yk-1, yk, yk+1, …, yk+n-1, yk+n, i.e.

The odd number 2n+1 is usually named filter width (or span in Mnova). The greater the filter width the more intense the smoothing effect.

The moving average algorithm is particularly damaging when the filter passes through peaks which are narrow compared to the filter width.

The user can select the desired filter width (Span) in the dialog box.

Whittaker Smoother: A smoothing algorithm is proposed by Whittaker in1923. If y is a series of m data points and z is the smooth series which should approximate y, we minimize:

The first term measures the fit of z to y, the second term is a so-called penalty: it discourages changes in z. The influence of the penalty is tuned by the parameter λ; the larger a λ is chosen, the smoother z will be, at the cost of a worse fit to the data.

The user can select the Smooth Factor in the dialog box or let Mnova detect the best one automatically.

'Savitzky-Golay algorithm' : A much better procedure than simply averaging points is to perform a least squares fit of a small set of consecutive data points to a polynomial and take the calculated central point of the fitted polynomial curve as the new smoothed data point.

Savitzky and Golay  showed that a set of integers (A-n, A-(n-1) …, An-1, An) could be derived and used as weighting coefficients to carry out the smoothing operation. The use of these weighting coefficients, known as convolution integers, turns out to be exactly equivalent to fitting the data to a polynomial, as just described, and it is computationally more effective and much faster. Therefore, the smoothed data point (yk)s by the Savitzky-Golay algorithm is given by the following equation:

Sets of convolution integers, instead of the smoothed signal, can be used to obtain directly, instead, its 1st, 2nd, …, mth order derivative, therefore the Savitzky-Golay algorithm is very useful for calculation of the derivatives of noisy signals consisting of discrete and equidistant points.

The smoothing effect of the Savitzky-Golay algorithm is not as aggressive as in the case of the moving average and the loss and/or distortion to vital information is comparatively limited. However, it should be stressed that both algorithms are "lossy", i.e. part of the original information is lost or distorted. This type of smoothing has only cosmetic value.

The user can select the 'order' and the 'Width'.

Wavelets:  Wavelet thresholding is the basis of wavelet based noise reduction. For a function f with Gaussian noise

this means that the function f is restored.

Hard thresholding is a simple “keep or kill” selection. All wavelet coefficients below a threshold λ are zeroed.

Soft thresholding shrinks the coefficients towards zero:

The most important step is now a proper choice of the threshold λ.  

A universal threshold consists of:

where n is the sample size and σ the scale of the noise on a standard deviation scale.

The overall procedure of noise suppression consists of a wavelet transform (WT), which yields the wavelet coefficients cj;k, followed by thresholding of these coefficients and by an inverse wavelet transform (IWT), which restores the original spectrum.

The user can select the scale (σ) and the fraction threshold (λ%) or also the soft or the Universal Threshold (by ticking the applicable box).