Preprocessing transforms#

Baseline and detrending#

Baseline correction methods for spectroscopic data.

class spectoprep.preprocessing.baseline.ALSBaselineCorrection(lam=10000.0, p=0.001, niter=10)[source]#

Bases: BaseEstimator, TransformerMixin

Asymmetric Least Squares Baseline Correction.

This method estimates the baseline of spectra by fitting a smooth curve that preferentially lies beneath the data points.

Parameters:

lam (float, default=1e4) – Smoothness parameter. Higher values make the baseline smoother.
p (float, default=0.001) – Asymmetry parameter. Small values (<<1) force the baseline to lie below the data points.
niter (int, default=10) – Number of iterations for the ALS algorithm.

baseline_#

Estimated baseline for each spectrum.

Type:: ndarray of shape (n_samples, n_features)

is_fitted_#

Flag indicating if the transformer has been fitted.

Type:: bool

fit(X, y=None)[source]#

Fit the baseline for the input data X.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Transform the input data X by subtracting the baseline.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Baseline-corrected spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.baseline.DetrendTransformer(method='polynomial', order=2, dspline=100)[source]#

Bases: BaseEstimator, TransformerMixin

A transformer for detrending time series or spectral data using various methods.

Parameters:

method (str, default='polynomial') – The detrending method to use: - ‘simple’: Linear detrend between first and last points - ‘polynomial’: Polynomial detrend of specified order - ‘spline’: Spline detrend with specified order and spacing
order (int, default=2) – The order of the polynomial or spline fit. Ignored if method=’simple’.
dspline (int, default=100) – The spacing between spline knots. Only used if method=’spline’.

fit(X, y=None)[source]#

Fit the transformer (no-op).

Parameters:

X (array-like of shape (n_samples, n_features)) – Input data.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply detrending to the input data.

Parameters:: X (array-like of shape (n_samples, n_features)) – Input data to detrend.
Returns:: X_detrended – Detrended data.
Return type:: array-like of shape (n_samples, n_features)

Smoothing#

Smoothing methods for spectroscopic data.

class spectoprep.preprocessing.smoothing.SavitzkyGolay(filter_win=11, poly_order=2, deriv_order=0)[source]#

Bases: BaseEstimator, TransformerMixin

Savitzky-Golay filter for smoothing and differentiation of data.

Parameters:

filter_win (int, default=11) – Length of the filter window (must be positive odd integer).
poly_order (int, default=2) – Order of the polynomial used to fit the samples. Must be less than filter_win.
deriv_order (int, default=0) – Order of the derivative to compute. 0 means smoothing only.

Notes

The Savitzky-Golay filter is a digital smoothing polynomial filter that can preserve the high-frequency components of the signal better than standard averaging techniques.

fit(X, y=None)[source]#

Validate parameters.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply Savitzky-Golay filter to the data.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Filtered spectra.
Return type:: ndarray of shape (n_samples, n_features)

Scatter correction#

Scatter correction methods for spectroscopic data preprocessing.

class spectoprep.preprocessing.scatter.StandardNormalVariate[source]#

Bases: BaseEstimator, TransformerMixin

Standard Normal Variate (SNV) transformation.

SNV is a row-wise transformation that centers and scales each spectrum individually. It’s commonly used to remove scatter effects in spectroscopic data.

mean_#

Mean of each sample (row) computed during fit.

Type:: ndarray of shape (n_samples, 1)

std_#

Standard deviation of each sample computed during fit.

Type:: ndarray of shape (n_samples, 1)

is_fitted_#

Flag indicating if the transformer has been fitted.

Type:: bool

fit(X, y=None)[source]#

Compute mean and standard deviation of each sample.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply the StandardNormalVariate transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.scatter.MultiplicativeScatterCorrection[source]#

Bases: BaseEstimator, TransformerMixin

Multiplicative Scatter Correction (MSC) for spectroscopic data.

MSC performs a linear regression of each spectrum on a reference spectrum (usually the mean spectrum) and corrects using the estimated coefficients.

mean_reference#

Reference spectrum (mean of all spectra by default).

Type:: ndarray of shape (n_features,)

fit(X, y=None)[source]#

Compute the reference spectrum.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply the MSC transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.scatter.ExtendedMultiplicativeScatterCorrection(order=2)[source]#

Bases: BaseEstimator, TransformerMixin

Extended Multiplicative Scatter Correction (EMSC) for spectroscopic data.

EMSC extends the MSC method by incorporating polynomial terms to account for more complex spectral variations.

Parameters:: order (int, default=2) – Order of the polynomial used in the correction.

reference_spectrum#

Reference spectrum (mean of training spectra).

Type:: ndarray of shape (n_features,)

fit(X, y=None)[source]#

Compute the reference spectrum.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply the EMSC transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.scatter.LocalizedSNV(window_size=11)[source]#

Bases: BaseEstimator, TransformerMixin

Localized Standard Normal Variate (LSNV) using a sliding window.

LSNV applies the SNV transformation using a local window around each wavelength point rather than the entire spectrum.

Parameters:: window_size (int, default=11) – Size of the sliding window. Must be odd.

fit(X, y=None)[source]#

No-op.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply LSNV transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.scatter.RobustNormalVariate(lower_percentile=25, upper_percentile=75)[source]#

Bases: BaseEstimator, TransformerMixin

Robust Normal Variate (RNV) Preprocessing.

RNV is a robust version of SNV that uses percentiles instead of mean and standard deviation to reduce the influence of outliers.

Parameters:

lower_percentile (float, default=25) – The lower percentile for robust scaling.
upper_percentile (float, default=75) – The upper percentile for robust scaling.

fit(X, y=None)[source]#

No-op.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply RNV transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed spectra.
Return type:: ndarray of shape (n_samples, n_features)

Normalization and scaling#

Normalization methods for spectroscopic data.

class spectoprep.preprocessing.norml.Normalization(method='minmax', feature_range=(0, 1))[source]#

Bases: BaseEstimator, TransformerMixin

General normalization transformer that supports different methods.

Parameters:

method (str, default='minmax') – Normalization method: - ‘minmax’: Scales each spectrum to a range specified by feature_range - ‘zscore’: Standardizes each spectrum (mean=0, std=1)
feature_range (tuple (min, max), default=(0, 1)) – Range to scale the spectra to when method=’minmax’.

min_#

Minimum values of each sample (for minmax scaling).

Type:: ndarray of shape (n_samples, 1)

max_#

Maximum values of each sample (for minmax scaling).

Type:: ndarray of shape (n_samples, 1)

mean_#

Mean values of each sample (for zscore scaling).

Type:: ndarray of shape (n_samples, 1)

std_#

Standard deviation of each sample (for zscore scaling).

Type:: ndarray of shape (n_samples, 1)

is_fitted_#

Flag indicating if the transformer has been fitted.

Type:: bool

fit(X, y=None)[source]#

Fit the normalization parameters.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply the normalization transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Normalized spectra.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.norml.Autoscaling[source]#

Bases: BaseEstimator, TransformerMixin

Autoscaling (column-wise standardization).

Centers and scales data to unit variance along columns (features).

mean_#

Mean value for each feature.

Type:: ndarray of shape (n_features,)

std_#

Standard deviation for each feature.

Type:: ndarray of shape (n_features,)

fit(X, y=None)[source]#

Compute the mean and standard deviation of each feature.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Center and scale the data.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Transformed array.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.norml.MeanCentering[source]#

Bases: BaseEstimator, TransformerMixin

Mean Centering transformation.

Centers data by subtracting the column means, without scaling.

mean_#

Mean value for each feature.

Type:: ndarray of shape (n_features,)

fit(X, y=None)[source]#

Compute the mean of each feature.

Parameters:

X (array-like of shape (n_samples, n_features)) – The input spectra.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Center the data.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input spectra.
Returns:: X_transformed – Centered array.
Return type:: ndarray of shape (n_samples, n_features)

class spectoprep.preprocessing.norml.GlobalScaler(factor=1.0, mean=False, std=False)[source]#

Bases: BaseEstimator, TransformerMixin

Applies global scaling to spectra by a constant factor with optional mean centering and standardization.

Parameters:

factor (float, default=1.0) – Scaling factor to multiply spectra.
mean (bool, default=False) – Whether to subtract the mean of each feature.
std (bool, default=False) – Whether to divide by the standard deviation of each feature.

fit(X, y=None)[source]#

Compute mean and standard deviation if needed.

Parameters:

X (array-like of shape (n_samples, n_features)) – The training input samples.
y (None) – Ignored.

Returns:

self – Returns self.

Return type:

object

transform(X)[source]#

Apply global scaling transformation.

Parameters:: X (array-like of shape (n_samples, n_features)) – The input samples.
Returns:: X_scaled – Transformed samples.
Return type:: array-like of shape (n_samples, n_features)

class spectoprep.preprocessing.norml.RowStandardizer[source]#

Bases: BaseEstimator, TransformerMixin

Standardizes each row independently (i.e. across columns).

fit(X, y=None)[source]#

transform(X, y=None)[source]#

class spectoprep.preprocessing.norml.ColumnStandardizer[source]#

Bases: BaseEstimator, TransformerMixin

Standardizes columns using StandardScaler fitted on the training set.

fit(X, y=None)[source]#

transform(X, y=None)[source]#