Preprocessing

Real-world datasets rarely come in the shape a machine learning algorithm expects. The preprocessing module provides utilities for transforming raw data into a form your estimator can consume:

• Categorical encoding — convert categorical columns into a numeric representation via one-hot encoding.
• Numerical scaling — standardize numerical features so they share a common scale.

Preprocessing transformers implement the Transformer trait: call fit to learn the transformation parameters from your data, then transform to apply it. Like the estimators, optional parameters are hidden behind Default::default().

One-Hot Encoding

The OneHotEncoder converts one or more categorical columns into indicator (0/1) columns. You tell it which column indices are categorical when you build the parameters via OneHotEncoderParams::from_cat_idx.

use smartcore::linalg::basic::matrix::DenseMatrix;
use smartcore::preprocessing::categorical::{OneHotEncoder, OneHotEncoderParams};

// A small dataset where column 0 is categorical ("species" code as f32).
let rows: Vec<Vec<f32>> = vec![
    vec![0.0, 1.1, 2.2],
    vec![1.0, 3.3, 4.4],
    vec![0.0, 5.5, 6.6],
];
let x = DenseMatrix::from_2d_vec(&rows).unwrap();
// Column 0 is the categorical feature
let params = OneHotEncoderParams::from_cat_idx(&[0]);
// Fit the encoder then transform the data
let encoder = OneHotEncoder::fit(&x, params).unwrap();
let x_encoded = encoder.transform(&x).unwrap();

The fitted OneHotEncoder is Serialize/Deserialize (behind the serde feature) so it can be persisted alongside a trained model — see the model persistence section for the serde pattern.

Standard Scaling

The StandardScaler standardizes numerical features by removing the mean and scaling to unit variance:

\[z = \frac{x - \mu}{\sigma}\]

It implements UnsupervisedEstimator and Transformer, so the fit/transform interface is the same as the rest of the library.

use smartcore::linalg::basic::matrix::DenseMatrix;
use smartcore::preprocessing::numerical::StandardScaler;

let rows: Vec<Vec<f32>> = vec![
    vec![0.0, 10.0],
    vec![2.0, 20.0],
    vec![4.0, 30.0],
    vec![6.0, 40.0],
];
let x = DenseMatrix::from_2d_vec(&rows).unwrap();
// Fit the scaler on the data, then transform
let scaler = StandardScaler::fit(&x, Default::default()).unwrap();
let x_scaled = scaler.transform(&x).unwrap();
// Each column now has mean ~0 and variance ~1

Pass an instance of StandardScalerParameters instead of Default::default() to control whether the scaler centers and/or scales the data.

Series Encoding

For ordinal or one-hot encoding of a single categorical series (rather than a full matrix), the CategoryMapper helper maps categories to integer codes and back, and can emit one-hot vectors. See the module docs for the full API.