Pipelines, interpretability & adjacent problems

A model is only one part of a machine-learning system.

The system must repeat the same preparation on every row, explain its behaviour honestly, and use evidence that matches the question being asked.

Why this matters

A notebook can look convincing while hiding three production failures:

1. training and serving prepare data differently;
2. an explanation is treated as proof that the model is right or causal;
3. an ordinary random split is used for a question about the future, ranking, or intervention.

Plain-English mental model

Part 1 — Split first, then fit one pipeline

PipelinePipelineAn ordered chain that fits data transformations and a model together, then applies that same chain to new rows. has a precise job: chain transformations and an estimator behind one fit and predict interface.

The order is:

split rows → fit the entire pipeline on training rows → transform and predict held-out rows

Not:

fit preprocessing on every row → cross-validate the model

The second version leaks validation information into imputation values, scaling statistics, selected features, and category vocabularies.

ColumnTransformer routes columns deliberately

A column transformerColumn transformerA pipeline step that sends selected columns through different transformations and joins the resulting features. sends different columns down different preparation branches, then joins their outputs.

For a support ticket:

• waiting hours and contact count are numeric;
• channel is categorical;
• a custom rule may apply log(1 + x) to non-negative counts;
• the estimator receives the joined numeric feature matrix.

Unknown categories need an explicit serving policy. OneHotEncoder(handle_unknown="ignore") keeps the feature width stable and encodes an unseen category as zeros for that encoder. It does not mean the new category is harmless: log it, monitor it, and decide whether retraining is needed.

A custom transformer follows fit and transform

fit(X, y) learns any allowed state from training data and returns the transformer.

transform(X) applies the learned rule to rows. A stateless rule can return from fit without learning values, but it still follows the interface so the pipeline can clone and validate it.

In cross-validation, pass the whole pipeline to the search or validation function. Do not fit prepare first.

In production, version:

• the fitted pipeline artifact;
• input schema and feature order;
• library and code versions;
• training-data reference;
• metric, threshold, and approval notes.

Part 2 — Ask an explanation a precise question

Interpretability means making a model’s behaviour understandable enough for a declared purpose. Different people need different evidence:

• a developer asks, “Which inputs drive this one surprising score?”;
• a reviewer asks, “Does the model depend on a prohibited shortcut?”;
• an operator asks, “When should I distrust or override it?”;
• an affected person may need a faithful, actionable reason under the applicable policy and law.

No single chart answers all four.

Four common tools

Permutation importance is often a useful model-agnostic start:

1. measure the fitted model on validation data;
2. shuffle one feature column;
3. measure the score again;
4. repeat and report the distribution of score changes.

If two features carry the same information, shuffling either one may barely hurt because the other remains. That is a property of the fitted model and data, not a contradiction.

Local and global views answer different questions

The lab uses a transparent formula so every local bar can be checked exactly. The lower chart is only partial-dependence-like teaching, not the library implementation.

Try the same waiting time with chat and phone. The average curve rises smoothly, while the channel curves behave differently. An average can conceal an interaction.

Explanation is a diagnostic, not a verdict

An explanation can reveal:

• reliance on a suspicious identifier;
• unstable behaviour near a threshold;
• a subgroup pattern hidden by an average;
• mismatch between domain expectations and model logic.

It cannot by itself prove:

• the prediction is correct;
• the feature caused the outcome;
• the system is fair or lawful;
• the input was measured accurately;
• the model will remain reliable after deployment.

Check fidelity, stability, meaningful baselines, data lineage, subgroup metrics, and human consequences alongside the chart.

Part 3 — Some problems need different evidence

Ordinary tabular prediction often assumes rows can be treated as exchangeable after grouping and time constraints are handled. The following problems make a different structure central.

Part 4 — Time series predicts after a cutoff

A time seriesTime seriesObservations ordered through time, where earlier values may help predict later values and future information must not leak backwards. is a sequence of observations ordered through time. Examples include hourly demand, daily sign-ups, and sensor readings.

Before modelling, declare:

• forecast origin: the cutoff when the prediction is made;
• horizon: how far ahead you predict;
• frequency: hourly, daily, weekly, or another interval;
• available information: what is actually known at the origin;
• retraining cadence: when the fitted pipeline may be updated.

Three common model families are:

• ARIMA: autoregressive integrated moving average. It combines past values, differencing that can stabilise a changing level, and past forecast errors.
• Prophet: an additive forecasting model with trend, seasonal, and holiday components.
• feature-based models: ordinary estimators using lagged values, rolling summaries, calendar fields, and known future inputs.

These are options, not a quality ranking. Start with a seasonal-naive baseline such as “same hour last week”, then compare at the real horizon.

Walk forward instead of shuffling

BacktestingBacktestingReplaying a forecasting or decision procedure at past cutoffs so each simulated prediction uses only information available at that time. replays predictions at past cutoffs. Every validation window must occur after its training window.

This expanding window imitates repeated retraining with all earlier data. A rolling window would keep only the most recent training period. Choose the one that matches production and possible drift.

Avoid:

• random shuffling across time;
• computing rolling features with future rows;
• choosing a model on one lucky cutoff;
• evaluating one-step forecasts when production needs twelve steps;
• ignoring missing intervals and delayed labels.

Part 5 — Recommenders retrieve and rank items

A recommender does not merely predict one label. It usually:

1. generates a manageable candidate set from a large catalogue;
2. scores the candidates for a user or context;
3. re-ranks for rules such as availability, freshness, safety, and diversity.

Collaborative filteringCollaborative filteringA recommendation approach that learns from patterns of user-item interaction so preferences shared across users and items inform suggestions. learns from user–item interaction patterns. Similar preference patterns can connect a user to an item even when no hand-written item feature says they match.

Matrix factorisationMatrix factorisationApproximating a user-item interaction table with smaller user and item vector tables whose dot products estimate affinity. approximates a sparse user–item table with smaller user and item vector tables. The dot product between a user vector and item vector becomes an affinity score.

The vector positions are latent dimensions: useful numerical directions learned to explain interaction patterns. Do not automatically give each position a human label.

A two-tower modelTwo-tower modelA recommender with one network that produces a query or user vector and another that produces an item vector so candidates can be compared efficiently. generalises this pattern:

• one network turns a user, query, and context into a vector;
• another turns item features into a vector;
• vector similarity retrieves candidates efficiently;
• a later ranker can use richer cross-features.

Evaluate on later interactions, not randomly mixed clicks. Report retrieval and ranking metrics at realistic list sizes, plus new-user, new-item, sparse-history, popularity, and exposure slices.

An unclicked item may never have been shown. Treating every unobserved interaction as dislike creates biased training evidence. Online experiments measure product consequences but still need guardrails for safety, diversity, latency, and long-term effects.

Part 6 — Causal questions ask what changes if we act

Prediction asks:

Who is likely to cancel?

Causal inferenceCausal inferenceReasoning about how an outcome would change under an intervention, using an experiment or explicit assumptions that make the comparison identifiable. asks:

Whose cancellation probability would change because we send a retention message?

Those are different targets. A person at high risk may cancel whether or not a message is sent. Another person at moderate risk may be persuadable.

A confounderConfounderA variable that influences both a treatment or exposure and an outcome, which can create a misleading association between them. influences both the treatment and outcome. For example, support severity may make an agent more likely to offer a discount and also make cancellation more likely. Comparing discounted and non-discounted customers directly then mixes treatment effect with severity.

Random assignment breaks that treatment-selection link on average and is the clearest design when ethical and practical. With observational data, causal estimates require explicit assumptions, appropriate adjustment, overlap between treatment groups, and sensitivity checks. A predictive pipeline or SHAP plot does not supply those conditions.

Uplift modellingUplift modellingEstimating how much an action changes an outcome for different people or groups, rather than only predicting the outcome itself. estimates a conditional treatment effect:

uplift(x) = expected outcome if treated at x
          - expected outcome if not treated at x

Only one outcome is observed for each person. This missing counterfactual is the central difficulty.

These are invented potential-outcome probabilities for arithmetic practice, not estimates from observational data. Segment B has the highest outcome probability if messaged, but segment C has the largest incremental change. Production uplift needs experimental or defensible causal evidence, uncertainty, costs, harms, and a no-action policy.

Worked system example — a support operations pipeline

Suppose a team wants to forecast ticket volume, prioritise current tickets, recommend help articles, and test a proactive message.

The production interface should log:

• pipeline version and schema result;
• raw feature lineage and prediction timestamp;
• model score, threshold, and action taken;
• explanation method, baseline, and version when one is shown;
• forecast origin and horizon;
• recommendation candidates shown, their order, and later interaction;
• treatment assignment and outcome window for experiments.

Never log sensitive raw data merely because an explanation library can display it.

Common mistakes

Exercises

Knowledge check

Retrieval checkpoint

Practice activity

Decision: route a new ticket to specialist review
Unit: one ticket at its arrival timestamp

Pipeline:
  numeric -> train-fold imputation + scaling
  category -> train-fold imputation + unknown-safe encoding
  text-derived counts -> shared, versioned custom transformer
  estimator -> calibrated classifier

Local question:
  why did this model score this ticket highly?
  method -> SHAP with documented background
  limit -> model attribution, not cause

Global question:
  does the model rely on channel after controlling the review protocol?
  methods -> permutation importance + subgroup metrics + partial response

Evidence:
  group by customer, split through time, fit the whole pipeline per fold
  protect a later final test period

Serving:
  validate schema -> pipeline.predict_proba -> threshold -> capacity fallback
  log pipeline version, timestamp, score, action, and monitored slices

Key takeaways

Flashcards

One-page sketchnote summary

split → pipeline.fit(train) → predict(held_out) · explain the model ≠ explain the world · time/ranking/treatment need their own evidence

Resources