Unsupervised learning

Unsupervised learning looks for patterns when no column contains the right answer.

That can reveal useful groups and simpler views. It cannot tell you what a group means, whether it is fair, or what action to take.

Why this matters

Real datasets often arrive without an answer column. A support team may not have ticket topics. A product team may want to explore behaviour patterns. An engineer may need a compact representation before a downstream model.

Unsupervised learningUnsupervised learningFinding structure in examples that do not include a desired target answer, such as groups, compressed representations, or unusual cases. The discovered structure still needs human interpretation and evaluation. helps explore that structure. The responsible next step is investigation and evaluation, not automatically naming a discovered group “a customer type”.

Plain-English mental model

Step 1 — Know what problem you have

Use unsupervised methods when no reliable target answer is available during fitting.

• Clustering: propose groups of similar rows.
• Dimensionality reduction: represent many columns with fewer coordinates.
• Anomaly detection: find unusual rows under a fitted normality rule.
• Supervised prediction: learn a known target; this is different.

Clusters are not labels. If you manually label clusters and later predict that label, you have created a supervised task with all the usual split, leakage, and fairness requirements.

Step 2 — Prepare the representation before searching for groups

Distance-based methods treat numeric units literally.

If one feature is yearly spend in pounds and another is number of visits, raw spend can dominate Euclidean distance. Fit scaling on the data allowed at fit time, then apply the frozen transform consistently.

Decide deliberately how to encode:

• numeric measurements;
• categories;
• missing values;
• dates and seasonality;
• text, images, or embeddings;
• sensitive features and proxies.

Do not silently include an outcome, future field, account identifier, or timestamp that leaks the answer you hope to investigate.

Step 3 — k-means: assign, average, repeat

k-meansk-meansA clustering algorithm that repeatedly assigns each example to its nearest centre and moves each centre to the mean of its assigned examples. You choose the number of clusters, k, before fitting. needs a chosen number of clusters, k.

1. Start with k centroidsCentroidThe arithmetic mean position of the examples assigned to a cluster. In k-means it is the cluster's movable centre, and it may not be an actual example., which are centre positions.
2. Assign each row to its closest centroid.
3. Move each centroid to the arithmetic mean of its assigned rows.
4. Repeat until assignments or centres stop changing enough.

The objective is inertiaInertiaThe sum of squared distances from every example to its assigned k-means centroid. Lower inertia always improves as k grows, so it cannot choose k by itself.: the sum of squared distances to assigned centres. Squaring makes far points count more.

inertia = Σ ||row - assigned_centroid||²

Worked example — two small piles

Rows are positions on one line: 0, 1, 8, 9. Start centres at 0 and 9.

Update centres:

left centre  = mean(0, 1) = 0.5
right centre = mean(8, 9) = 8.5

One more assignment gives the same groups. That is convergence for this start, not proof of globally best or meaningful clusters.

Expected output:

assignments: {0: [0.0, 1.0], 1: [8.0, 9.0]}
updated centres: [0.5, 8.5]

k-means assumptions and limitations

k-means is a useful baseline when compact, roughly round groups with similar scale are plausible after sensible preprocessing.

It struggles with:

• long, curved, or nested shapes;
• very different densities and sizes;
• outliers, because means move toward them;
• categorical variables treated as arbitrary numbers;
• high-dimensional noisy data;
• a k chosen only because a plot looked tidy.

Use multiple starts. Its objective has local optima: different initial centroids can lead to different final partitions.

Step 4 — Select k using evidence, not certainty

Inertia always falls as k increases. At k = number of rows, every row can get its own centroid and inertia is zero. So inertia alone cannot select a useful count.

The elbow heuristic looks for diminishing improvement. It is a prompt to investigate, not a mathematical answer.

A silhouette scoreSilhouette scoreA cluster-quality heuristic comparing how close an example is to its own cluster with how close it is to the nearest other cluster. It is useful evidence, not proof that clusters are meaningful. compares within-cluster closeness to nearest-other-cluster closeness. Higher can suggest better separated geometry, but it also inherits your metric and representation.

Better evidence combines:

1. stability across seeds, samples, and nearby parameters;
2. inspection of representative rows and feature summaries;
3. domain review of whether the distinction can support a real decision;
4. a held-out downstream task, when clustering is used as a feature or workflow;
5. monitoring for drift after deployment.

Step 5 — Hierarchical clustering keeps the alternatives

Hierarchical clusteringHierarchical clusteringClustering that creates a tree of nested groups by repeatedly merging nearby groups or splitting a large group. A chosen cut through the tree determines the final groups. creates a dendrogram, a tree of nested merges.

Agglomerative clustering starts with one row per cluster, then repeatedly merges the closest clusters. A horizontal cut at one height gives a small number of broad groups; a lower cut gives more groups.

The outcome depends on a linkage rule:

• single linkage: nearest pair across two groups; can chain;
• complete linkage: farthest pair; favours compact groups;
• average linkage: average cross-group distance;
• Ward linkage: merges that increase within-group variance least, usually with Euclidean distance.

It is valuable for exploration because it does not require choosing a final count before fitting. It can be memory- and time-intensive for large row counts.

Step 6 — DBSCAN finds density and permits noise

DBSCANDBSCANDensity-Based Spatial Clustering of Applications with Noise, a clustering method that connects dense regions and can mark isolated examples as noise rather than forcing them into a cluster. means Density-Based Spatial Clustering of Applications with Noise.

It needs two important settings:

• eps: radius considered close;
• min_samples: enough nearby points to form a dense core.

Core points connect into dense clusters. Border points can join a core. Isolated points become noise label -1 rather than being forced into a group.

This can discover irregular shapes and handle outliers better than k-means. It is highly sensitive to feature scale and eps, and one global density setting struggles when real groups have very different densities.

Step 7 — PCA makes a smaller linear coordinate system

PCAPrincipal component analysisA linear dimensionality-reduction method that projects data onto new orthogonal directions ordered by how much variation in the fitted data they explain. finds new orthogonal axes:

• first principal component: direction with most fitted-data variation;
• second: next most variation, perpendicular to the first;
• later components: remaining independent directions.

Projecting from many columns to two components makes a plot. Keeping enough components can compress data before a downstream model.

Explained varianceExplained varianceThe proportion of variation in fitted data captured by one or more principal components. It measures reconstruction-oriented variation, not business usefulness or causality. is how much fitted-data variation retained components capture. It does not mean feature importance, prediction quality, causal importance, or business value.

PCA workflow

1. Split first when a later supervised evaluation exists.
2. Fit imputation and scaling on training rows only.
3. Fit PCA on those transformed training rows.
4. Transform validation, test, and future rows with the frozen fit.
5. Inspect reconstruction, explained variance, stability, and downstream effect.

PCA can be distorted by unscaled units and outliers. Component signs can flip with no change in meaning. A component mixes original features, so explain it by inspecting loadings and examples, cautiously.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

pipeline = Pipeline([
    ("scale", StandardScaler()),
    ("pca", PCA(n_components=0.90, random_state=0)),
    ("cluster", KMeans(n_clusters=4, n_init=20, random_state=0)),
])

pipeline.fit(training_rows)
cluster_ids = pipeline.predict(new_rows)

print(pipeline.named_steps["pca"].n_components_)
print(cluster_ids.tolist())

Expected output has a data-dependent number of retained components and one cluster ID per new row:

3
[1, 0, 1, 3]

The code is copyable but needs installed scikit-learn and a real numeric table. n_components=0.90 asks PCA to retain at least 90% of the training variance; it does not promise 90% task performance.

Step 8 — Use t-SNE and UMAP as maps, not rulers

t-SNEt-SNEt-distributed Stochastic Neighbour Embedding, a nonlinear method for visualising local neighbourhoods in two or three dimensions. Global distances, cluster size, and blank space in its map are not reliable measurements. and UMAPUMAPUniform Manifold Approximation and Projection, a nonlinear dimensionality-reduction method often used for visual exploration and embeddings. Its output depends on choices such as neighbours, minimum distance, randomness, and preprocessing. make nonlinear low-dimensional views. They are often useful for exploring embedding neighbourhoods, checking possible label problems, or deciding what to inspect next.

They can mislead when treated as an objective map:

• local neighbours are more trustworthy than long-range distance;
• island size and empty space are not counts or density measurements;
• different seeds and parameters can change appearance;
• preprocessing and distance metric matter;
• plots do not validate a downstream classifier.

For t-SNE, perplexity controls a neighbourhood scale. For UMAP, n_neighbors and min_dist influence the local-to-broader structure trade-off and point compactness. Run more than one setting; compare to original-space neighbours and known metadata.

Worked system example — exploring support tickets responsibly

A support team has 200,000 ticket embeddings and no reliable topic label.

1. Remove identifiers and protect raw text access.
2. Sample a time slice and compute documented embeddings.
3. Inspect nearest neighbours for a small sample first.
4. Compare PCA for compression and UMAP only as an exploration view.
5. Cluster candidate representations with several seeds and settings.
6. Summarise clusters using representative, access-controlled tickets.
7. Ask support specialists whether a group suggests a useful workflow.
8. Validate an actual action: for example, whether a proposed routing rule reduces hand-offs on a later period.

The cluster is an investigation tool. The deployment decision needs an explicit outcome and evaluation.

Common mistakes

Exercises

Knowledge check

Practice activity

from math import dist

points = [(0.0, 0.0), (1.0, 0.0), (8.0, 0.0), (9.0, 0.0)]
centres = [(0.0, 0.0), (9.0, 0.0)]

groups = {0: [], 1: []}
for point in points:
    nearest = min(range(2), key=lambda i: dist(point, centres[i]))
    groups[nearest].append(point)

updated = []
for index in range(2):
    xs = [point[0] for point in groups[index]]
    ys = [point[1] for point in groups[index]]
    updated.append((sum(xs) / len(xs), sum(ys) / len(ys)))

print(groups)
print(updated)

{0: [(0.0, 0.0), (1.0, 0.0)], 1: [(8.0, 0.0), (9.0, 0.0)]}
[(0.5, 0.0), (8.5, 0.0)]

Key takeaways

Flashcards

One-page sketchnote summary

assign = nearest_centroid(rows, centres)
centres = mean_each_group(assign)
components = fit_pca(training_rows)
map = transform(future_rows, components)

Resources