Whole-Slide Images, Tiling, and Stain Normalization — Adventures in Medical CV

So You Want to Do Medical Computer Vision

Medical imaging is nothing like standard CV datasets. You’re not resizing 224x224 JPEGs of cats and dogs. You’re dealing with whole-slide images (WSIs) that are 50,000 x 50,000 pixels and weigh multiple gigabytes. A pathology slide scanned at 40x magnification is about 100,000 x 80,000 pixels. That’s 8 billion pixels in one image.

You can’t throw that into a ResNet. You can’t even load it into RAM.

The Whole-Slide Image Problem

A WSI is a pyramid of resolutions. The scanner captures the slide at max magnification, then down-samples into multiple levels so you can zoom in and out without loading the entire gigapixel image.

Level 0: 100,000 x 80,000 px  (40x)
Level 1: 50,000 x 40,000 px   (20x)
Level 2: 25,000 x 20,000 px   (10x)
Level 3: 12,500 x 10,000 px   (5x)
Level 4: 6,250 x 5,000 px     (2.5x)

The file format is usually SVS, TIFF, or DICOM. These aren’t handled by standard image libraries.

Reading WSIs Without Crashing Your Machine

OpenSlide reads multi-resolution medical image formats without loading the whole thing into memory:

import openslide                                          # Library purpose-built for multi-resolution medical image formats (SVS, TIFF, DICOM)

slide = openslide.OpenSlide("patient_123.svs")            # Opens the file handle without loading pixel data — the pyramid index is read lazily

print(f"Levels: {slide.level_count}")                    # Number of resolution levels in the pyramid (typically 5–6 for a 40x scan)
print(f"Level 0 dimensions: {slide.level_dimensions[0]}")  # Full-res dimensions at 40x — e.g. (100000, 80000); just a tuple, no memory allocated

thumbnail = slide.get_thumbnail((1024, 1024))             # Reads a downsampled preview by selecting the appropriate pyramid level internally — useful for quick UI previews

region = slide.read_region((20000, 30000), 0, (1024, 1024))  # Reads a 1024x1024 pixel region from level 0 at (x=20000, y=30000) — only this patch is decoded into RAM

Read specific regions. Never the whole file.

The Tiling Pipeline

Since you can’t feed a 100,000px image into a neural network, you chop it into tiles (patches). Usually 256x256 or 512x512 at 20x magnification.

But you don’t tile the whole slide — most of it is empty background. First do tissue detection:

import openslide                                          # WSI reader that navigates the multi-resolution pyramid without loading the full slide
import numpy as np                                        # Used for pixel-level thresholding and coordinate arithmetic on the tissue mask
from PIL import Image                                     # Converts OpenSlide's RGBA region into a standard RGB PIL image for downstream use

def extract_tissue_tiles(svs_path, tile_size=256, threshold=0.1):
                                                          # tile_size=256: standard patch size for WSI models (balances context vs batch size)
                                                          # threshold=0.1: at least 10% of the patch must contain tissue to keep it
    slide = openslide.OpenSlide(svs_path)                 # Opens the slide handle — only metadata is loaded, not pixel data

    level = slide.level_count - 1                         # Uses the lowest-resolution pyramid level for the tissue mask (fastest to read)
    thumbnail = slide.read_region((0, 0), level, slide.level_dimensions[level])
                                                          # Reads the full downsampled thumbnail — e.g. 6250x5000 instead of 100000x80000
    thumbnail = thumbnail.convert("RGB")                  # OpenSlide returns RGBA; strip alpha for NumPy processing
    thumbnail_np = np.array(thumbnail)                    # Convert PIL image to a (H, W, 3) array for thresholding

    gray = np.mean(thumbnail_np, axis=2)                  # Average across RGB channels to get a single-channel intensity map
    tissue_mask = gray < 220                              # Binary mask: pixels darker than 220 (on 0–255 scale) are considered tissue, not background glass

    scale = slide.level_dimensions[0][0] / slide.level_dimensions[level][0]
                                                          # Downscale factor from level 0 to the thumbnail level (e.g. 16x for 5 levels down)

    tiles = []                                            # Accumulator for valid (tissue-containing) PIL Image patches
    step = int(tile_size * scale)                         # Step size at the thumbnail level corresponding to one tile at level 0

    for y in range(0, tissue_mask.shape[0] * int(scale) - tile_size, tile_size):
                                                          # Iterate y in level-0 pixel coordinates; shape[0] * scale recovers the full-res height
        for x in range(0, tissue_mask.shape[1] * int(scale) - tile_size, tile_size):
                                                          # Iterate x in level-0 coordinates; ensures tiles cover the entire slide area
            tx, ty = int(x / scale), int(y / scale)       # Map level-0 coordinates back to thumbnail coordinates for the mask lookup
            if tx >= tissue_mask.shape[1] or ty >= tissue_mask.shape[0]:
                continue                                  # Skip tiles whose mapped position falls outside the thumbnail bounds

            patch = tissue_mask[ty:ty + 1, tx:tx + 1]     # Sample a single-pixel row from the mask at the tile's location
            if np.mean(patch) > threshold:                # If the fraction of tissue pixels in this mask sample exceeds 10%, read the full tile
                tile = slide.read_region((x, y), 0, (tile_size, tile_size))
                                                          # Reads only the needed 256x256 region from level 0 — no surrounding pixels decoded
                tiles.append(tile)                        # Append the PIL Image tile to the results list

    return tiles                                          # Returns only tissue-bearing patches; empty-background tiles are discarded

This is simplified. In practice you’d use OTSU thresholding or contour-based filtering.

How Many Tiles Per Slide?

A typical biopsy slide at 20x with 50% tissue coverage, 256x256 tiles:

  • ~15,000 to 40,000 tiles per slide

A study with 1,000 slides = 15 to 40 million tiles. That’s not “load into a DataFrame and train.” That’s “build a data pipeline.”

Stain Normalization — The Silent Model Killer

Slides from different hospitals look completely different. H&E stain at one lab might look purple. At another lab, it’s pink. At a third, it’s faded.

If you train on Hospital A data and deploy on Hospital B slides, your accuracy drops off a cliff. Not a minor issue — your model goes from 96% to 52%.

The standard fix is Macenko’s method. It separates the H&E stains into individual channels, then matches the stain appearance to a target image:

import numpy as np                                        # All stain normalization is linear algebra on optical density vectors

def stain_normalize(source, target, alpha=1, beta=0.15):
                                                          # source: tile from a hospital with unknown stain appearance
                                                          # target: reference tile whose stain appearance we want to match
                                                          # alpha, beta: percentile trimming params (unused in this simplified version)
    source_od = -np.log(np.maximum(source / 255.0, 1e-6))
                                                          # Convert source RGB [0-255] to optical density: OD = -log(I / I_0)
                                                          # OD space makes stain separation linear because Beer-Lambert law is additive
    target_od = -np.log(np.maximum(target / 255.0, 1e-6))
                                                          # Same conversion for the target reference slide

    source_reshape = source_od.reshape((-1, 3))           # Flatten spatial dims (H*W, 3) so each row is a pixel's OD vector
    target_reshape = target_od.reshape((-1, 3))           # Same reshape for target to compute stain vectors per channel

    source_mask = np.mean(source_reshape, axis=1) > 0.05  # Keep only pixels whose mean OD > 0.05 — filters out glass background (low OD)
    target_mask = np.mean(target_reshape, axis=1) > 0.05  # Same foreground mask for the target; stain vectors should come from tissue only

    source_foreground = source_reshape[source_mask]       # Foreground-only pixel OD matrix for SVD — glass pixels would distort the stain basis
    target_foreground = target_reshape[target_mask]       # Target foreground; both matrices are (N_fg, 3) where N_fg is foreground pixel count

    _, _, source_V = np.linalg.svd(source_foreground, full_matrices=False)
                                                          # SVD on source foreground: V (3x3) rows are the principal stain directions
                                                          # First two rows correspond to Hematoxylin (blue/purple) and Eosin (pink/red)
    _, _, target_V = np.linalg.svd(target_foreground, full_matrices=False)
                                                          # SVD on target foreground: target's own stain basis to map into

    source_stains = source_od @ source_V.T                # Project every pixel's OD vector into the source stain basis (H&E space)
    target_stains = target_od @ target_V.T                # Project every pixel's OD vector into the target stain basis

    for i in range(3):                                    # Match the mean and std of each stain channel (H, E, residual) to the target
        source_std = np.std(source_stains[:, i])           # Standard deviation of the i-th stain component in the source
        target_std = np.std(target_stains[:, i])           # Target standard deviation — the spread we want to match
        source_mean = np.mean(source_stains[:, i])         # Mean of the i-th stain component in the source
        target_mean = np.mean(target_stains[:, i])         # Target mean — the intensity centroid we shift toward
        source_stains[:, i] = (source_stains[:, i] - source_mean) * \
            (target_std / source_std) + target_mean
                                                          # Z-score the source stain, then scale by target std and shift by target mean
                                                          # This aligns the statistical distribution of H&E intensities between labs

    normalized_od = source_stains @ target_V              # Project the adjusted stain coefficients back into RGB OD space via target basis
    normalized = np.exp(-normalized_od).reshape(source.shape)
                                                          # Invert Beer-Lambert: OD -> transmission -> RGB; reshape back to original (H, W, 3)

    return np.clip(normalized * 255, 0, 255).astype(np.uint8)
                                                          # Scale OD back to [0-255], clip to valid range, and return uint8 for model ingestion

Normalize every tile before feeding it to the model. The model sees consistent stain appearance regardless of lab.

Deep Learning-Based Normalization

StainGAN and CycleGAN-based approaches exist. They work better but cost more compute. Macenko or Reinhard are good enough for most projects.

The Data Pipeline for WSI

class WSIDataset(Dataset):                                # Extends PyTorch Dataset — yields tiles on demand without holding pixel data in memory
    def __init__(self, slide_paths, tile_size=256, transform=None):
        self.slide_paths = slide_paths                    # List of SVS/TIFF file paths; each can be 1–2 GB on disk
        self.tile_size = tile_size                        # Standard 256x256 or 512x512 at 20x magnification
        self.transform = transform                        # Augmentation pipeline (flip, rotate, color jitter, stain normalize)
        self.tiles = []                                   # Flat list of (slide_path, x, y) tuples — pixel data is never stored here
        self._precompute_tiles()                          # Runs tissue detection on all slides upfront; slow but avoids repeated mask computation

    def _precompute_tiles(self):                          # Scans every slide once to build the coordinate index
        for path in self.slide_paths:                     # Iterate over all SVS files in the cohort (hundreds to thousands)
            slide = openslide.OpenSlide(path)             # Open handle — reads pyramid metadata but no pixel data yet
            for x, y in self._tissue_coords(slide):       # Generator yielding (x, y) top-left corners of tissue-bearing regions
                self.tiles.append((path, x, y))           # Store only the reference and coordinate — ~24 bytes per tile vs ~1 MB for the pixels

    def __getitem__(self, idx):                           # Called by the DataLoader for each batch item
        path, x, y = self.tiles[idx]                      # Look up the precomputed coordinate for this sample index
        slide = openslide.OpenSlide(path)                 # Open the slide handle again (or fetch from LRU cache in production)
        tile = slide.read_region((x, y), 0, (self.tile_size, self.tile_size))
                                                          # Lazy decode only the 256x256 region at level 0 — no adjacent pixels loaded
        tile = tile.convert("RGB")                        # OpenSlide returns RGBA; strip alpha channel for 3-channel model input

        if self.transform:                                # Apply stain normalization + augmentation if provided
            tile = self.transform(tile)                   # Usually a torchvision Compose: ToTensor -> MacenkoNormalize -> RandomFlip

        return tile                                       # Returns a single PIL Image (or Tensor if transform converts it) for the batch collator

Store coordinates, not pixels. A dataset of 1,000 slides has ~20 million tile coordinates. Storing those plus a slide path reference is a few hundred MB. Storing all pixels would be 4 TB.

What Actually Goes Wrong

1. Slide scanning artifacts. Tissue folds, bubbles, out-of-focus regions, ink marks. Your model learns “black triangle = cancer” because all cancer slides have ink marks.

def detect_artifacts(tile):                               # Filters out tiles with blur (out-of-focus scan) or excessive edges (tissue fold/ink)
    laplacian = cv2.Laplacian(np.array(tile), cv2.CV_64F) # Laplacian edge detection — high-frequency content indicates sharp focus
    focus_score = laplacian.var()                          # Variance of the Laplacian: low variance means the tile is blurry (no fine detail)

    gray = cv2.cvtColor(np.array(tile), cv2.COLOR_RGB2GRAY) # Single-channel intensity for edge detection with Canny
    edges = cv2.Canny(gray, 50, 150)                       # Canny with low=50, high=150 — tuned for H&E stain edge contrast
    edge_density = np.mean(edges > 0)                      # Fraction of pixels that are edge pixels in the binary Canny output

    if focus_score < 100 or edge_density > 0.3:            # Blur threshold: < 100 means out-of-focus (common in WSI scanning artifacts)
                                                           # Edge-threshold: > 0.3 means tissue fold, ink mark, or bubble (too many sharp boundaries)
        return False                                        # Reject this tile — it would confuse the model with non-tissue features
    return True                                             # Accept the tile as valid tissue for training or inference

2. Label noise. Pathology labels are made by humans. Inter-observer agreement is around 70-85%. Account for it — soft labels, label smoothing, or multiple reviewers.

3. Memory management. Don’t keep 100 slides open. Each OpenSlide handle uses significant memory. Use an LRU cache:

from functools import lru_cache                            # lru_cache imported but not used in this snippet — could decorate get_slide for automatic eviction

class SlideCache:                                          # Manual LRU-like cache for OpenSlide handles to cap OS file descriptor usage
    def __init__(self, max_slides=10):                     # max_slides=10: typical physical memory limit — each handle reserves significant RAM for the pyramid index
        self.max_slides = max_slides                       # Upper bound on concurrently open slide handles
        self.cache = {}                                    # Dict mapping file path -> OpenSlide object; insertion order tracks recency

    def get_slide(self, path):                             # Request a slide by path; returns cached handle if available
        if path not in self.cache:                         # Cache miss — need to open a new handle
            if len(self.cache) >= self.max_slides:         # At capacity: evict the oldest (first inserted) slide handle
                oldest = next(iter(self.cache))            # dict insertion order is preserved in Python 3.7+; first key is least recently opened
                self.cache[oldest].close()                 # Close the OpenSlide file handle to release OS file descriptors and allocated memory
                del self.cache[oldest]                     # Remove the evicted entry from the dict
            self.cache[path] = openslide.OpenSlide(path)   # Open a new handle — reads the pyramid index but doesn't decode pixels
        return self.cache[path]                            # Return the (possibly just-opened) slide handle for region reads

4. Magnification consistency. Your dataset might mix 20x and 40x slides. Standardize to one magnification before training.

The Real Recommendation

Starting a medical imaging project:

  • Use OpenSlide for reading. Nothing else handles SVS/TIFF properly.
  • Tile at 20x, 256x256. Community standard.
  • Do stain normalization on every tile before training. Macenko is fine.
  • Precompute tile coordinates, not pixels.
  • Make a tissue mask. Don’t waste compute on blank glass.
  • Add focus filtering.
  • Expect 30-50% of your time to be data pipeline.

Medical CV is not harder than regular CV. It’s just different. You’re not fighting architecture choices — you’re fighting file formats, storage costs, and inconsistent stains.

Comments

Leave a comment

What is 2 + 3?
Please solve the arithmetic question above to verify you're human.