Your Model Is Only as Good as Your Scanner
Medical images aren’t “pictures.” They’re physical measurements reconstructed into pixels. Every modality comes with hardware-level trade-offs.
Training on clean academic datasets and deploying on real clinical scans is a fast track to heartbreak.
The Signal vs. Noise Triangle
Every imaging modality faces this:
Resolution
▲
/ \
/ \
/ \
/ \
/ \
Dose ──────────────────────► Duration
You cannot maximize all three. Higher resolution needs more signal. More signal means higher radiation dose (CT, PET) or longer scan time (MRI). Longer scans mean motion artifacts.
CT — Dose vs. Noise
CT uses X-rays. More radiation = clearer image. Less radiation = grainier image (quantum mottle).
The fix: Train on low-dose scans or augment with simulated noise. Real CT noise is signal-dependent (Poisson):
import numpy as np # NumPy: core array ops for CT HU manipulation and Poisson noise generation
from scipy.ndimage import gaussian_filter # SciPy: unused here but available for post-smoothing low-dose CT
def simulate_low_dose(ct_slice, target_mAs_ratio=0.25): # Simulate reduced tube current; mAs ratio 0.25 ≈ 75% dose reduction
hu_image = ct_slice * 1.0 # Cast to float64 — HU values are typically int16 from DICOM; avoid integer division
mu_image = (hu_image + 1000) / 1000.0 # Convert Hounsfield Units to linear attenuation coefficient μ; μ_water = 1.0 at 0 HU
mu_image = np.clip(mu_image, 0.001, None) # Floor at 0.001 to prevent log(0) in Poisson — air (-1000 HU) would give μ=0
photon_count = mu_image * 100000 * target_mAs_ratio # Expected photon count per detector element; higher μ = more attenuation = fewer photons
noisy_photons = np.random.poisson(photon_count).astype(np.float32) # Poisson noise models quantum mottle — the dominant CT noise source, signal-dependent
noisy_mu = noisy_photons / (100000 * target_mAs_ratio) # Invert: measured photons → reconstructed μ (noisy version)
noisy_hu = noisy_mu * 1000 - 1000 # Convert μ back to Hounsfield Units: HU = 1000 × (μ − μ_water) / μ_water
return noisy_hu # Returns simulated low-dose CT slice in HU; train your model on this paired with clean reference
Add a denoising preprocessor in inference:
from skimage.restoration import denoise_nl_means # Non-local means denoising — preserves edges while suppressing quantum mottle
def preprocess_ct(slice_hu): # Inference-time denoising for low-dose CT; runs per 2D slice
slice_hu = np.clip(slice_hu, -200, 400) # Soft-tissue window: -200 HU (air/fat) to 400 HU (bone); discard irrelevant range
denoised = denoise_nl_means( # NLM works by averaging similar patches across the whole image — excellent for CT noise
slice_hu / 600.0, # Normalize to [≈-0.33, ≈0.67] so patch distances are well-scaled for the h parameter
patch_size=5, # 5×5 patch — small enough to capture local texture, large enough for reliable statistics
patch_distance=9, # Search window radius of 9 pixels — limits computational cost while finding enough similar patches
h=0.1, # Filter strength: 0.1 × standard deviation of noise; too low → under-denoised, too high → oversmoothed
fast_mode=True # Uses integral images for faster patch distance computation — acceptable quality, ~4× speedup
) * 600.0 # Rescale back to Hounsfield Units
return denoised # Cleaned slice ready for your segmentation/detection model
MRI — The Duration Problem
High-resolution MRI takes 5-10 minutes. Patients breathe, their hearts beat, they move. Motion artifacts look like ghosting.
The fix — k-space augmentation: MRI data is acquired in frequency space (k-space). Motion corrupts specific lines. Simulate this during training:
import numpy as np # NumPy: FFT routines for k-space manipulation and random motion line selection
def simulate_motion_artifact(mri_slice, num_corrupt_lines=20): # Bulk motion during readout: swap k-space lines to mimic patient movement
kspace = np.fft.fft2(mri_slice) # 2D FFT: image space → k-space (frequency/phase encoding domain)
kspace_shifted = np.fft.fftshift(kspace) # Shift zero-frequency component to center — DC is at [height/2, width/2]
height = kspace_shifted.shape[0] # Number of phase-encoding steps (rows in k-space)
center = height // 2 # Center row index — central k-space lines encode contrast; peripheral encode fine detail
corrupted = kspace_shifted.copy() # Work on a copy so each motion realization is independent; keep original for comparison
for _ in range(num_corrupt_lines): # Each iteration simulates one "shot" where the patient moved during readout
line = np.random.randint(center + 20, height - 10) # Corrupt only outer k-space lines: central lines would obliterate image
shift = np.random.randint(-5, 5) # Random displacement in phase-encoding direction — mimics sudden body motion
source_line = line + shift # The line that was actually acquired instead of the intended line
if 0 <= source_line < height: # Clamp check: shifting off the k-space grid would cause index errors
corrupted[line, :] = kspace_shifted[source_line, :] # Replace corrupted line with the mis-acquired line
result = np.fft.ifft2(np.fft.ifftshift(corrupted)) # Inverse shift then inverse FFT: corrupted k-space → image with motion ghosting
return np.abs(result) # Magnitude image: discard phase; clinical MRI typically displays magnitude only
For inference, entropy-based motion correction:
from scipy.optimize import minimize # SciPy optimizer: L-BFGS-B for entropy minimization over k-space degrees of freedom
import numpy as np # NumPy: FFT for transform loops; flatten/reshape for optimizer interface
def correct_motion(kspace_data): # Self-nav entropy minimization: motion-free images have lower entropy than motion-corrupted ones
def image_entropy(kspace_flat): # Objective function: returns negative entropy of the image reconstructed from flat k-space
kspace = kspace_flat.reshape(kspace_data.shape) # Reshape flat parameter vector back to 2D k-space matrix
image = np.abs(np.fft.ifft2(np.fft.ifftshift(kspace))) # Full reconstruction pipeline: shift → iFFT → magnitude
image = image / image.sum() # Normalize to probability distribution (sum = 1) for Shannon entropy calculation
return -np.sum(image * np.log(image + 1e-10)) # Negative Shannon entropy: minimizer finds lowest-entropy (sharpest) image
result = minimize( # Quasi-Newton optimizer: searches k-space values to minimize image entropy
image_entropy, # Objective: entropy as proxy for motion artifact severity
kspace_data.flatten(), # Initial guess: the acquired (potentially motion-corrupted) k-space
method="L-BFGS-B", # Limited-memory BFGS with bounds — handles high-dimensional k-space without full Hessian
options={"maxiter": 50} # 50 iterations max — each iteration requires a full FFT; trade-off between quality and runtime
)
corrected = result.x.reshape(kspace_data.shape) # Reshape the optimal k-space back to 2D
return np.abs(np.fft.ifft2(np.fft.ifftshift(corrected))) # Reconstruct the motion-corrected image from optimized k-space
Takes a few seconds per slice, but saves you from throwing out whole scans.
Ultrasound — The Bone and Air Wall
Ultrasound cannot see through bone or air. Sound waves reflect off bone and scatter in air-filled organs.
The fix — learn shadow patterns:
def detect_acoustic_shadow(ultrasound_slice, threshold=0.1): # Identify regions below a rib/bone/gas where signal is absent
normalized = ultrasound_slice / (ultrasound_slice.max() + 1e-6) # Scale to [0, 1] — the 1e-6 prevents division by zero for dark frames
column_intensity = normalized.mean(axis=1) # Mean intensity across each row: shadows appear as vertically dim columns
shadow_mask = np.zeros_like(normalized) # Initialize binary mask — 0 = visible tissue, 1 = acoustic shadow
for col in range(normalized.shape[1]): # Iterate over each column (A-line) independently — shadows are column-localized
if column_intensity[col] < threshold: # If this column's mean intensity is suspiciously low, it may be shadowed
shadow_start = np.argmax(normalized[:, col] > threshold * 2) # Find the first pixel exceeding 2× threshold — marks the shadow boundary
shadow_mask[shadow_start:, col] = 1.0 # Everything below the shadow boundary in this column is marked as missing signal
return shadow_mask # Binary mask: 1 where ultrasound cannot penetrate; feed as extra channel to your CNN
shadow_mask = detect_acoustic_shadow(us_frame) # Example: generate shadow mask for one B-mode ultrasound frame
model_input = np.stack([us_frame, shadow_mask], axis=-1) # Stack intensity + mask as 2-channel input — model learns to ignore shadowed regions
Don’t try to inpaint ultrasound shadows — results are medically dangerous. Instead tell your model where the signal is missing.
PET — The Count Rate Nightmare
PET measures radioactive tracer concentration. Lower dose or shorter scan = fewer counts = more noise.
The fix — count-level augmentation and smoothing:
def simulate_low_count_pet(pet_slice, count_fraction=0.25): # Simulate reduced injected dose or shorter acquisition: fewer counts = noisier
max_val = pet_slice.max() # Peak tracer uptake in this slice — used to normalize activity to a relative scale
activity = pet_slice / max_val # Normalize to [0, 1]: relative tracer concentration (SUV-like proportional)
low_count = np.random.poisson(activity * 1000 * count_fraction) # Poisson noise on counts: fewer total counts = higher relative noise (lower SNR)
low_count = low_count / (1000 * count_fraction) * max_val # Rescale back to original intensity range — preserves SUV quantification
return low_count.astype(pet_slice.dtype) # Return noisy slice in original dtype (typically float32 from DICOM)
def gaussian_post_filter(pet_slice, sigma=1.5): # Optional post-reconstruction smoothing: Gaussian kernel matches clinical PET reconstruction
from scipy.ndimage import gaussian_filter # Import inside function to keep it self-contained
return gaussian_filter(pet_slice, sigma=sigma) # sigma=1.5 mm: matches typical clinical PET reconstruction filter (e.g., 4-6 mm FWHM)
The Hardware Variability Trap
A scan on a 1.5T MRI looks different from the same patient on a 3.0T MRI. GE, Siemens, and Philips all process raw data differently.
The fix — histogram matching and domain randomization:
import numpy as np # NumPy: random augmentation, meshgrid for bias field, basic statistics
from skimage.exposure import match_histograms # skimage: match cumulative distribution of intensities across scanner vendors
def normalize_across_scanners(image, target_histogram=None): # Z-score normalization with optional histogram matching to a reference scanner
image = np.clip(image, -100, 300) # Clip to soft-tissue window: -100 HU to 300 HU — ignores air (-1024) and dense bone (1000+)
mean = image.mean() # Mean intensity of the clipped region — varies between vendors due to different reconstruction kernels
std = image.std() + 1e-6 # Standard deviation with epsilon — prevents division by zero for uniform noise-only regions
normalized = (image - mean) / std # Z-score normalization: centers intensity distribution at 0, scales to unit variance
if target_histogram is not None: # If a reference scanner histogram is provided, adjust this slice's distribution to match
normalized = match_histograms(normalized, target_histogram) # Per-pixel CDF matching — aligns dynamic range across Siemens/GE/Philips
return normalized # Normalized slice: closer distribution to the reference scanner; improves cross-vendor model generalisation
def random_intensity_augment(image): # Domain randomization: applies random gamma and bias field to simulate scanner variability
gamma = np.random.uniform(0.8, 1.2) # Random gamma correction: 1.0 = linear; < 1 brightens dark tissue, > 1 darkens bright tissue
image = image ** gamma # Power-law transform: models nonlinear reconstruction transfer functions across scanner vendors
x = np.linspace(-1, 1, image.shape[1]) # Horizontal coordinate normalized to [-1, 1] — centre of image is (0, 0)
y = np.linspace(-1, 1, image.shape[0]) # Vertical coordinate normalized to [-1, 1]
xx, yy = np.meshgrid(x, y) # 2D grid of coordinates for per-pixel bias computation
bias = 1 + 0.1 * np.random.randn() * (xx**2 + yy**2) # Quadratic bias field: simulates slow-varying intensity inhomogeneity from B1 field non-uniformity
image = image * bias # Apply bias field multiplicatively — mimics receive-coil sensitivity profile variation
return image # Augmented slice: your model sees different "scanner feels" during training and becomes vendor-agnostic
Multi-Modal Registration
Real workflows use multiple modalities — CT fused with PET fused with MRI. Registration is required and never perfect:
import SimpleITK as sitk # SimpleITK: NiftyReg-powered registration framework with medical image I/O
def register_pet_to_ct(pet_path, ct_path): # Register low-resolution PET (functional) to high-resolution CT (anatomical) — standard clinical fusion
pet = sitk.ReadImage(pet_path) # Read PET DICOM/NIfTI: typically 128×128×? with large (3-5 mm) voxels, limited FOV
ct = sitk.ReadImage(ct_path) # Read CT DICOM/NIfTI: typically 512×512×? with small (0.5-1 mm) voxels, full anatomical coverage
resampler = sitk.ResampleImageFilter() # Filter that will warp PET to CT physical space using the final transform
resampler.SetReferenceImage(ct) # Set CT as reference: resampled PET will have same origin, spacing, direction, and size as CT
resampler.SetInterpolator(sitk.sitkLinear) # Linear interpolation for PET counts — nearest would destroy SUV quantification; cubic is overkill
resampler.SetDefaultPixelValue(0) # Voxels outside PET FOV after transform are set to zero (no tracer activity)
transform = sitk.CenteredTransformInitializer( # Compute initial rigid alignment from image centroids (center-of-mass)
ct, pet, # Fixed (CT) and moving (PET) images for initializer
sitk.Euler3DTransform(), # Rigid transform with 3 rotation + 3 translation DOF — patient position difference, not deformation
sitk.CenteredTransformInitializerFilter.GEOMETRY # Use geometric centres (centroids of the image grids), not intensity moments
)
optimizer = sitk.RegularStepGradientDescentOptimizerv4() # Gradient descent with adaptive step size — robust to local minima in mutual information
optimizer.SetLearningRate(1.0) # Initial step size of 1.0 (in transform parameter space) — aggressive but decreased on plateau
optimizer.SetMinimumStepLength(0.001) # Stopping criterion: step size below 0.001 means the optimizer converged
registration = sitk.ImageRegistrationMethod() # Main registration object: ties together metric, optimizer, transform, interpolator
registration.SetOptimizer(optimizer) # Attach the gradient descent optimizer
registration.SetInterpolator(sitk.sitkLinear) # Interpolation for metric evaluation — linear is fast and sufficiently accurate
registration.SetMetricAsMattesMutualInformation(numberOfHistogramBins=50) # Mattes MI: measures statistical dependence between CT (HU) and PET (counts) — both modalities have no direct intensity relationship
registration.SetInitialTransform(transform) # Start from the centroid-initialized rigid transform
final_transform = registration.Execute( # Run the registration: iteratively adjusts the Euler transform to maximise MI
sitk.Cast(ct, sitk.sitkFloat32), # Cast CT to float32 — SimpleITK expects matching pixel types for metric computation
sitk.Cast(pet, sitk.sitkFloat32) # Cast PET to float32
)
resampler.SetTransform(final_transform) # Apply the optimized transform to the resampler
registered_pet = resampler.Execute(pet) # Resample PET into CT space: the output PET image matches CT geometry
return sitk.GetArrayFromImage(registered_pet) # Return as NumPy array — shape matches CT, voxel values represent registered SUV
The Practical Preprocessing Pipeline
class ClinicalPreprocessor: # End-to-end pipeline that bridges raw DICOM physics to normalized input tensors
def __init__(self, modality="CT", target_spacing=(1.0, 1.0, 1.0)): # Target spacing: 1 mm isotropic for most deep learning pipelines
self.modality = modality # Modality tag: controls modality-specific clipping and default augmentations
self.target_spacing = target_spacing # Resample target: CT slices are often 0.5-3 mm; models expect consistent voxel spacing
def __call__(self, volume, metadata): # Callable interface: accepts 3D volume + dict with "spacing" key (e.g., (0.7, 0.7, 1.25))
volume = self.resample(volume, metadata["spacing"]) # Resample from acquisition spacing → target spacing: preserves anatomical scale for CNNs
if self.modality == "CT": # CT-specific: clip to diagnostic soft-tissue window — excludes air and metal/bone extremes
volume = np.clip(volume, -200, 400) # [-200, 400] HU: covers fat (-100), water (0), muscle (40), and contrast-enhanced tissue (100-300)
elif self.modality == "PET": # PET-specific: clip negative SUV values to zero — negative counts are reconstruction artifacts, not real uptake
volume = np.clip(volume, 0, volume.max()) # SUV range: typically 0-15; max() preserves individual scan range
volume = (volume - volume.mean()) / (volume.std() + 1e-6) # Z-score normalisation across the whole volume: centres to zero mean, unit variance
if np.random.random() < 0.3: # 30% chance: apply Gaussian noise to simulate small coil/respiratory artifacts during training
volume += np.random.normal(0, 0.05, volume.shape) # Additive Gaussian noise: std 0.05 after z-score normalisation
if np.random.random() < 0.2: # 20% chance: apply smooth bias field to simulate MRI inhomogeneity or CT beam hardening
volume = self.apply_bias_field(volume) # Low-frequency intensity modulation; helps model generalise across field strengths (1.5T vs 3.0T)
return volume.astype(np.float32) # Cast to float32 — standard dtype for PyTorch/TensorFlow medical imaging pipelines
The Bottom Line
Medical imaging ML is 80% physics, 20% model architecture. If you don’t account for how the data was acquired, your model will fail in the real world.
Things that will hurt you:
- Training on high-dose CT, deploying on low-dose
- Training on motion-free MRI, deploying on restless patients
- Training on one vendor’s data, deploying on another’s
- Ignoring ultrasound shadowing
Things that help:
- Simulate acquisition physics during training
- Use histogram matching across scanners
- Add artifact masks as input channels
- Validate on multi-center, multi-vendor data
The physics doesn’t care about your gradient descent. But if you build for it, your model will work in any hospital.
Comments
Leave a comment