18. Which hospital did the slide come from — batch effects · Dr. Seok's Tumour-Infiltrating Lymphocytes, First Steps

So far we have treated the infiltration rate as a property of a patient. But the value came off a slide, and the slide was made in a hospital.

Hospitals differ in fixation time, in stain concentration, in scanner. The CNN learned from colour intensity and texture, so it can respond to those differences.

Fortunately the TCGA barcode carries a clue. The two characters in TCGA-XX-YYYY identify the tissue source site (TSS) that sent the sample.

The same cancer type, and yet the median runs from 0.47% to 3.25% across sites — a sevenfold spread.

Python Runs in your browser

import til

d = til.slides()
d = d[d.tumor == "BRCA"].copy()
d["site"] = d.slide.str[5:7]

t = d.groupby("site").til_pct.agg(slides="count", median="median").round(2)
t = t[t.slides >= 12].sort_values("median", ascending=False)
print("breast cancer, overall median %.2f%%" % d.til_pct.median())
print()
print(t.to_string())
print()
print("highest site / lowest site = %.1fx" % (t["median"].max()/t["median"].min()))

This is a batch effect. Too large to dismiss as chance.

Python Runs in your browser

import til
from scipy import stats

d = til.slides()
for T in ("BRCA", "LUAD", "STAD"):
    g = d[d.tumor == T].copy()
    g["site"] = g.slide.str[5:7]
    groups = [h.til_pct.values for _, h in g.groupby("site") if len(h) >= 12]
    h, p = stats.kruskal(*groups)
    print("%s  %d sites  Kruskal-Wallis  p = %.2e" % (T, len(groups), p))
print()
print("All three cancer types show a clear difference between sites.")

Switch cancer types and look at the per-site medians. The horizontal line is that cancer type's overall median.

Loading…

Be careful here. A difference between sites does not automatically mean technical noise.

Sites can differ in their patients — ethnicity, age, stage, subtype.
Sites can differ in surgical practice — how much was resected, which region was sampled.
And they differ in staining and scanning.

We cannot separate those three, because site and patient characteristics are tangled together. So what we may say ends here — "there are large differences between sites, and this data cannot identify their cause."

You can also measure the machine's own wobble. As lesson 3 showed, this data holds both a map made by model v1 and a probability map made by model v2 for the same slide. Look at how many of the same cells the two models painted red and you get the agreement between model versions.

Same slide, same lab, the next version of the model. And yet only half the red cells agree.

Python Runs in your browser

import til
import numpy as np

v1 = til.tilmap("TCGA-AA-3977") == 2          # model v1's verdict
p, tissue = til.probmap("TCGA-AA-3977")       # model v2's probabilities

print("cells v1 painted red: %s" % format(int(v1.sum()), ","))
print()
print("%6s%10s%10s%8s" % ("thr", "v2 cells", "overlap", "Dice"))
for thr in (0.10, 0.15, 0.20, 0.25, 0.30):
    v2 = (p >= thr) & tissue
    overlap = int((v1 & v2).sum())
    dice = 2 * overlap / (int(v2.sum()) + int(v1.sum()))
    print("%6.2f%10s%10s%8.3f" % (thr, format(int(v2.sum()), ","), format(overlap, ","), dice))
print()
print("Even at the threshold that matches the counts (0.20), Dice is 0.46 - under half.")

The infiltration rate as a number comes out similar from the two models (12.1% against 12.0%). But the maps agree only half the time.

So a study that uses the infiltration rate will mostly survive a change of model version. A study that uses "where the lymphocytes are" — spatial analysis, tumour-immune boundaries, neighbourhood relationships — is far more sensitive to it.

The cluster count and the depth we built in lessons 8 and 9 are exactly those kinds of value. To use them you must fix the model version and state it.

🤔 Check yourself

The median infiltration rate differed sevenfold across tissue source sites. What is the right conclusion?

🤔 Check yourself

A Dice of 0.46 between model v1 and v2 most threatens which kind of analysis?