17. Put it all in one pot — Simpson's paradox · Dr. Seok's Tumour-Infiltrating Lymphocytes, First Steps

Testing each cancer type separately is a nuisance, and few of them come out significant. Would it not be far more powerful to put all 4,691 patients together?

Let us try.

z = +3.75, p = 0.00018. A strong result. And the opposite direction to lesson 14.

Python Runs in your browser

import til

p = til.patients().dropna(subset=["OS", "OS_time"])
r = til.surv_test(p)          # ignore cancer type, split at the overall median

print("%s patients, all at once" % format(len(p), ","))
print("  cut point   infiltration %.2f%%" % p.til_pct.median())
print("  high group %s · low group %s" % (format(r["n_hi"], ","), format(r["n_lo"], ",")))
print("  z = %.2f   p = %.2e" % (r["z"], r["p"]))
print()
print("z is positive - meaning the high-infiltration side dies sooner.")

Split by cancer type in lesson 14, z was mostly negative — more infiltration, longer life. Pool everything and it comes out positive.

Both calculations are correct. Which should you believe?

Pooled, +3.75; within cancer types, −3.97. A textbook Simpson's paradox.

Python Runs in your browser

import til
import numpy as np

p = til.patients().dropna(subset=["OS", "OS_time"])

# treat cancer type as a stratum and compare only within strata (stratified log-rank)
O = E = V = 0.0
for t, g in p.groupby("tumor"):
    if len(g) < 30:
        continue
    hi, lo = til.split(g)
    time = np.concatenate([hi.OS_time.values, lo.OS_time.values])
    ev   = np.concatenate([hi.OS.values, lo.OS.values])
    grp  = np.concatenate([np.ones(len(hi)), np.zeros(len(lo))])
    for ti in np.sort(np.unique(time[ev == 1])):
        n, n1 = (time >= ti).sum(), ((time >= ti) & (grp == 1)).sum()
        d, d1 = ((time == ti) & (ev == 1)).sum(), ((time == ti) & (ev == 1) & (grp == 1)).sum()
        if n < 2:
            continue
        O += d1; E += d * n1 / n
        V += d * (n1/n) * (1 - n1/n) * (n - d) / (n - 1)

from scipy import stats
z = (O - E) / np.sqrt(V)
print("stratified log-rank  z = %.2f   p = %.2e" % (z, 2*stats.norm.sf(abs(z))))
print()
print("The same patients. Comparing only within a cancer type flipped the sign.")

Press the two modes and compare, then find the reason in the plot below.

Loading…

The reason is in the widget's scatter plot. Put one dot per cancer type, with its median infiltration rate on x and its five-year survival on y —

Breast and prostate — low infiltration (1.3%, 1.5%) and very good prognosis (over 80% at five years).
Stomach, rectum and cervix — high infiltration (9–12%) and correspondingly worse prognosis.

So pool everything and gather "the high-infiltration side", and that group fills up with patients from cancer types that had a bad prognosis to begin with. What we measured was not the effect of infiltration but the difference between cancer types.

pooled z = +3.75 · stratified z = −3.97

A relationship at the level of cancer types overturned the conclusion at the level of patients — a cousin of the ecological fallacy.

Python Runs in your browser

import til
import numpy as np

p = til.patients().dropna(subset=["OS", "OS_time"])
print("%-6s%6s%12s%12s" % ("type", "n", "median TIL", "5-yr surv"))
for t, g in p.groupby("tumor"):
    c = np.array(til.km_curve(g.OS_time.values, g.OS.values))
    s5 = np.interp(1825, c[:, 0], c[:, 1])
    print("%-6s%6d%11.2f%%%11.1f%%" % (t, len(g), g.til_pct.median(), 100*s5))
print()
print("Cancer types with lower infiltration tend to have higher five-year survival.")
print("That relationship is what fixed the sign of the pooled test.")

🤔 Check yourself

When the pooled and stratified tests disagree in sign, which is clinically useful?

🤔 Check yourself

What is the root cause of the Simpson's paradox here?