14. Reversion efficiency — what exactly are we counting? · Dr. Seok's Cancer-Cell Digital Twin, First Steps

The table in the last lesson said reversion 100%. That looks like a fine number. But what was the denominator of that 100%?

There are at least three possible definitions.

Definition	Denominator	What it measures
Overall normal fraction	All 256 initial states	How healthy the whole landscape is
Reversion efficiency	The 32 that were cancer	Whether the treatment works on patients
Cancer attractor abolished	—(yes/no)	Whether the possibility of cancer remains

The three answer different questions. And they give completely different values for the same treatment.

Take everything as the denominator and the 224 states that were healthy anyway inflate the score.

Python Runs in your browser

import bn

net = bn.twin()
base = net.landscape()
cancer0 = base.states_with("cancer")
normal0 = base.states_with("normal")

print(bn.pad("treatment", 26) + bn.pad("overall normal", 18)
      + bn.pad("reversion eff.", 18) + "cancer gone?")
print("-" * 74)
for name, p in [("untreated", None), ("DDX54 knockout", {"DDX54": 0}),
                ("PDL1 knockout", {"PDL1": 0}), ("p53 knockout", {"p53": 0})]:
    L = net.landscape(p)
    whole = L.count("normal") / L.total * 100
    eff = sum(1 for s in cancer0 if L.fate[s] == "normal") / len(cancer0) * 100
    gone = "yes" if L.count("cancer") == 0 else "no"
    print(bn.pad(name, 26) + f"{whole:>9.1f}%" + " " * 9
          + f"{eff:>9.1f}%" + " " * 9 + gone)

print()
print("Untreated, the 'overall normal fraction' is already 87.5% - high before any treatment.")
print("Read only that column and every treatment looks marvellous. The denominator is wrong.")

The test for a metric: does it read 0 when you do nothing?

Python Runs in your browser

import bn

net = bn.twin()
base = net.landscape()
cancer0 = base.states_with("cancer")

# A good metric is one that reads clearly differently before and after treatment
print("value of each metric with no treatment at all")
print("-" * 50)
print(f"  overall normal fraction : {base.count('normal')/base.total*100:5.1f}%   <- baseline already high")
print(f"  reversion efficiency    : {0:5.1f}%   <- baseline is 0")
print()
print("Pick the metric whose baseline is 0 and you can read 'treatment effect' honestly.")
print("That is why we use reversion efficiency.")

The habit of quoting the assumption alongside the number is what saves a researcher.

Python Runs in your browser

import bn

net = bn.twin()
base = net.landscape()

# Make plain that basin size is not 'the distribution of real cells'
print("the assumptions behind counting the cancer basin as 32")
print("-" * 60)
print("  - all 256 initial states are equally likely to occur")
print("  - i.e. the 8 genes take arbitrary values independently of each other")
print()
print("And in a real cell?")
print("  - CD47 and PDL1 both follow DDX54, so they are nearly always equal")
print("  - a cell is never born with T_Kill already switched on")
print()
print("So '32/256 = 12.5%' is not a probability. Read it as")
print("'how widely this circuit catches the cancer state'.")

🎮 Denominator bench

Change the denominator and watch the 'effect' of the same treatment move.

Identical computation, and the number swings around.

Loading…

🤔 Check yourself

What is the right denominator for 'reversion efficiency'?

🤔 Check yourself

Why is it a problem that the 'overall normal fraction' is already 87.5% untreated?

🤔 Check yourself

Why must 'the cancer basin is 12.5%' not be read as 'a 12.5% chance of cancer'?