Why items are hard
Anatomy of difficulty
Two questions behind every hard item. For a cell: what does the probe confuse it with, and how far apart are the two labels in the Cell Ontology? For a perturbation condition: is the measured response weak, generic, or specific, and which of those is hard to predict?
Representation: what the probe confuses
For every cell type, the label the pooled classical models most often predict instead, the share of errors it accounts for, and its distance from the true label in the Cell Ontology. Sort any column; search a type or a panel.
How much of a cell's difficulty is its type
Share of variance in cell difficulty explained by cell type alone, and by stratum (type × donor × context), per panel.
Where the stratum bar is far longer than the type bar, difficulty is driven by which donor or sample a cell came from, not by its type: the scBaseCount bone marrow, liver and multitissue panels.
Ontology covariates
Distance of wrong predictions
Perturbation: weak, generic or specific
Every condition is binned by its measured response before any model is scored. The bins are defined once and apply to every screen.
Under the unseen-perturbation split, no-reaction conditions are the easiest bin: a near-zero response is matched by a near-zero prediction, and the mean shift is small too. Generic responses are the hardest, because the mean shift already is the generic response. Under the unseen-combination split the order flips: with the perturbation seen in other contexts, specific responses transfer and are easy, while no-reaction and generic conditions leave a model nothing specific to transfer.
Bin shares by screen, unseen-perturbation split
Spearman correlation of difficulty with condition covariates
Cosine to the context mean shift is the strongest predictor of difficulty on both scales: the more a condition looks like the average response, the harder it is to beat the average.
Within each screen
Target class, genetic screens
Transcription-factor targets are easier under the unseen-perturbation split: their responses are large and distinctive. Targets that are also in the K562 essential screen are slightly harder there, and much easier under the unseen-combination split, where the same target has been seen in the other K562 screen.