Architecture · method selection
The right method
is rarely the
loudest one
Every computational task hides a scientific choice: which method fits this data. The literature makes one answer very loud, and for a given dataset that answer is often the wrong one.
- 01 CONTEXT
- 02 SPREAD
- 03 RECALL
- 04 SCORE
CONTEXT
one cohort · 92 lesional against 82 normal, as RPKM
- corpus
- 35,897
- RNA-seq DE articles
- methods measured
- 16
- named in that corpus
- head share
- 92.5%
- of named methods
- tail evidence
- 5
- papers, 4 one thread
Above: sixteen differential-expression methods, by how often PubMed names them. Bars and the curve are measured counts on a linear axis; the cohort is the real GSE54456 design; the search window is schematic, and what it contains is measured in fig. 04. Retrieved 2026-07-02.
The three premises
Why a scientific harness has to own method selection
- premise 01
Biological context decides which method is correct.
Sample size, the form the data arrives in and the variation between samples change which assumptions hold, and therefore which method is applicable.
3contexts, three different correct answersBlock 01 · three contexts - premise 02
The applicable method is often sparse in the literature.
Candidate methods are distributed with a long tail. Three tools hold most of the field, and the method that fits a given dataset can rest on a handful of papers.
92.5%of named methods are three tools and their relativesBlock 02 · the distribution - premise 03
A language model returns the head, and so does a generic search.
A model answering from its own weights reproduces the most frequent answer, and relevance ranking on the generic task returns the same consensus.
0 / 50papers in the retrieval window name a rank testBlock 03 · the search window
What follows
None of the three is fixed by a larger model
One premise is a property of the biology, two are properties of the literature and of how a model is trained on it. A system that selects methods has to be built to handle all three.
- 01
Read the context first
Inspect the data before choosing anything. Sample size, the form it arrives in and the variation between samples are what make a method applicable or not.
- 02
Retrieve around what the inspection found
Search on the condition rather than on the generic task, and weight a few methodological sources above the volume of consensus usage.
- 03
Score the candidates, and escalate the rest
Grade every candidate on applicability to this context, keep a contested result contested, and hand back the cases no evidence settles.
Block 04 builds this, and names the ten capabilities it takes.Skip to the response
Three cohorts, one question, three different right answers
Nothing about the question changes across these three. Only the biology does, and with it the method that can honestly answer it.
fig. 01 · three contexts, three answers
Read the context first, and the method follows from it
Each column is one real study design. Three properties of the cohort are shown and the rest are named, because three is enough to change the answer. Watch the last row: the same question answered from a model’s own weights returns one method in all three contexts, including the one where no single method is defensible.
The same question, asked of three cohorts: how do you find the genes that differ?
cycling
A controlled experiment
Three mice per group, one perturbation
- samples per group
- 3
- data as it arrives
- raw counts
- variation between samples
- low, inbred and controlled
+ tissue, platform, read depth, batch, and the rest of the record
selected on this context
Count model · DESeq2 or edgeR
Three replicates carry too little data per gene to estimate variance, so the model borrows strength across thousands of genes. A rank test cannot reach significance at all.
A patient cohort
GSE54456 · 92 lesional against 82 normal skin
- samples per group
- 92 against 82
- data as it arrives
- RPKM, not raw counts
- variation between samples
- high, patient to patient
+ lesion severity, treatment history, site, subtype, and the rest of the record
selected on this context
Rank test · Wilcoxon rank-sum
At population scale the count model’s distributional assumption is violated in ways that inflate false positives, and it cannot consume RPKM as provided. A rank test assumes no distribution and has ample power at this size.
A multi-site cohort
Forty patients per group, collected across three sites
- samples per group
- 40
- data as it arrives
- raw counts
- variation between samples
- high, plus site and batch
+ protocol version, operator, storage time, subtype, and the rest of the record
selected on this context
No single answer · escalate
The count model’s assumption is the liability at this sample size, and the rank test cannot adjust for the site structure. Two credible options remain and no available evidence settles the trade-off, so the choice belongs to a scientist.
DESeq2context 01
DESeq2context 02
DESeq2context 03
Recorded in the skill-validation log: an ungrounded model asked to build a DEG method for GSE54456 defaulted to a count model, and only grounding plus a context gate on the data flipped the constructed method.
what this block establishesA method is applicable or not applicable to a context. A system that answers before it has read the context is answering a different question from the one it was asked.
Three tools hold most of the field. The rest is a long thin tail
Sixteen methods, measured inside one corpus of 35,897 articles. Move the cut in the figure and the concentration becomes a number.
Articles naming each method · within the 35,897-article RNA-seq differential-expression corpus
Click a method to move the cut between head and tail.
Article axis
The cut, at rank 3
- Head · ranks 1 to 3
- 72%
- Tail · ranks 4 to 16
- 28%
1,465 of 2,036 measured mentions
13 methods share what is left
Mentions are non-exclusive. A paper may name several methods.
edgeR
- Statistical model
- Negative-binomial GLM
- Family
- Distribution-based
- Articles
- 385
- Share of corpus
- 1.07%
Articles and share are measured. Statistical model and family are an editorial classification taken from the method’s primary reference.
Resolve 1.83% against 92.5%
The same counts answer two different questions
| Quantity | Articles | % of 1,632 |
|---|---|---|
| Full corpus | 35,897 | — |
| Papers naming any recognized DE method | 1,632 | 100% |
| Distribution-based (count / linear) tools, union | 1,509 | 92.5% |
| of which the big three: DESeq2, edgeR, limma | 1,282 | 78.6% |
| Distribution-free rank-test family, union | 152 | 9.3% |
| Overlap, papers naming both | 29 | 1.8% |
The method-labeled subset is the denominator for the sentence "X% of the field uses Y". B = 1,509 plus 152 minus 29 = 1,632 by inclusion and exclusion; shares exceed 100% slightly because 29 papers name both families.
Source · PubMed title/abstract counts, retrieved 2026-07-02, within one fixed corpus. Footprints are non-exclusive. The rank-frequency line spans more than two orders of magnitude; no power-law claim is made.
And the evidence for the tail method is thinner than the tail
A generous keyword search finds 23 papers that might recommend a rank test at large sample sizes. Reading all 23 leaves five that genuinely do, and four of those five are one paper and the debate it started.
From any mention of a rank test to genuine advocacy for it at large n
Select a stage to see how the number was obtained.
97 → 23 → 5 → 1
Bars are on a log scale. The final stage is a single article.
Consensus against this stage
1,282 papers name one of the big three; 5 sit at this stage.
All 23 abstracts read and classified A / B / C
Reading the 23 splits them into 5 genuine large-n advocacy, 10 distribution-free methods argued for a different reason, and 8 false-positive co-mentions. The keyword count overstates support by roughly 4.6×.
- AGenuine large-n advocacy
- 5
- BDistribution-free method, different rationale
- 10
- CFalse-positive co-mention
- 8
The five advocacy papers
Anchor. Permutation analysis of human population RNA-seq; recommends the rank-sum test at large n.
doi:10.1186/s13059-022-02648-4Authors’ reply; reaffirms the rank-sum test as most robust.
doi:10.1186/s13059-024-03232-8Rebuttal. Argues part of the effect is a normalization artifact.
doi:10.1186/s13059-024-03231-9Winsorization comment on the anchor paper.
doi:10.1186/s13059-024-03230-wThe one independent source: a rank test on raw data outperforms limma-voom at large n.
doi:10.1038/s41598-019-41315-w
Two of the five argue against the recommendation and are counted anyway. A contested result stays contested.
Source · PubMed, retrieved 2026-07-02. Stages 1 and 2 are Boolean counts; stages 3 and 4 are all 23 abstracts read and classified, so the last two numbers are manual work and are reported as such.
what this block establishesSparse is not the same as wrong, and popular is not the same as applicable. The evidence for the applicable method here is thin and actively contested, and both facts have to survive into the answer.
Reading the first page of results does not reach the tail
A tool-using system reads the top results, not the field. Probe those 50 papers for a rank test and nothing answers.
The 50 most relevant papers for the corpus query, under PubMed “Best Match”
Probe the window for a method and see how much of it responds.
n = 50 · read by hand
— / 50
Every square is one record in the window. 41 of them name no differential-expression tool or test at all.
- Names DESeq2 and / or edgeR
- 8
- Names another distribution-based tool only
- 1
- Names no DE tool or test in title / abstract
- 41
- Names any distribution-free test
- 0
A unit chart. The four classes partition the 50 records; square positions carry no meaning.
- DESeq2
- 8
- edgeR
- 7
- limma / voom
- 3
- Cuffdiff / Cufflinks
- 1
- Any other named tool
- 0
- t-test / ANOVA
- 0
- Wilcoxon / rank-sum / distribution-free
- 0
- Papers advocating a rank test at large n
- 0
Mentions, non-exclusive, which is why they exceed the 9 papers that name any tool.
Paper types
Source · PubMed “Best Match” top 50 for the corpus query, retrieved 2026-07-02; all 50 titles and abstracts classified by hand. “Best Match” is a proprietary learned ranker, so a different engine would return a different window.
Two senses of the word “parametric”, kept apart
Parametric memory
a property of the language model
Knowledge stored implicitly in a model’s weights, as opposed to non-parametric memory, which is an external retrieval store (Lewis et al., 2020). Because the weights compress a training corpus, a model answering cold reproduces its most frequent answer. This is the sense used in the prose on this page.
Distribution-based method
a property of a statistical test
A method is distribution-based, or parametric in the statistical sense, if it assumes the data follow a specified distribution, such as the negative binomial. A distribution-free method assumes none and works on ranks. This page says distribution-based and distribution-free throughout, and never uses the bare word for a statistical method.
In this example the two senses happen to align. Parametric memory returns DESeq2, which is a distribution-based method, while the context-appropriate answer is a distribution-free one. The mechanism is training-corpus frequency, so the argument holds equally where the tail method is distribution-based.
what this block establishesReaching the tail is not a matter of searching harder. The head of a relevance ranking is the consensus, so the query itself has to be formulated around the condition the data showed.
Score the method against the context, then escalate what it cannot settle
The three premises leave one way out. Inspect the data, build the retrieval around what the inspection found, and score every candidate on how applicable it is to this context rather than on how often the field names it.
This is the third requirement, built. Seven dimensions, both candidates, scored under a named context, with an honest third outcome when the dimensions disagree and no evidence settles them.
Seven dimensions of applicability, scored for both candidates
The same three contexts as fig. 01. Watch the scores move, not the popularity.
- samples per group
- 82
- Population scale · n ≥ 20 per group
- data as it arrives
- RPKM, as distributed
- a count model cannot consume it
- batch or covariates
- None
- no adjustment required
Distributional assumption
Experimental context
Real cohorts carry outliers, heterogeneity and batch structure the model does not capture.
Assumes no distribution. It asks only whether one group tends to outrank the other.
Input as provided
Data processing and QC
Requires raw integer counts. RPKM cannot be consumed correctly.
Indifferent to normalization, since ranks survive any monotone transform.
Error calibration at this n
Data-to-signal evaluation
Under permuted labels, actual FDR sometimes exceeded 20% at a nominal 5%.
Cannot be misspecified, so nominal error rates are honest at any n.
Power at this n
Domain-specific methods
Strength borrowed across thousands of genes is what makes small designs work.
Ample power at population scale, where the low-n weakness disappears.
Covariate and batch adjustment
Experimental context
A GLM design matrix absorbs batch and covariate terms directly.
No adjustment available; none required by this condition.
Precedent on the named dataset
Scientific retrieval
No published run on GSE54456 as distributed.
The original GSE54456 study identified DEGs with the rank-sum test on RPKM.
Method evidence base for this condition
Contradiction detection
1,282 naming papers, and a demonstrated failure mode at this scale.
5 verified advocacy papers, 4 of them one thread; actively contested.
corpus footprint
1,282
1,282 corpus papers name one of the big three.
5
5 verified advocacy papers, 4 of them one debate thread.
Rank test · Wilcoxon rank-sum
- The data ship as RPKM, and a count model requires raw integer counts.
- At 82 per group the rank test has ample power, so its low-n weakness does not apply.
- This is what the original GSE54456 study did, verified from its Methods.
contestedThe large-n recommendation is well-supported and actively debated: a 2024 rebuttal attributes part of the effect to normalization, and the authors replied. Treat it as a strong contested result.
Rubric cells are an editorial reading of the cited primary literature evaluated against the context above. The sample-size bands read two published results, population scale of tens to hundreds per group and a benchmark tested only to n = 12, and are not a published threshold. Corpus footprints are measured PubMed counts.
Ten capabilities carry that, and they are architectural
Inspect the data, formulate retrieval around what the inspection found, score the candidates, keep a contested result contested, escalate what nothing settles, and record the choice so it can be challenged. Each is a capability in the architecture, in its own words.
Experimental context
in method selectionReads the data before reasoning about it: input format, per-group n, batch structure, known confounders. This is what produces the triggering condition at all.
- What it does
- Captures study design, cohort definition, tissue and model system, platform, batch structure, sample quality and known confounders.
- Why it is necessary
- The same expression change means different things in lesional skin, a cell line, or a mouse model. Context is what licenses a mechanistic claim.
- What a system does wrong without it
- The system produces statistically valid, biologically meaningless results, and cannot detect that it has done so.
what this block establishesMethod selection is a function of the harness rather than a property of a larger model. Either it is grounded, context-gated and scored, or it is recalled.
Where this analysis can be attacked
Naming frequency is a proxy, the family labels are hand-assigned, the keyword filter is arbitrary, and the recommendation at the centre of the example is contested. Open any objection.
Raised against the analysis. Each response is a measurement, not a rebuttal.
True, these are naming-frequency proxies. The magnitude (92.5% against 9.3%) dwarfs any plausible measurement bias, the retrieval window points the same way with a starker 0 of 50, and the recorded model behavior matches the prediction.
What no response removes. These bound what the numbers can be used to claim.
Every count on this page was retrieved from PubMed (NLM) via E-utilities ESearch on 2026-07-02 against one fixed corpus: ("RNA-seq"[tiab] OR "RNA sequencing"[tiab]) AND ("differential expression"[tiab] OR "differentially expressed"[tiab]). Each method footprint is CORPUS AND <method token>[tiab]. Classifications of a method’s statistical family are editorial and labeled where they appear.
- 01Li Y, Ge X, Peng F, Li W, Li JJ. Exaggerated false positives by popular differential expression methods when analyzing human population samples. Genome Biol 2022;23:79.doi:10.1186/s13059-022-02648-4The anchor. Recommends the rank-sum test for large-n population RNA-seq.
- 02Hejblum BP, et al. Neglecting the impact of normalization. Genome Biol 2024;25:281.doi:10.1186/s13059-024-03231-9Rebuttal.
- 03Ge X, et al. Response to "Neglecting normalization" and "Winsorization". Genome Biol 2024;25:283.doi:10.1186/s13059-024-03232-8Reply.
- 04Winsorization comment. Genome Biol 2024.doi:10.1186/s13059-024-03230-w
- 05Independent large-n rank-test finding. Sci Rep 2019;9.doi:10.1038/s41598-019-41315-w
- 06Li B, Tsoi LC, Swindell WR, et al. Transcriptome analysis of psoriasis in a large case-control sample. J Invest Dermatol 2014;134:1828 to 1838.doi:10.1038/jid.2014.28The GSE54456 study. Used the rank-sum test on RPKM, verified from its Methods.
- 07Li D, et al. An evaluation of RNA-seq differential analysis methods. PLoS One 2022;17:e0264246.doi:10.1371/journal.pone.0264246Favours DESeq2 at n ≥ 6; tested only to n = 12.
- 08Warden CD, Wu X. Critical differential expression assessment for individual bulk RNA-seq projects. bioRxiv 2024.doi:10.1101/2024.02.10.579728At larger sample sizes standard tests gain utility; assess each project individually.
- 09Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol 2014;15:550.doi:10.1186/s13059-014-0550-8
- 10Robinson MD, McCarthy DJ, Smyth GK. edgeR. Bioinformatics 2010;26:139 to 140.doi:10.1093/bioinformatics/btp616
- 11Ritchie ME, et al. limma powers differential expression analyses. Nucleic Acids Res 2015;43:e47.doi:10.1093/nar/gkv007
- 12Law CW, Chen Y, Shi W, Smyth GK. voom. Genome Biol 2014;15:R29.doi:10.1186/gb-2014-15-2-r29
- 13Lewis P, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS 2020.Origin of the parametric / non-parametric memory distinction.
- 14Clauset A, Shalizi CR, Newman MEJ. Power-law distributions in empirical data. SIAM Review 2009;51:661 to 703.
See these capabilities in a whole run
The work loop applies the same layers to one scientific problem, stage by stage, including the quality-control stage that names a confounder before anything is interpreted.