Cornell University · Preprint 2026

Contrastive Discovery

Open-Ended Scientific Discovery over Competing Explanations

A discovery agent should not just ask is this hypothesis true? It should ask which of the competing explanations does the evidence favor?

Ziang Liu*, James J. Kim*, Yijia Dai, Jennifer J. Sun

Department of Computer Science, Cornell University *Equal contribution

Paper Code BibTeX
Left: a hypothesis generator builds a contrast set of three competing hypotheses from a dataset, and a team of agents (coder, analyst, reviewer, revisor) produces experimental evidence. Right, top: Contrastive Discovery updates belief over all three hypotheses, so H1 is resolved to alternative H2. Right, bottom: pointwise discovery tracks only H1, so H1 is rejected but the alternatives remain unknown.
Contrastive vs. pointwise discovery. Each step poses a scientific question with a contrast set of competing explanations, runs an experiment, and updates belief over all of them. The same evidence that rejects H1 resolves the question to H2. Pointwise discovery only tracks H1: it learns the claim is unlikely, but not what is likely instead.

TL;DR

Most LLM discovery agents are pointwise: they score one hypothesis at a time. Rejecting a hypothesis then says what is unlikely, not which alternative is better supported, so negative evidence is spent without resolving anything. Contrastive Discovery makes the unit of discovery a scientific question together with the mutually exclusive, collectively exhaustive explanations that could answer it, and scores how evidence redistributes belief across them. One experiment can then rule out a hypothesis and identify the better-supported alternative.

01 · How it works

Resolve questions, not hypotheses

We keep the belief-shift principle of surprisal-based discovery but change what belief is held over: a set of explanations for the same question, rather than a single claim and its implicit complement.

1

Pose a question, write out the alternatives

At each step the agent poses a local scientific question and builds a contrast set of K explanations that are mutually exclusive and cover the outcomes the evidence should distinguish (examples below).

2

Elicit belief before and after the experiment

An LLM elicitor allocates 100 units of belief across the alternatives before seeing any evidence, then again after the agent designs and runs its analysis. Five samples are averaged and the alternatives are shuffled to reduce positional bias.

3

Score the shift, read off the outcome

The discovery score is the entropy-scaled belief shift κt = DKL(ppost ‖ pprior) / log2 K, which guides Monte-Carlo tree search. A question is resolved when one explanation clears a posterior threshold: confirmative if it was already leading, corrective if the evidence moved support to a different explanation. Alternatives that fall below a low threshold are ruled out.

What a contrast set looks like

Contrast sets generated by the agent, shown verbatim and before any evidence. Each is one scientific question and the alternatives that could answer it.

DatasetScientific questionCompeting explanations
Materials Project How does magnetic ordering affect bulk modulus?
  1. Significant only for ferromagnetic materials
  2. Significant only for antiferromagnetic materials
  3. Significant for both
  4. Insignificant for both
BLADE Mortgage Given a poor credit score, self-employment status leads to…
  1. Higher denial rates than non-self-employed
  2. No different denial rates
  3. Lower denial rates than non-self-employed
SDSS Galaxy Controlling for galaxy axis ratios, the effect of redshift on stellar velocity dispersion…
  1. Increases (strengthens)
  2. Decreases (attenuates)
  3. Disappears
NLS Raw The trend in vehicle market value across 1985 / 1990 / 1996 is…
  1. Significant linear positive
  2. Significant linear negative
  3. Nonlinear or no significant trend

Pointwise discovery is the K = 2 special case {h, ¬h}: the complement absorbs every alternative, so a rejection cannot say which one gained support.

02 · Results

Pointwise rejections hide contrastive resolutions

Six datasets spanning materials science, astronomy, mortgage discrimination, online discourse, software engineering, and longitudinal social behavior; a shared agent scaffold and the same budget of 200 evidence-generating attempts for every method.

Left: stacked bars of prior belief for pointwise vs. contrastive hypotheses across three elicitor models; pointwise mass concentrates in the maybe-true bin. Right: a flow diagram in which 89% of top pointwise hypotheses are rejected, and under contrastive re-adjudication 73% resolve to an alternative.
Where pointwise surprisal spends its evidence. Left: pointwise proposals collapse onto the “maybe true” prior bin (59–82% across three elicitors), so surprisal mostly surfaces disconfirmations; contrast-set hypotheses spread across prior beliefs. Right: of the top-50 pointwise hypotheses, 89% are rejected. Re-adjudicating the same hypotheses with a post-hoc contrast set and the identical evidence resolves 73% to a specific alternative, and unresolved outcomes drop from 96% to 23%.

More resolved questions at every budget

Line plot of mean number of resolutions versus top-K budget. Contrastive rises to about 162 at K=200; pointwise surprisal, novelty, and feasibility baselines and the confirmative-only contrastive curve all stay near 80 to 98. The shaded region between them is labeled Corrective Gap.
Resolved questions vs. top-K budget (mean over six datasets, ±1 s.e.). Pointwise baselines ranked by surprisal, novelty, or feasibility hit the same ceiling: an isolated hypothesis can only be confirmed or rejected. Contrastive Discovery resolves substantially more, and almost all of the gain is corrective resolutions (shaded), where evidence moved support to another explanation. Resolving more does not sacrifice rejection: at K = 200, 0.91 of contrastive units reach a verdict versus 0.82 of pointwise units.

Where the gain comes from

Three line plots versus top-K budget: resolution rate, normalized realized information gain, and margin gain, for five variants from pointwise through full contrastive. Full contrastive is highest on all three.
Ablating three factors: ranking (pointwise surprisal vs. κt), evaluation (focal hypothesis vs. full contrast set), and when the set is built (post hoc vs. during discovery). Evaluating evidence against the full contrast set is the biggest step. Contrastive ranking helps, but only when paired with contrastive evaluation. Building the alternatives during discovery, so they shape the question and the analysis, is best on all three metrics.

03 · Do the resolutions hold up?

Checked against references the model does not control

A resolution records where one model’s belief settled, so we test it against blinded experts, the raw evidence, other model families, and matched costs.

Blinded human experts agree

38 evaluators with MS or PhD degrees judged 60 questions from the question, alternatives, and evidence alone. The expert majority chose the same explanation as Contrastive Discovery on 78.4% of resolved questions, and on 81.8% of the questions where pointwise discovery had rejected its hypothesis and stopped.

Resolutions follow the evidence

Given only the numerical results for 14 claims with documented findings, the elicitor ranked the documented answer first in all 11 cases where a confidence interval could be recovered. Reversing the sign of the evidence reversed the resolution on all 14 claims; removing the evidence prevented any resolution.

Consistent across model families

Re-eliciting beliefs on 180 contrast sets with gpt-4o, gemini-3-flash, and claude-sonnet-4-6: the three models choose the same leading explanation on 55% of sets before seeing the evidence and on 88.9% after, increasing in every dataset.

The gain survives matched cost

Capping both methods at the same total tokens, Contrastive Discovery resolves 148.3 questions against 87.7; at the same dollar cost, 145.8 against 92.2. The advantage holds at every resolution threshold from 0.60 to 0.95 and in each of the six datasets.

Agentic science should represent discovery not as a stream of accepted or rejected claims, but as resolving among competing explanations.

04 · Citation

BibTeX

@misc{liu2026contrastive,
  title  = {Contrastive Discovery: Open-Ended Scientific Discovery over Competing Explanations},
  author = {Liu, Ziang and Kim, James J. and Dai, Yijia and Sun, Jennifer J.},
  year   = {2026},
  note   = {Preprint}
}