BiG—SURE Paper soon
EMNLP 2026 · Main Conference

Does an LLM’s answer
survive a temperature change?

BiG-SURE estimates uncertainty without model internals. It simply asks whether a model’s low-temperature answers and high-temperature answers still mean the same thing.

See it in action Paper coming soon
LOW T 0.1
HIGH T 1.0
Strong agreement implies high confidence.
Interactive walkthrough

Watch uncertainty emerge.

Two saved examples. One remains certain across temperatures; the other becomes uncertain. Follow the signal from prompt to score.

Choose example
01 Query

What planet is known as the Red Planet?

Greedy answerMars
02a Anchors T = 0.1
02b Probes T = 1.0
03 Semantic agreement
low-T anchor high-T probe light dots · low agreement dark dots · high agreement
04 Spectral energy

Square every connection, then take the mean. Its complement is uncertainty.

Unorm = 1 − Σ Wij2MN
Unorm
normalized uncertainty
Σ
sum over every anchor–probe pair
Wij
semantic agreement between anchor i and probe j
M
low-temperature anchors
N
high-temperature probes
0.00UNCERTAINTY

LOW The model is certain.

Why it works

One score.
Three guarantees.

The normalized measure is bounded in [0, 1] and behaves exactly as a semantic uncertainty estimator should.

≈

Semantic identity

Uncertainty is zero when all sampled responses mean the same thing.

U = 0
≠

Semantic disjointness

Uncertainty is one when the response sets are entirely semantically different.

U = 1
↗

Monotonicity

More semantically unique responses can only increase uncertainty.

ΔU ≥ 0
Results

Stronger abstention,
across modalities.

BiG-SURE is evaluated as an abstention score: can uncertainty separate incorrect answers from correct ones? Higher AUROC is better.

6/7

dataset & language groups where SURE achieves the best average AUROC

Trivia-QAtext · factual
Best prior.722
BiG-SURE.753
SVAMPtext · math
Best prior.846
BiG-SURE.877
OK-VQAvision + text
Best prior.735
BiG-SURE.752
7

dataset / language groups

11

LLM & VLM configurations

3

text, multilingual & vision settings

Refer to the paper for further details.

Sensitivity studies

Stable by design.

The signal persists across practical sampling choices. Explore the two ablations directly from the paper.

Temperature sensitivity grid from the BiG-SURE paper
01 / Temperature

Cross-temperature choices

Performance across low- and high-temperature pairings.
Sample count sensitivity plot from the BiG-SURE paper
02 / Samples

How many probes?

Sensitivity to the number of low- and high-temperature samples.
BibTeX

Cite the work.

Author details and archival links can be inserted here when the paper is de-anonymized.

@inproceedings{bigsure2026,
  title     = {BiG-SURE: Bipartite Graph Spectral Energy for
               Uncertainty and Reliability Estimation of LLMs},
  booktitle = {Proceedings of EMNLP},
  year      = {2026}
}