1Technical University of Denmark 2Pioneer Center for AI 3University of Copenhagen
TL;DR Before answering an unlabeled test question, a vision-language model imagines similar practice problems: it writes each question and its answer first, then creates an image that fits. It learns from them and then answers. No labels. No external models.
The key idea
Test-time self-improvement usually learns from the model's own answers, such as majority votes or self-judgments. When the model is wrong, it teaches itself to stay wrong. SIMIT turns this around. The model writes a new question, decides the answer first, and only then creates an image that makes the answer true. Solving an unfamiliar question is hard. Writing a practice problem whose answer you already know is much easier.
Q: What color is this cat?
Consensus-based self-training, e.g. TTRL
On VizWiz, “unanswerable” is TTRL's majority vote in 99.2% of training steps.
Sampled answers are illustrative.
SIMIT: the same model, no labels
Q: Is the cat sitting or lying down?
A: Sitting
Q: How many legs does the cat have?
A: Four
Answer fixed first, image generated to match it.
Learn from them in context, then answer: Brown
A model does not need to solve the test query correctly to write useful practice problems, because it controls the answers it writes.
How it works
SIMIT runs on a unified multimodal model (BAGEL-7B) that can both understand and generate images. No teacher, retriever or verifier is used. The only other components are deterministic renderers, which draw whatever the model specifies.
Synthesizerwrites question, answer and image-description triplets
Routerdecides how each image should be made
Artistpaints natural images natively
Architectwrites specs that renderers draw exactly
Criticchecks that each image supports its answer
Solverlearns from the samples, then answersSwipe the figure sideways to explore it.
Imagined samples become few-shot demonstrations. No weight access needed, so it also fits API-only models.
Fine-tune on one query's samples, answer it, then reset. Nothing carries over between queries.
Pool the samples from every test query and fine-tune once, so practice for one query helps the others.
Fine-tune on the pooled samples, then also show each query its own samples in context. Best overall.
Mean relative gain over zero-shot BAGEL-7B across 17 benchmarks.
Beyond natural images
Unified models can paint photos, but they garble text, numbers and structure. For structured visuals, SIMIT gives the model a skill library. The model writes a compact specification, and a deterministic renderer draws it exactly. Drag the slider to compare.
Click a render to see the prompt that produced it.
Try it
Ask any question about any image in the online demo, or add SIMIT to your own code with one pip install.
Upload an image, ask a question, and compare the base model's answer with SIMIT's self-improved one. Choose BAGEL-7B, Lance or Qwen3.8-27B. Free on Hugging Face ZeroGPU.
Imagine practice problems and answer with them in a few lines. Works with BAGEL, Lance and any Hugging Face vision-language model.
pip install simit
from simit import SIMIT
model = SIMIT.from_pretrained(
"ByteDance-Seed/BAGEL-7B-MoT")
# imagine practice problems, then use them
demos = model.imagine(image, question)
answer = model.answer(image, question, demos)
Results
Compared with test-time scaling and label-free reinforcement learning on the same BAGEL-7B model, across 17 benchmarks in five domains. Extra compute does not explain the gap. SIMIT-ICL uses 25× the zero-shot runtime, while label-free RL methods that run longer reach at most +2.07%.
Each method at its best configuration. The right column shows end-to-end runtime relative to zero-shot. Hover a row for per-domain scores.
Relative improvement over BAGEL-7B. Hover a bar for raw scores.
Why does it work?
Consensus rewards mostly sharpen answers the majority already gets right. Imagined supervision is created independently of the model's prediction on the test query, so it can correct errors.
Test queries are split by whether Self-Consistency already answers correctly. The chart shows how often each method moves them.
Supervision each method encourages on queries the base model gets wrong, as judged by a blind external evaluator.
SIMIT corrects 14.4% of initially wrong answers while breaking only 2.0% of correct ones. It is the only method whose supervision on hard queries is more often correct than incorrect. Its overall supervision-error rate is about 12%, against 33–38% for label-free RL. δ is the gap in supervision-error rate between queries the base model gets wrong and those it gets right. A large positive δ means errors pile up exactly where the model is already weak.
Explore
22 real test queries from the paper (Fig. 4 and Appendix N). For each one, compare the practice problems SIMIT imagined with the human-labeled neighbors that retrieval baselines use, and see who answered correctly. All results are precomputed, so no model runs in your browser.
RICES and TTT-NN retrieve nearest neighbors from a labeled pool, which SIMIT never uses. For compact display these examples disable adaptive budgeting and difficulty filtering and use K = 2 samples. A ✓ follows the paper's per-example judgment.
Citation
@article{kaya2026simit,
title = {{SIMIT}: Self-Improving Vision-Language Models via Imagination at Test-Time},
author = {Kaya, Mehmet Onurcan and Elliott, Desmond and Papadopoulos, Dim P.},
journal = {arXiv preprint arXiv:XXXX.XXXXX},
year = {2026}
}