# Gemini Decision 1: official API experiment

Run date: 2026-11-12. All inference uses Google directly, not OpenRouter.
Jev inference was not run. No credential is included in these files.

Python 3 standard library only. Download the scripts and JSON/JSONL files together.

```sh
python3 analyze.py                 # reproduce tables offline
python3 benchmark.py --dry-run     # inspect 24 tickets, questions, and schedule
# Set GEMINI_API_KEY in your own shell, then:
python3 benchmark.py --out decision-results
python3 ablations.py --out decision-probes
# Only after investigating an interruption, explicitly resume saved successful jobs:
python3 ablations.py --out decision-probes --resume
```

The main experiment has 24 manually authored synthetic tickets, 2 questions each,
and 2 repeats: 48 requests. Labels were fixed before inference and never sent to
Google. The shuffled schedule uses seed 20261112. The urgency threshold is 0.5.
Both repeats are the same 24 independent cases, not 48 independent cases.

The supplemental script has 37 successful requests:

- Shared-state batches: 1/10/100 questions, each repeated 3 times (9 requests).
  The 100-question variant repeats the same 10 questions 10 times. It measures
  request scaling, not 100 different tasks or hard independent reasoning cases.
- The same 10 checks sent separately once each (10 requests, sequential).
- Initial missing-option probes (6 requests): both the schema and its related
  instruction changed. These are pilot data, not the article's controlled result.
- Clean vs injected ticket, same questions, 3 repeats each (6 requests).
- Confirmation of missing-option effect with identical instructions and only
  the unknown criterion changed, 3 repeats each (6 requests).

The first supplemental run saved 23 successes and then stopped on SSL
UNEXPECTED_EOF_WHILE_READING at option-False repeat 2. That failure was transcribed
from the terminal into failures.jsonl because the initial runner did not log
failures. The runner was then updated to log failures and support explicit resume;
the remaining jobs were completed. The matched-option confirmation was added
after identifying the pilot confound. There are 85 saved successful responses
(48 main + 37 supplemental), plus 1 failed attempt. A separate 1-request smoke
check preceded these runs and is excluded from tables.

Each urllib call opens its own connection. Wall-clock latency includes connection,
network, service, and response parsing. It is not model-only latency. The 10
separate checks were run once; batch sizes were measured three times, with order
1/10/100 then 100/10/1 then 1/10/100. This is a small observational comparison,
not a controlled network benchmark. No model/provider fallback or automatic retry.

The API reports token usage, not the project's bill or Free/Paid tier. Cost
calculations in the article use listed paid pricing, not an observed charge.
The experiment does not establish calibration, prompt-injection safety, general
accuracy, multimodal quality, long-context performance, or superiority to Jev.
