How we measure AI answers, and prove we never revised them

1. The panel

• Fixed wording. Each question is asked word for word every day, in English, from the United States, with no follow-ups. The panel is versioned; the current version is v2 (effective 2026-10-20).
• Engines. ChatGPT with web search, Google AI Mode, Gemini and Perplexity (Sonar), through licensed DataForSEO APIs, the same sources our public tools use.
• Search demand alongside. 88 keywords (monthly Google volume, CPC, difficulty) and 30 Google results pages (top 10 and whether an AI Overview appears) are collected the same morning.
• Budget rule. A hard daily spending cap stops collection rather than exceed it; questions are collected in a fixed priority order, so a short day loses the lowest-priority questions first, and the day's record says so.

2. What we record per answer

• The full answer text, model name and timestamp
• Every cited URL and domain
• Brands or venues named, in order of first mention
• For ticker-mapped questions, the public listing of each tracked brand (as of the panel version; private companies are marked as such)

Names are matched against a fixed list per question plus any other proper names the engine returns; matching rules are versioned with the panel.

3. AI Nightlife Index

The AI Nightlife Index aggregates the California venue questions by city. For each venue we report: the share of answers that name it (across engines and days), its average position when named, how many engines name it, and which sources the engines cite for it. Editions are monthly; the first covers October 20, 2026 through November 15 and publishes Monday, November 16, 2026. Venues named fewer than three times in a month are not ranked. We never rank a venue on a single answer, and we do not accept payment for placement.

4. Provably unrevised history

Right after collection each day, the folder is sealed:

1. SHA-256 of every file, including a frozen copy of that day's question list and engine settings
2. A Merkle root over those hashes, and a link to the previous day (prev_manifest_sha256), forming a hash chain: a day cannot be changed, removed or inserted without breaking every later day
3. An Ed25519 signature over the manifest
4. An OpenTimestamps proof that anchors the manifest in the Bitcoin blockchain, so even we cannot backdate it

Only the proofs are published, in the public repository wrenfielddata/wrenfield-history-proofs, together with a verifier (the first upload of the proofs sealed so far is pending; they are timestamped already, so publishing later does not weaken them). The data stays private; anyone we share a day's files with can confirm they match. Sealed so far: 1 day(s), from 2026-10-19. The first day (Oct 19, 2026) was sealed a few hours after collection; every later day is sealed minutes after.

pip install cryptography opentimestamps-client
python3 verify.py proofs/2026/2026-10-19 --data ./2026-10-19
ots verify proofs/2026/2026-10-19/manifest.json.ots

5. Limits

• AI answers vary between runs, users and locations. One day is a snapshot; we publish trends across many days.
• API answers can differ from the consumer apps. We state the engine and model with every number.
• Our questions are phrased by us. Real users ask in many ways, so the index measures how engines answer common phrasings, not total exposure.
• A timestamp proves when data existed and that it has not changed since, not that it is correct.

FAQ