AIQ Overview

Theme preference system; currently light.

Mem

Memory systems, readers, and writers.

  • 3 studies
  • 6 comparisons
  • 189 original files

Study method

What each comparison tests, what stays fixed, and how to read its measures.

Change studyLuna reasoning effort

13 reader + 4 writer inputs · 53 calls including calibration

Reader · 13 inputs
Thirteen identical saved questions, prompts, and retrieved contexts. Completeness and exact quote binding remain separate checks.
Writer · 4 inputs
Four identical extraction inputs, including English and Chinese conditional decisions. All profiles preserve the two tested conditional-scope and owner samples. Subject and effective-time omissions remain.

Reader completeness and quote binding are separate checks. Writer resource use does not establish a semantic quality ranking.

Conditions

Inputs, shared settings, and execution environment
  • Controlled synthetic text fixtures with real provider calls. Saved outputs are historical observations, not simulated scores.
  • GPT-6 Luna, OpenAI provider, no fallback. Effort is the only explicit model parameter changed; temperature is omitted.
  • Full 128,000 output allowance for every call; every response ended normally. Thirteen identical reader inputs and four extraction inputs per profile, plus two separate calibrations.
  • Three blocks rotate profile order. Actual cache-read and cache-write tokens remain in the original metrics. Reader contexts use the retained Qwen3-Embedding-8B 4096-dimensional retrieval.
  • Fifty-three calls; new settled spend $0.016351425. Reader and writer costs in the tables exclude the two calibrations.

Limits

What this study can and cannot support
  • A small diagnostic pass, not a general intelligence ranking or a statistically stable quality order. Repeated questions share facts and are not independent projects.
  • A matching literal quote proves source binding. It does not prove semantic entailment, complete answers, tenant isolation, or customer acceptance.
  • Missing or infrastructure-invalid measurements remain unavailable. They are not semantic zeros. Research evidence is neither Calibration nor Official Core evidence.
  • No native storage, reconciliation, retrieval, or final Agent tool loop was rerun.
  • None saved about $0.04 per thousand reader answers in this workload. Four writer samples do not justify a routing policy. No default configuration changed.
  • The original report suggested exploring none for routine extraction. The later interpretation shown here retains one medium default. The original report remains byte-identical.

Compared versions

3 profiles · recorded model, version, and settings
Exact configurations in the selected study
Profile Version or modelConfiguration
Luna lowopenai/gpt-6-luna
Settings
provider
OpenAI; no fallback
effort
low
outputAllowance
128,000 tokens
Luna mediumopenai/gpt-6-luna
Settings
provider
OpenAI; no fallback
effort
medium
outputAllowance
128,000 tokens
Luna noneopenai/gpt-6-luna
Settings
provider
OpenAI; no fallback
effort
none
outputAllowance
128,000 tokens