Mem

Memory systems, readers, and writers.

Study method

What each comparison tests, what stays fixed, and how to read its measures.

Change studyLuna / DeepSeek Flash 1017

Study

13 paired questions · 3 paired writer inputs

Reader · 13 inputs: Thirteen identical saved questions, prompts, and retrieved contexts. Completeness and exact quote binding remain separate checks.
Writer · 3 paired inputs: Three identical English extraction requests. The extra Chinese DeepSeek sample is available as a case and is excluded from this paired table.

Reader completeness and quote binding are separate checks. Writer resource use does not establish a semantic quality ranking.

Inputs, shared settings, and execution environment

Controlled synthetic text fixtures with real provider calls. Saved outputs are historical observations, not simulated scores.
Thirteen byte-identical reader prompts and contexts from the RAGFlow nightly run. No embedding or retrieval rerun.
Luna medium: openai/gpt-6-luna, OpenAI, full 128,000 output allowance. DeepSeek low: deepseek/deepseek-v4.1-flash-1017, DeepSeek, full 393,216 output allowance, no fallback.
The compared profiles use different reasoning effort and output allowances. This is a practical-profile comparison, not equal reasoning compute.
Three paired English extraction requests plus a separate Chinese DeepSeek sample. Eighteen new calls including one calibration; new spend $0.024455124.

What this study can and cannot support

A small diagnostic pass, not a general intelligence ranking or a statistically stable quality order. Repeated questions share facts and are not independent projects.
A matching literal quote proves source binding. It does not prove semantic entailment, complete answers, tenant isolation, or customer acceptance.
Missing or infrastructure-invalid measurements remain unavailable. They are not semantic zeros. Research evidence is neither Calibration nor Official Core evidence.
Retained Luna calls and fresh DeepSeek calls occurred at different times. Mean and median latency differ in direction; neither establishes stable provider speed.
The same two incomplete answers lack STREAM-0-286 in their frozen context. Model replacement did not repair retrieval.
One writer used a Markdown JSON fence that the pinned native RAGFlow parser accepts. Strict raw JSON rejection is not a product failure. Native ingestion and downstream memory behavior were not rerun.

2 profiles · recorded model, version, and settings

Exact configurations in the selected study
Profile	Version or model	Configuration
DeepSeek Flash 1017	`deepseek/deepseek-v4.1-flash-1017`	Settings provider DeepSeek; no fallback effort low outputAllowance 393,216 tokens
Luna medium	`openai/gpt-6-luna`	Settings provider OpenAI effort medium outputAllowance 128,000 tokens control Retained RAGFlow nightly calls