# DeepSeek V4.1 Flash 1017: bounded Luna comparison

Date: 2026-10-20. Status: completed. Scope: model replay with fixed retrieved evidence and memory-extraction requests.

Keep GPT-6-Luna as the current default. In this small test, DeepSeek V4.1 Flash 1017 matches Luna on complete answers, costs more, and has mixed answer latency. Its memory extraction is faster in three paired requests and improves some effective-date fields, but it still omits the project from some independent memory atoms. This does not establish a reason to change the default or a repair for retrieval.

## Version and controls

- Model: `deepseek/deepseek-v4.1-flash-1017`, the [October 17 release](https://api-docs.deepseek.com/news/news261017/).
- OpenRouter routing was pinned to `deepseek`, with no provider fallback. Public [endpoint metadata](endpoints.json) was saved before execution.
- DeepSeek used low reasoning and its full 393,216 output-token allowance. The retained Luna control used medium reasoning and its full 128,000 allowance. This compares practical profiles, not equal reasoning compute. All eighteen new responses ended normally with `finish_reason: stop`.
- The thirteen reader requests reuse exact saved prompts and evidence from the RAGFlow nightly test. Those contexts came from Qwen3-Embedding-8B full 4096-dimensional indexes. No new embedding or retrieval calls were made.
- Three writer requests reuse the native RAGFlow extraction prompts and inputs byte for byte. One additional Chinese input was frozen before calls and has no Luna control.
- Calibration was separate. No scored answer was retried, repaired, or replaced. The source oracle, contexts, and prompts were not tuned after the formal run began.
- All new calls passed through `cargo make benchmark-budget` and the existing cumulative $10 ledger. A $0.75 tranche and full output reservations controlled admission.

See [frozen protocol](protocol.json), [cases and retained controls](cases.json), [reader oracle](oracle.json), and [calibration result](calibration-result.json).

## Fixed-context answers

| Measure, same thirteen questions and contexts | Retained GPT-6-Luna | DeepSeek V4.1 Flash 1017 |
| --- | ---: | ---: |
| Complete semantic answers | 11/13 | 11/13 |
| Complete answers with bound quotes | 11/13 | 11/13 |
| Individual quotes match supplied text | 28/28 | 35/35 |
| Mean model-request time | 4.54 s | 5.05 s |
| Median model-request time | 4.11 s | 3.63 s |
| Settled cost for these thirteen calls | $0.004811275 | $0.019369416 |
| Completion tokens, including reasoning | 4,034 | 10,708 |
| Reasoning tokens | 1,198 | 7,293 |

DeepSeek's reader cost is about 4.0 times Luna's in this run. The absolute difference is about $0.0146 for thirteen questions. Its median is lower, but two incomplete questions take 11.59 and 12.08 seconds and raise the mean. These are single observations at different request times; they do not establish a reliable provider speed ranking. Token counts also use different model tokenizers. The measurements are whole non-streaming request times, not time to first token.

Both models miss the same Larch current-decision fact, `STREAM-0-286`, in `post-memory-0-0` and `post-combined-001`. That fact is absent from the frozen returned contexts. DeepSeek discusses the earlier BATCH decision and the missing August 15 choice, but cannot establish the requested current delivery mode. The other eleven answers retain the requested subjects, historical states, recovery causes, policy exceptions, and unknown-owner boundaries.

This is a model comparison with fixed retrieved evidence. It is not a new end-to-end RAGFlow score. A valid literal quote is not proof that every claim is complete or entailed. See [manual semantic review](reader-review.json) and [metrics](metrics.json).

## Memory extraction

| Measure, three identical English writer inputs | Retained GPT-6-Luna | DeepSeek V4.1 Flash 1017 |
| --- | ---: | ---: |
| Mean model-request time | 8.31 s | 4.57 s |
| Median model-request time | 7.43 s | 5.08 s |
| Settled writer cost | $0.001142100 | $0.002913504 |
| Returned memory atoms across selected types | 9 | 17 |

DeepSeek explicitly retains the Larch recovery instruction and preserves the ORBIT audit condition, staging boundary, Mira's ownership, and lack of production authority. In the Larch retry update, it uses August 22 for the effective date instead of Luna's October capture date.

The root problem remains visible: several Larch recovery, retry, and owner atoms lack the project name. More atoms also repeat information across episodic, semantic, and procedural types. That can increase storage, embedding, and retrieval work; the downstream effect was not measured here. The extra Chinese sample preserves the project identifiers, audit condition, environment, and owner, but still uses capture time as a precise effective timestamp.

One writer response wraps valid JSON in a Markdown fence. Direct `json.loads` accepts three of four responses. The pinned RAGFlow [native extraction parser](https://github.com/infiniflow/ragflow/blob/7eb673bfc3ab1cc22532fe23a1e9bb85c34e48bd/internal/service/memory_extractor.go#L657) already removes that fence. All four outputs have its expected array/object shape after the same normalization. This is source-backed compatibility evidence; native ingestion, storage, deduplication, and later retrieval were not rerun. The raw response and strict JSON result remain unchanged. See [writer review](writer-review.json).

## Cost, cancellation, and retained state

- New requests: **18**, all completed: one calibration, thirteen readers, three paired writers, and one Chinese writer.
- New tokens: **26,675 prompt + 14,275 completion = 40,950**. Provider-reported cached input: **2,304 tokens**. Settled cost includes the actual cache discount.
- New settled spend: **$0.024455124**.
- Cumulative settled spend: **$2.049098675**.
- Cumulative conservative exposure, including earlier reservations: **$2.121279375 / $10**.
- Both budget gateway processes exited successfully. No new request is unresolved. See [new request ledger](request-ledger.json).

The user cancelled the local Hy-Memory trial. Its download was stopped before inference began. The task-owned directory, portable runtime, download cache, and partial model were deleted, releasing about **1.18 GB of allocated files**. Existing model caches were preserved. There was no paid inference for Hy-Memory. See [cleanup receipt](hy-cleanup.json).

No tracked ELF source or default model configuration was changed. No Docker service or local model server was started for this comparison. The report and raw evidence remain local under ignored `tmp/`.