test result changed spec

This commit is contained in:
nvaldez
2026-10-05 07:30:51 +02:00
committed by nvaldez
parent 4c84213a30
commit 9ff5af588a
2 changed files with 223 additions and 14 deletions
+143 -13
View File
@@ -1,6 +1,6 @@
---
name: Fiona-Assistant Specification
version: 1.7.0
version: 1.8.0
title: Project Spec Sheet
created: 2026-09-23
updated: 2026-10-02
@@ -128,7 +128,16 @@ gives that file frontmatter `name` and `version` (*7.3.1.1.1.*)
so `fmlint` accepts it. Revision 1.6.0 specifies `librarian move`
(*7.3.3.*) and leaves it unbuilt. Revision 1.7.0 makes that
command rewrite links to the moved paths across the whole wiki
(*7.3.3.6.*).
(*7.3.3.6.*). Revision 1.8.0 adds the findings of the 2026-10-02
manual test and leaves them unbuilt: one-line CLI errors with no
traceback (*1.2.7.*, *7.1.1.4.*), `--version` (*2.3.*), a quiet
default stderr (*7.17.3.*), an ingest hint for a drop-zone file that
already moved (*7.1.5.2.7.*), a dry run that does not log writes
(*7.1.5.8.1.*), a lint-clean ingest log line (*7.1.5.6.1.*), a
follow-up hint for style violations left on a page (*7.1.5.5.6.*),
single-file `mdoctor` (*7.4.4.*), compact housekeeping sanity details
(*7.2.5.1.*), a quiet graph overwrite (*7.1.4.5.9.1.*), and a usage
section in the README (*7.18.4.*).
Each feature section starts with a `Status` line. Requirements use
`- *N.M.K.*`. Each `*acceptance*` sits under the requirement it tests.
@@ -203,12 +212,18 @@ These apply to every feature.
A position read from the layout engine is written back in `rem`.
A theme length number uses the same scale: 16 is `1em`.
- *acceptance*: The graph template contains no `px` length.
- *1.2.7.* `NOT_IMPLEMENTED` An expected failure, such as a missing
path, a refusal, or bad input, prints one `Error: <message>` line on
stderr and exits 1. It prints no Python traceback. A traceback is
logged only at `DEBUG`.
- *acceptance*: A second `wiki init` on the same directory writes
no line containing `Traceback` on stderr (*7.1.1.4.*).
---
## 2. Package and Runtime
> Status: Implemented
> Status: Partially Implemented
Python `>= 3.11` (also 3.12 and 3.13). The default development
install from the repository root is `pip install -e ".[dev]"`.
@@ -273,6 +288,12 @@ pymarkdown resets the root logger mid-run.
exists.
- *acceptance*: A fresh install does not fail at import because of
a missing `html_js_lint` package.
- *2.3.* `NOT_IMPLEMENTED` `fiona-assistant --version` prints
`fiona-assistant <package version>` on stdout and exits 0. The text
is the same string `wiki ingest` writes to `ingest.tool`
(*7.1.5.5.3.*).
- *acceptance*: The stdout of `--version` equals the `ingest.tool`
value of a page ingested by the same install.
---
@@ -537,6 +558,10 @@ A pymarkdown API failure prints
`Error: pymarkdown fix failed: <error>` on stderr and exits 1.
The command does not skip `personal/` or `raw/`.
`NOT_IMPLEMENTED` `chklint` and `fix` also take one markdown file in
place of `[directory]` (*7.4.4.*). Today a file prints
`Error: '<path>' is not a valid directory.` and exits 1.
### 4.3. fmlint
`fiona-assistant fmlint` with no subcommand means `chklint .`.
@@ -798,7 +823,7 @@ section 7.14.
`fiona-assistant wiki graph [directory] [-o FILE] [--format html|json]
[--include-pattern GLOB] [--exclude-pattern GLOB]
[--wikilinks|--no-wikilinks] [--json]
[--external-links|--no-external-links]
[--external-links|--no-external-links] [-f|--force]
[--serve [--host HOST] [--port PORT] [--no-open]]`.
Defaults: output `wiki_graph.html` in the current directory, format
@@ -809,7 +834,9 @@ A path in `fiona.wiki.graph.ignore_files` is not a node
also matches `table_of_contents.md` at the scan root.
`NOT_IMPLEMENTED` `--serve` runs a local live view instead of
writing once (*7.1.4.19.*). `--host`, `--port`, and `--no-open`
are accepted only with `--serve`.
are accepted only with `--serve`. `NOT_IMPLEMENTED` `-f/--force`
overwrites an existing output file without the prompt of
*7.1.4.5.9.* (*7.1.4.5.9.1.*).
#### 4.7.6. wiki ingest
@@ -895,6 +922,7 @@ Errors, each on stderr with exit 1:
```text
Error: Source not found: <path>
Error: Source not found: <path> (moved to <raw path> by an earlier ingest)
Error: Source is a directory: <path>
Error: Unsupported URL scheme: <scheme>
Error: Download failed: <url>: <reason>
@@ -908,6 +936,7 @@ Error: Page exists: <path>. Use --force to replace it.
A failed conversion prints `Error: Conversion failed: <converter>:
<reason>` on stderr and exits 3, not 1, because the raw copy stays.
`NOT_IMPLEMENTED` The `(moved to ...)` form is *7.1.5.2.7.*.
### 4.8. housekeeping
@@ -1590,7 +1619,7 @@ Requirements for the `wiki` subcommands.
#### 7.1.1. wiki init
> Status: Implemented
> Status: Partially Implemented
Implements sections 3 and 4.7.1.
@@ -1610,6 +1639,14 @@ Implements sections 3 and 4.7.1.
- *acceptance*: After init, `.gitignore` contains `__pycache__/`
and `personal/`. A line `keep-me` written before init is still
in the file.
- *7.1.1.4.* `NOT_IMPLEMENTED` The CLI turns the refusal of *3.2.2.*
into one stderr line and exit 1 (*1.2.7.*):
`Error: Directory '<path>' already contains llm-wiki files:
<names>. Run fiona-assistant wiki update to refresh the template.`
The line is one line, even though it is wrapped here. `<names>` is
the comma-separated list from the `FileExistsError` message.
- *acceptance*: A second `wiki init` exits 1, stderr is exactly that
line, and no file changes.
#### 7.1.2. wiki update
@@ -1967,6 +2004,15 @@ used.
or hides orphans. It starts checked. The label does not change.
- *7.1.4.5.9.* Overwrite of an existing output file asks for
confirmation on a TTY.
- *7.1.4.5.9.1.* `NOT_IMPLEMENTED` `-f/--force` skips that
prompt. Without a TTY and without `--force`, the file is still
overwritten and one `WARNING` record names it. The housekeeping
graph step (*7.2.6.*) passes `--force`, because
`wiki_graph.html` is the one file it owns (*7.2.7.*), and logs
that overwrite at `INFO`.
- *acceptance*: Housekeeping on a wiki that already has
`wiki_graph.html`, at `LOG_LEVEL=WARNING`, writes no
`Overwriting` line on stderr.
- *7.1.4.5.10.* Files above `max_files` are skipped after a warning.
The first `max_files` files are drawn.
- *7.1.4.5.11.* The run completes in under 10 seconds for 1000 files
@@ -2484,7 +2530,7 @@ used.
#### 7.1.5. wiki ingest
> Status: Implemented
> Status: Partially Implemented
Implements section 4.7.6. An ingest has two phases. The tool runs
the first: it turns one source into a file on disk, archives a copy
@@ -2579,6 +2625,17 @@ Candidates for later formats are in Appendix C.
- *acceptance*:
`https://host/job/performance_TL153/2108/console` has the slug
`performance-tl153-2108-console`.
- *7.1.5.2.7.* `NOT_IMPLEMENTED` A local path is resolved against
the current directory. `./`, `.\`, and either separator are
accepted on Windows. A missing path inside the wiki's `ingest/`
is often a file an earlier run already moved (*7.1.5.3.6.*).
When a file with the same name exists under `raw/`, the error
adds one space and `(moved to <raw path> by an earlier ingest)`.
Several matches use the first in sorted path order.
- *acceptance*: Ingesting `./ingest/a.md` and then
`.\ingest\a.md` exits 0, then exits 1 with
`Error: Source not found: .\ingest\a.md (moved to
raw/articles/a.md by an earlier ingest)` on one line.
- *7.1.5.3.* Archive.
- *7.1.5.3.1.* The default raw path is
`raw/<category>/<slug>.<ext>`. The category is the raw category
@@ -2771,6 +2828,14 @@ Candidates for later formats are in Appendix C.
`<N> markdown style violations left`.
- *acceptance*: Every fixture page passes fmlint with no
violation.
- *7.1.5.5.6.* `NOT_IMPLEMENTED` When violations are left, human
stdout prints, before the `Next: refine` line,
`Next: fix the style violations listed by fiona-assistant mdoctor chklint <page path>.`
The page must pass `wiki sanity` before it is refined. The JSON
`warnings` text does not change.
- *acceptance*: An HTML fixture with a bare URL that the fixer
leaves prints that line with the page path. A clean fixture
does not.
- *7.1.5.6.* Record. A run that archives appends one
bullet to `wiki/log.md` and bumps that file's patch `version` and
`updated`. The bullet is
@@ -2779,6 +2844,22 @@ Candidates for later formats are in Appendix C.
does not edit `wiki/index.md` or any other page.
- *acceptance*: One successful run adds one line to `log.md` and
leaves `index.md` unchanged.
- *7.1.5.6.1.* `NOT_IMPLEMENTED` The bullet passes the wiki's own
markdownlint. The origin is a code span, so a URL is not a bare
URL (MD034). Each `->` part starts a continuation line indented
two spaces, so no line is longer than the bullet's longest path:
```text
- <YYYY-MM-DD> - ingest `<origin>`
-> <raw path>
-> <page path> (raw)
```
The bullet still counts as one entry for *3.3.4.* and the first
acceptance above.
- *acceptance*: After ingesting a URL of 90 characters into a
fresh wiki, `wiki sanity` reports no markdownlint hit in
`wiki/log.md`.
- *7.1.5.7.* `--raw-only` runs fetch, archive, and
record, and skips convert and write. It exits 0.
- *acceptance*: No page is written and `conversion` is `none`.
@@ -2790,6 +2871,13 @@ Candidates for later formats are in Appendix C.
true in JSON.
- *acceptance*: File hashes under the wiki match before and after
a dry run, and an `ingest/` source is still in place.
- *7.1.5.8.1.* `NOT_IMPLEMENTED` A dry run logs no record that says
a file was archived, moved, or written. Those records read
`dry run: would archive <raw path>` and
`dry run: would write <page path>`.
- *acceptance*: A dry run at `LOG_LEVEL=INFO` has `would archive`
on stderr and no line that starts with `archived` after the
logger name.
- *7.1.5.9.* `--pending` lists every page under
`wiki/` whose frontmatter `status` is `raw`, sorted by path, and
exits 0.
@@ -2826,7 +2914,7 @@ Candidates for later formats are in Appendix C.
### 7.2. housekeeping
> Status: Implemented
> Status: Partially Implemented
Implements section 4.8. Housekeeping is the routine an LLM runs to
maintain a wiki. It gathers what needs work; the LLM does the work.
@@ -2883,6 +2971,13 @@ Paths in details and actions are relative to the wiki root.
`Fix the violations, then run fiona-assistant housekeeping again.`
- *acceptance*: One dead link makes the step `FAIL` and a detail
line names the file.
- *7.2.5.1.* `NOT_IMPLEMENTED` The details are one line per
violation, `<check>: <path>:<line>:<column> <code> <message>`,
with the path relative to the wiki root. Report headers,
DEFINITIONS, and SUMMARY lines are not details. The full report
stays with `wiki sanity`.
- *acceptance*: A wiki with three MD013 hits and one dead link
gives exactly four `sanity` detail lines.
- *7.2.6.* `graph` writes `<wiki root>/wiki_graph.html` with the
defaults of section 4.7.5 and the wiki's graph settings. The summary
is `wrote wiki_graph.html (<nodes> nodes, <edges> edges)`. A
@@ -3150,7 +3245,7 @@ in any case.
### 7.4. mdoctor
> Status: Implemented
> Status: Partially Implemented
Implements section 4.2 and the pymarkdown keys in section 5.1.
Settings load at construction. The default settings path is the
@@ -3175,6 +3270,12 @@ pymarkdown defaults.
- *acceptance*: `scan_path` is called with `respect_gitignore`
false. Inside the working directory the exclude list contains
`.git/` and each plain directory name from `.gitignore`.
- *7.4.4.* `NOT_IMPLEMENTED` `chklint` and `fix` accept one markdown
file as well as a directory, the same as `fmlint chklint`. A path
that does not exist prints `Error: '<path>' does not exist.` on
stderr and exits 1.
- *acceptance*: `mdoctor fix wiki/sources/a.md` lists only that
file, and every other file hash is unchanged.
### 7.5. fmlint
@@ -3463,7 +3564,7 @@ on an existing wiki. It does not create the wiki tree.
### 7.17. Logging
> Status: Implemented
> Status: Partially Implemented
The contract is section 2. Reports stay on stdout. Diagnostics go
through the logging module to stderr and, when configured, to the
@@ -3474,6 +3575,16 @@ file sink.
the human report stays on stdout.
- *7.17.2.* Dropping `file_log_location` leaves no file handler.
- *acceptance*: No log file is created for that run.
- *7.17.3.* `NOT_IMPLEMENTED` The shipped `fiona-settings.json` sets
`fiona.log_level` to `WARNING`, the same as the default in
section 5.1. `fiona.file_log_level` stays `INFO`, so `fiona.log`
keeps the detail. PowerShell shows every native stderr line as a
`NativeCommandError`, so an `INFO` record looks like a failure
even when the exit code is 0. When this is built, the shipped
document in section 5.1 changes to match.
- *acceptance*: In a fresh wiki, `wiki sanity` on the clean
template and `wiki ingest` of a local markdown file write nothing
on stderr, and `fiona.log` has their `INFO` lines.
### 7.18. Documentation
@@ -3495,6 +3606,17 @@ are behind the current CLIs.
- *7.18.3.* Documentation markdown passes `mdoctor chklint`.
- *acceptance*: `README.md` contains a markdown link to `SPEC.md`
and the lint report is clean.
- *7.18.4.* `NOT_IMPLEMENTED` The README Quick Start has a short
"After ingest" and "On Windows" part. It says that a file in
`ingest/` moves into `raw/` and a second run on it fails, that
`LOG_LEVEL=ERROR` hides diagnostics until *7.17.3.* ships, the loop
`housekeeping`, then fix style violations, then refine and set
`status: refined`, that `--version` (*2.3.*) or `pip show
fiona-assistant` gives the installed version, and that commands
run from the wiki root until section 7.20 is built. The Fiona skill
carries the same loop.
- *acceptance*: Each of those five points appears in `README.md`,
and the after-ingest loop appears in the Fiona skill.
### 7.19. Tool Roles
@@ -3607,11 +3729,14 @@ is the pip-argument source both scripts call.
| P1 | wiki update / check-update | 7.1.2 | Implemented |
| P1 | Git save, snapshot, restore, clean, undo, status | 7.12 | Implemented |
| P1 | wiki sanity, including dead links | 7.1.3 | Implemented |
| P1 | housekeeping | 7.2 | Implemented |
| P1 | housekeeping | 7.2 | Partially Implemented |
| P1 | wiki init | 7.1.1 | Partially Implemented |
| P1 | CLI errors without traceback | 1.2.7 | Planned |
| P1 | wiki graph | 7.1.4 | Partially Implemented |
| P2 | fmlint | 7.5 | Implemented |
| P2 | json-lint | 7.11 | Implemented |
| P2 | mdoctor fix | 7.4 | Implemented |
| P2 | mdoctor fix | 7.4 | Partially Implemented |
| P2 | Logging | 7.17 | Partially Implemented |
| P2 | Settings schema | 5 | Implemented |
| P2 | Sanity reporting, coverage, progress | 7.14 | Implemented |
| P2 | Shellcheck | 7.9 | Planned |
@@ -3621,7 +3746,8 @@ is the pip-argument source both scripts call.
| P2 | Wiki root discovery | 7.20 | Planned |
| P2 | settings setup | 7.16 | Planned |
| P2 | Bootstrap install modes | 7.21 | Implemented |
| P2 | wiki ingest | 7.1.5 | Implemented |
| P2 | wiki ingest | 7.1.5 | Partially Implemented |
| P3 | `--version` | 2.3 | Planned |
| P3 | fmlint enforcement of quality signals | 6.2 | Planned |
| P3 | Backup archive | 7.13 | Planned |
| P3 | Documentation refresh | 7.18 | Partially Implemented |
@@ -3671,6 +3797,10 @@ is the pip-argument source both scripts call.
- `librarian move` (*7.3.3.4.4.*) does not rewrite links in HTML
tags (`<a href>`, `<img src>`). Whether it should is not
decided.
- `git undo` and `git clean` discard work with no prompt. The
2026-10-02 manual test lost a `git save` commit this way. A TTY
prompt with `--yes` (*1.2.2.*) would make agents pass `--yes` on
every call. Not decided.
---
+80 -1
View File
@@ -3,6 +3,7 @@
- [Done](#done)
- [P0 — Packaging and Structure Fixes](#p0--packaging-and-structure-fixes)
- [Code defects — 2026-09-29 audit](#code-defects--2026-09-29-audit)
- [Manual test findings — 2026-10-02](#manual-test-findings--2026-10-02)
- [P1 — Wiki Lifecycle and Version Control](#p1--wiki-lifecycle-and-version-control)
- [P2 — Quality Tooling](#p2--quality-tooling)
- [P3 — Metadata, Backup, and Documentation](#p3--metadata-backup-and-documentation)
@@ -348,6 +349,81 @@ items below are done. The command defects after them are still open.
`main()`, which parses the arguments a second time. Call the
library functions directly so each command is parsed once.
## Manual test findings — 2026-10-02
Black-box test from the README and SPEC only. Report:
`tests/wikitest/wiki/queries/fiona-assistant-manual-test-2026-10-02.md`.
Each item below was re-run on 2026-10-02 before it was written down.
SPEC 1.8.0 holds the requirements. Order is the build order.
- [ ] Second `wiki init` prints a Python traceback (SPEC 1.2.7,
7.1.1.4). Reproduced: the `FileExistsError` reaches the top
level. Catch it in the `wiki init` CLI handler, print the one
`Error:` line on stderr, exit 1. Then sweep the other handlers
in `src/fiona/__main__.py` for expected errors that escape
- [ ] Test: second init, stderr has no `Traceback`, exit 1, file
hashes unchanged
- [ ] Shipped `fiona.log_level` is `INFO`, the code default is
`WARNING` (SPEC 7.17.3). Every INFO record shows as a red
`NativeCommandError` in PowerShell even on exit 0. Ship
`WARNING`, keep `file_log_level` `INFO`, and update the shipped
document in SPEC 5.1
- [ ] Test: fresh wiki, clean `wiki sanity` and a local
`wiki ingest` write nothing on stderr; `fiona.log` has INFO
- [ ] Existing wikis get this through `wiki update` as a
`fiona-settings-update.json` conflict. Check the housekeeping
merge action is clear enough for that one key
- [ ] Ingest log bullet fails the wiki's own lint (SPEC 7.1.5.6.1).
Reproduced in the test wiki: `wiki/log.md` has MD013 (112 and
156 chars) and MD034 (bare origin URL). Put the origin in a
code span and each `-> <path>` on a two-space continuation
line in `log_writer.py`
- [ ] Test: 90-char URL ingest, then `wiki sanity` has no
markdownlint hit in `wiki/log.md`; still one bullet per run
- [ ] `mdoctor chklint` and `mdoctor fix` reject a file path
(SPEC 4.2, 7.4.4). Reproduced: both print
`is not a valid directory`. Accept a single `.md` file the way
`fmlint chklint` does
- [ ] Test: `fix` on one file lists only that file; other hashes
unchanged; missing path gives `does not exist`
- [ ] Post-ingest hint for leftover style violations
(SPEC 7.1.5.5.6). Depends on the `mdoctor` file item above.
Print `Next: fix the style violations listed by fiona-assistant mdoctor chklint <page>.`
before `Next: refine`
- [ ] "Relative path fails on Windows" is a misdiagnosis
(SPEC 7.1.5.2.7). `./ingest/x.md` and `.\ingest\x.md` both
work from the wiki root. The tester's file was already moved
to `raw/articles/tester-note.md` by the earlier absolute-path
run (SPEC 7.1.5.3.6), so the second run failed. Make the error
say so: add `(moved to <raw path> by an earlier ingest)` when
the missing path is under `ingest/` and `raw/` has that name
- [ ] Test: both separators ingest; second run gets the hint
- [ ] `--dry-run` logs `archived <raw path>` although nothing is
written (SPEC 7.1.5.8.1). Reproduced. Log
`dry run: would archive` / `dry run: would write` instead
- [ ] Housekeeping sanity details repeat the whole PyMarkdown report
(SPEC 7.2.5.1). One line per violation:
`<check>: <path>:<line>:<column> <code> <message>`
- [ ] Housekeeping logs `WARNING Overwriting existing file:
wiki_graph.html` on every run (SPEC 7.1.4.5.9.1). Add
`wiki graph -f/--force`; housekeeping passes it and logs at
INFO
- [ ] `fiona-assistant --version` (SPEC 2.3). Reproduced: argparse
rejects it. Print the same string as `ingest.tool`
- [ ] README "After ingest" and "On Windows" notes (SPEC 7.18.4), and
the same after-ingest loop in the Fiona skill. Bump both
versions
- [ ] Wiki root discovery from `wiki/` (SPEC 7.20) is already tracked
under P2. The tester calls it daily friction for agents. It is
blocked on the Appendix C rule; decide that rule, then consider
moving SPEC 7.20 to P1
- [ ] `librarian move` is already tracked under P1. The tester hit
`invalid choice`; no new action, since SPEC 4.1.2 marks it
`NOT_IMPLEMENTED` and the README does not list it
- [ ] Decide whether `git undo` and `git clean` prompt on a TTY
(SPEC Appendix C). The tester lost a `git save` commit with
`git undo`. Listed under P4 as well
## P1 — Wiki Lifecycle and Version Control
- [x] `wiki init` hardening (SPEC 7.1.1)
@@ -464,7 +540,7 @@ items below are done. The command defects after them are still open.
markdown file as bytes, detect a UTF-8 BOM, decode, call
`rewrite_text`. Disk is read only
- [ ] Inbound scan inside `plan` (SPEC 7.3.3.6): walk every
`*.md` under `wiki_root`, skip hidden folders (`.git`
`*.md` under `wiki_root`, skip hidden folders (`.git`\[\[[^|]*\|(.*)\]\]
included) and the moved files. Call `rewrite_text(...,
inbound=True)` on each. A file whose first part relative
to the root is in `READONLY_DIRS` goes to
@@ -1117,6 +1193,9 @@ WARNING run_tests: pytest short summary missing from output; failure details com
## P4 — Design Decisions Needed
- [ ] Should `git undo` and `git clean` prompt on a TTY, with `--yes`
to skip it? Agents would then pass `--yes` every time
(SPEC Appendix C, 2026-10-02 manual test)
- [ ] Librarian: version tracking for non-MD files (SPEC 7.3.2)
- [ ] Python files — format TBD
- [ ] Bash scripts — format TBD