4008 lines
185 KiB
Markdown
4008 lines
185 KiB
Markdown
---
|
|
name: Fiona-Assistant Specification
|
|
version: 1.9.1
|
|
title: Project Spec Sheet
|
|
created: 2026-09-23
|
|
updated: 2026-10-06
|
|
type: summary
|
|
---
|
|
|
|
# Fiona-Assistant Specification
|
|
|
|
<!-- toc-start: headings 2..6 -->
|
|
|
|
- [1. Product Overview](#1-product-overview)
|
|
- [1.1. Document Conventions](#11-document-conventions)
|
|
- [1.2. Constraints](#12-constraints)
|
|
- [2. Package and Runtime](#2-package-and-runtime)
|
|
- [3. llm-wiki Template](#3-llm-wiki-template)
|
|
- [3.1. What Counts as a Wiki](#31-what-counts-as-a-wiki)
|
|
- [3.2. Core Files and Read-only Zones](#32-core-files-and-read-only-zones)
|
|
- [3.3. Deployed Wiki Rules](#33-deployed-wiki-rules)
|
|
- [4. Command Catalog](#4-command-catalog)
|
|
- [4.1. librarian](#41-librarian)
|
|
- [4.1.1. librarian toc](#411-librarian-toc)
|
|
- [4.1.2. librarian move](#412-librarian-move)
|
|
- [4.2. mdoctor](#42-mdoctor)
|
|
- [4.3. fmlint](#43-fmlint)
|
|
- [4.4. python-ruff](#44-python-ruff)
|
|
- [4.5. json-lint](#45-json-lint)
|
|
- [4.6. git](#46-git)
|
|
- [4.7. wiki](#47-wiki)
|
|
- [4.7.1. wiki init](#471-wiki-init)
|
|
- [4.7.2. wiki check-update](#472-wiki-check-update)
|
|
- [4.7.3. wiki update](#473-wiki-update)
|
|
- [4.7.4. wiki sanity](#474-wiki-sanity)
|
|
- [4.7.5. wiki graph](#475-wiki-graph)
|
|
- [4.7.6. wiki ingest](#476-wiki-ingest)
|
|
- [4.8. housekeeping](#48-housekeeping)
|
|
- [4.9. Repository runners](#49-repository-runners)
|
|
- [5. Settings Schema](#5-settings-schema)
|
|
- [5.1. fiona-settings.json](#51-fiona-settingsjson)
|
|
- [5.2. settings_dev.json](#52-settings_devjson)
|
|
- [5.3. constellation themes](#53-constellation-themes)
|
|
- [6. Violation Rule Catalog](#6-violation-rule-catalog)
|
|
- [6.1. Shared Report](#61-shared-report)
|
|
- [6.2. Frontmatter Fields and FML Codes](#62-frontmatter-fields-and-fml-codes)
|
|
- [6.3. MDL Codes](#63-mdl-codes)
|
|
- [6.4. PYC Codes](#64-pyc-codes)
|
|
- [6.5. JSN Codes](#65-jsn-codes)
|
|
- [6.6. Base Linter Contract](#66-base-linter-contract)
|
|
- [7. Feature Requirements](#7-feature-requirements)
|
|
- [7.1. wiki](#71-wiki)
|
|
- [7.1.1. wiki init](#711-wiki-init)
|
|
- [7.1.2. wiki update](#712-wiki-update)
|
|
- [7.1.3. wiki sanity](#713-wiki-sanity)
|
|
- [7.1.4. wiki graph](#714-wiki-graph)
|
|
- [7.1.5. wiki ingest](#715-wiki-ingest)
|
|
- [7.2. housekeeping](#72-housekeeping)
|
|
- [7.3. librarian](#73-librarian)
|
|
- [7.3.1. Folder and In-file TOC](#731-folder-and-in-file-toc)
|
|
- [7.3.2. Non-markdown Version Tags](#732-non-markdown-version-tags)
|
|
- [7.3.3. Move with Link Rewrite](#733-move-with-link-rewrite)
|
|
- [7.4. mdoctor](#74-mdoctor)
|
|
- [7.5. fmlint](#75-fmlint)
|
|
- [7.6. mdlinter](#76-mdlinter)
|
|
- [7.7. pycustomlint](#77-pycustomlint)
|
|
- [7.8. python-ruff](#78-python-ruff)
|
|
- [7.9. Shellcheck](#79-shellcheck)
|
|
- [7.10. Markdown Code Blocks](#710-markdown-code-blocks)
|
|
- [7.11. json-lint](#711-json-lint)
|
|
- [7.12. Git Operations](#712-git-operations)
|
|
- [7.13. Backup Script](#713-backup-script)
|
|
- [7.14. Test and Sanity Runners](#714-test-and-sanity-runners)
|
|
- [7.15. html-js-lint](#715-html-js-lint)
|
|
- [7.16. settings setup](#716-settings-setup)
|
|
- [7.17. Logging](#717-logging)
|
|
- [7.18. Documentation](#718-documentation)
|
|
- [7.19. Tool Roles](#719-tool-roles)
|
|
- [7.20. Wiki Root Discovery](#720-wiki-root-discovery)
|
|
- [7.21. Bootstrap install modes](#721-bootstrap-install-modes)
|
|
- [Appendix A. Non-Goals](#appendix-a-non-goals)
|
|
- [Appendix B. Priority Order](#appendix-b-priority-order)
|
|
- [Appendix C. Open Questions](#appendix-c-open-questions)
|
|
- [Appendix D. Maintainer Notes](#appendix-d-maintainer-notes)
|
|
|
|
<!-- toc-end -->
|
|
|
|
This document is the build contract for Fiona-Assistant. A reader with
|
|
this file, and no source tree, must be able to produce the same
|
|
observable software. Status values are `Implemented`, `Partially
|
|
Implemented`, `Planned`, and `Won't Do`.
|
|
|
|
Revision 0.40.0 reorganizes the sheet into contracts. Revision 0.40.1
|
|
numbers nested requirements such as `7.14.1.1`. Revision 0.40.4 adds
|
|
the PYC001 exception in *6.4.2.*. Revision 0.41.0 installs one
|
|
`fiona` package and keeps tests and repo scripts outside it.
|
|
Revision 0.90.0 specifies `node_importance_min_point` in *7.1.4.16.*
|
|
and leaves that requirement unbuilt. Revision 0.92.0 specifies
|
|
`wiki ingest` in *7.1.5.* and leaves it unbuilt. Revision 0.93.0 adds
|
|
SHA-256 reuse across `raw/`, the complete `ingest` frontmatter map,
|
|
and the agent metadata check. Revision 0.94.0 makes `wiki update`
|
|
yield to the owner's markdown and write `<stem>-update.md` instead
|
|
(*7.1.2.11.*), and adds `fiona-assistant housekeeping` (*7.2.*).
|
|
Revision 0.96.0 compares a file without a version by SHA-256 and
|
|
shows both short hashes in the update report (*7.1.2.12.*).
|
|
Revision 0.97.0 rejects a module-level assignment that follows the
|
|
first function or class (*6.4.3.*). Revision 0.98.0 specifies that
|
|
`wiki graph` omits a file matched by `.gitignore` (*7.1.4.17.*) and
|
|
leaves that requirement unbuilt. Revision 0.99.0 builds `wiki ingest`
|
|
(*7.1.5.*). Revision 1.0.0 groups each module's functions under that
|
|
module and moves Non-Goals, Priority Order, and Open Questions
|
|
to Appendices A, B, and C. Revision 1.1.0 treats a differing
|
|
`fiona-settings.json` as a conflict and writes
|
|
`fiona-settings-update.json` (*7.1.2.3.*). Revision 1.2.0 folds
|
|
text line endings before the SHA-256 compare (*7.1.2.13.*) and
|
|
writes `corefiles.json` so a later update does not list a
|
|
core file whose version and hashes are unchanged (*7.1.2.14.*).
|
|
Revision 1.3.0 specifies directory nodes (*7.1.4.18.*) and
|
|
`wiki graph --serve` (*7.1.4.19.*) and leaves both unbuilt.
|
|
Revision 1.4.0 names the core-file record `corefiles.json`
|
|
(*7.1.2.14.*) and skips paths in `fiona.wiki.update.ignore_files`
|
|
(*7.1.2.15.*). The shipped list names `corefiles.json`. Only
|
|
`wiki update` writes that file. Revision 1.5.0 omits markdown
|
|
files matched by `fiona.wiki.graph.ignore_files` from the graph
|
|
(*7.1.4.20.*). The shipped list is `*/table_of_contents.md`,
|
|
because a table of contents links to every page. Revision 1.5.2
|
|
gives that file frontmatter `name` and `version` (*7.3.1.1.1.*)
|
|
so `fmlint` accepts it. Revision 1.6.0 specifies `librarian move`
|
|
(*7.3.3.*) and leaves it unbuilt. Revision 1.7.0 makes that
|
|
command rewrite links to the moved paths across the whole wiki
|
|
(*7.3.3.6.*). Revision 1.8.0 adds the findings of the 2026-10-02
|
|
manual test and leaves them unbuilt: one-line CLI errors with no
|
|
traceback (*1.2.7.*, *7.1.1.4.*), `--version` (*2.3.*), a quiet
|
|
default stderr (*7.17.3.*), an ingest hint for a drop-zone file that
|
|
already moved (*7.1.5.2.7.*), a dry run that does not log writes
|
|
(*7.1.5.8.1.*), a lint-clean ingest log line (*7.1.5.6.1.*), a
|
|
follow-up hint for style violations left on a page (*7.1.5.5.6.*),
|
|
single-file `mdoctor` (*7.4.4.*), compact housekeeping sanity details
|
|
(*7.2.5.1.*), a quiet graph overwrite (*7.1.4.5.9.1.*), and a usage
|
|
section in the README (*7.18.4.*). Revision 1.9.0 makes
|
|
`wiki ingest` stop with exit 4 on a URL that needs a login
|
|
(*7.1.5.2.8.*), adds `--origin` for a file saved by hand
|
|
(*7.1.5.2.9.*), and tells the agent how to get the source instead
|
|
(*7.1.5.14.*). All three are unbuilt.
|
|
|
|
Each feature section starts with a `Status` line. Requirements use
|
|
`- *N.M.K.*`. Each `*acceptance*` sits under the requirement it tests.
|
|
`NOT_IMPLEMENTED` marks work that is specified and not built yet.
|
|
Class names, file paths, and history live in Appendix D.
|
|
|
|
---
|
|
|
|
## 1. Product Overview
|
|
|
|
Fiona-Assistant is a CLI toolkit that manages, organizes, and checks
|
|
LLM wiki content with deterministic scripts. Reserve LLM tokens for
|
|
creative and analytical work.
|
|
|
|
### 1.1. Document Conventions
|
|
|
|
> Status: Implemented
|
|
|
|
- *1.1.1.* Main sections use `## N. Title`. Subsections use
|
|
`### N.M. Title`. Deeper headings use `#### N.M.K. Title`.
|
|
- *acceptance*: Every heading follows that numbering.
|
|
- *1.1.2.* A requirement is `- *N.M.K.* <text>`. A group is numbered
|
|
the same way, and each item under it adds one segment, for example
|
|
`*7.14.1.*` then `*7.14.1.1.*`. Its acceptance is the next nested
|
|
`- *acceptance*: <text>`, and that acceptance describes this
|
|
requirement only.
|
|
- *acceptance*: No acceptance describes a neighboring requirement.
|
|
- *acceptance*: The shared-engine bullets in section 7.14 are
|
|
numbered `7.14.1` and `7.14.1.1`, not plain dashes.
|
|
- *1.1.3.* Unbuilt requirements start with `` `NOT_IMPLEMENTED` ``.
|
|
- *acceptance*: Every planned item carries that tag.
|
|
- *1.1.4.* The block between `<!-- toc-start: headings 2..6 -->` and
|
|
`<!-- toc-end -->` is generated by `librarian toc file`. Leave that
|
|
block to the generator.
|
|
|
|
### 1.2. Constraints
|
|
|
|
These apply to every feature.
|
|
|
|
- *1.2.1.* Tool reports are parseable and use the shared shapes in
|
|
sections 4, 6, and 7.
|
|
- *acceptance*: Two linters that find violations print the same
|
|
section order from section 6.1.
|
|
- *1.2.2.* A command that needs a human decision prompts on a TTY.
|
|
The same command accepts an explicit flag (`--force`, `--yes`, or
|
|
`--non-interactive`) and then runs without a prompt.
|
|
- *acceptance*: `wiki update` on a non-TTY declines unless `--force`
|
|
or `--yes` is set.
|
|
- *1.2.3.* `personal/` and `raw/` are read-only for every automated
|
|
tool. Tools may resolve links that point into those zones. The one
|
|
exception is `wiki ingest` (*7.1.5.3.*): it may add a new file under
|
|
`raw/`. It never modifies, renames, or deletes a file already there.
|
|
- *acceptance*: `wiki update` leaves every file under `personal/`
|
|
and `raw/` byte-for-byte unchanged.
|
|
- *acceptance*: A second `wiki ingest` run leaves every file that
|
|
was already under `raw/` byte-for-byte unchanged.
|
|
- *acceptance*: Ingesting bytes whose SHA-256 matches any file
|
|
already under `raw/`, from any path or URL, adds no file under
|
|
`raw/` (*7.1.5.3.3.*).
|
|
- *1.2.4.* Code changes follow test-first development.
|
|
`python scripts/run_tests.py` is green, then
|
|
`python scripts/run_sanity.py` is green, before a change is finished.
|
|
- *acceptance*: A sanity run below the coverage gate in section 5.2
|
|
fails even when every test passes.
|
|
- *1.2.5.* Python passes Ruff. Markdown passes PyMarkdown with the
|
|
rules in section 5.1.
|
|
- *acceptance*: An unused import is reported by `python-ruff chklint`
|
|
with file, line, column, and rule code.
|
|
- *1.2.6.* Authored lengths in HTML and CSS use `em` or another
|
|
relative unit (`rem`, `%`, `vh`, `vw`). They are not written in
|
|
`px`. The wiki graph page in section 7.1.4 is that web surface.
|
|
A position read from the layout engine is written back in `rem`.
|
|
A theme length number uses the same scale: 16 is `1em`.
|
|
- *acceptance*: The graph template contains no `px` length.
|
|
- *1.2.7.* `NOT_IMPLEMENTED` An expected failure, such as a missing
|
|
path, a refusal, or bad input, prints one `Error: <message>` line on
|
|
stderr and exits 1. It prints no Python traceback. A traceback is
|
|
logged only at `DEBUG`.
|
|
- *acceptance*: A second `wiki init` on the same directory writes
|
|
no line containing `Traceback` on stderr (*7.1.1.4.*).
|
|
|
|
---
|
|
|
|
## 2. Package and Runtime
|
|
|
|
> Status: Partially Implemented
|
|
|
|
Python `>= 3.11` (also 3.12 and 3.13). The default development
|
|
install from the repository root is `pip install -e ".[dev]"`.
|
|
`bootstrap --local` installs the same tree as a normal package
|
|
(`pip install .`). Section 7.21 defines those modes. The build
|
|
requires setuptools 61 or newer.
|
|
|
|
Runtime dependencies:
|
|
|
|
| Package | Minimum |
|
|
| --- | --- |
|
|
| pymarkdownlnt | 0.9.0 |
|
|
| ruff | 0.16.8 |
|
|
| pyyaml | 6.0.3 |
|
|
| GitPython | 3.1.0 |
|
|
|
|
Development extras: `pytest>=7.0`, `pytest-cov>=4.0`, `pytest-mock`,
|
|
`pylint`. `html5validator` is not a dependency until section 7.15
|
|
is implemented. `beautifulsoup4`, `markdownify`, and `pypdf` become
|
|
runtime dependencies when section 7.1.5 is implemented.
|
|
|
|
One importable package, `fiona`, lives under `src/fiona/`. Its
|
|
subpackages are `librarian`, `mdoctor`, `customlinter`,
|
|
`python_ruff`, `gittool`, `sanity_runner`, and `wiki`. The wiki
|
|
template is package data at `src/fiona/data/llm_wiki/` and is loaded
|
|
with `importlib.resources`. It is not an importable module. Tests
|
|
live in repo-root `tests/` and are not installed. `scripts/` holds
|
|
`settings_dev.json`, shell helpers, and the launchers
|
|
`scripts/run_tests.py` and `scripts/run_sanity.py`.
|
|
|
|
Console scripts:
|
|
|
|
| Script | Entry |
|
|
| --- | --- |
|
|
| `fiona-assistant` | `fiona.__main__:main` |
|
|
| `librarian` | `fiona.librarian.librarian:main` |
|
|
| `mdoctor` | `fiona.mdoctor.mdoctor:main` |
|
|
|
|
`python-ruff`, `fmlint`, `json-lint`, and `git` are reached only
|
|
through `fiona-assistant`. `html-js-lint` is not installed until
|
|
section 7.15 is implemented.
|
|
|
|
Coverage measures `src/fiona`, omitting tests. The config lives in
|
|
`pyproject.toml`. The gate percentage is section 5.2.
|
|
|
|
Logging: every CLI entry calls `setup_logging()` before work. Level
|
|
order is `LOG_LEVEL` environment variable, then `fiona.log_level` in
|
|
`fiona-settings.json`, then `WARNING`. Records go to stderr. When
|
|
`fiona.file_log_location` is set, a file handler writes lines
|
|
`%(asctime)s %(levelname)s %(name)s: %(message)s` at or above
|
|
`fiona.file_log_level` (fallback: `fiona.log_level`). A relative path
|
|
is resolved from the current working directory. An unopenable path
|
|
logs a warning and keeps stderr logging. Human reports stay on
|
|
stdout. Package loggers set their level explicitly because
|
|
pymarkdown resets the root logger mid-run.
|
|
|
|
- *2.1.* The three console scripts above are installed by
|
|
`pip install -e .`.
|
|
- *acceptance*: `fiona-assistant` with no arguments prints help and
|
|
exits 0.
|
|
- *2.2.* `html-js-lint` is not an installed script until section 7.15
|
|
exists.
|
|
- *acceptance*: A fresh install does not fail at import because of
|
|
a missing `html_js_lint` package.
|
|
- *2.3.* `NOT_IMPLEMENTED` `fiona-assistant --version` prints
|
|
`fiona-assistant <package version>` on stdout and exits 0. The text
|
|
is the same string `wiki ingest` writes to `ingest.tool`
|
|
(*7.1.5.5.3.*).
|
|
- *acceptance*: The stdout of `--version` equals the `ingest.tool`
|
|
value of a page ingested by the same install.
|
|
|
|
---
|
|
|
|
## 3. llm-wiki Template
|
|
|
|
> Status: Implemented
|
|
|
|
`wiki init` copies this tree from the packaged template. Text files
|
|
replace every `DATE_PLACEHOLDER` with the current UTC date
|
|
`YYYY-MM-DD`. Binary files are copied unchanged. A glob in
|
|
`fiona.llm-wiki.ignore_files` skips that file, or that directory and
|
|
its subtree. A missing settings file or a missing key copies
|
|
everything. A non-list `ignore_files` value raises `ValueError` with
|
|
the text `fiona.llm-wiki.ignore_files must be a list of strings`.
|
|
|
|
```text
|
|
AGENTS.md
|
|
README.md
|
|
fiona-settings.json
|
|
constellation/colorful.json
|
|
constellation/dark.json
|
|
constellation/grayscale.json
|
|
constellation/night-sky.json
|
|
constellation/professional.json
|
|
table_of_contents.md
|
|
ingest/README.md
|
|
personal/README.md
|
|
raw/README.md
|
|
wiki/schema.md
|
|
wiki/index.md
|
|
wiki/log.md
|
|
utils/README.md
|
|
utils/scripts/example.py
|
|
utils/scripts/map.md
|
|
utils/skills/README.md
|
|
utils/skills/fiona/SKILL.md
|
|
utils/skills/fiona/markdown_rules.md
|
|
utils/skills/fiona/llm-wiki-mod.md
|
|
utils/skills/python-tdd/SKILL.md
|
|
utils/personas/default/SOUL.md
|
|
utils/personas/template/SOUL.md
|
|
utils/skills_local/README.md
|
|
```
|
|
|
|
The shipped `fiona-settings.json` is the default document in
|
|
section 5.1. `wiki/schema.md` is the page-type and tag taxonomy.
|
|
This spec's frontmatter table in section 6.2 is the field contract;
|
|
where the two disagree, this spec wins.
|
|
|
|
### 3.1. What Counts as a Wiki
|
|
|
|
A directory is an llm-wiki when it contains `fiona-settings.json`.
|
|
`wiki init` refuses when any of these already exist:
|
|
`AGENTS.md`, `fiona-settings.json`, `wiki/`, `ingest/`, `README.md`.
|
|
|
|
### 3.2. Core Files and Read-only Zones
|
|
|
|
Core files, matched by name at the wiki root:
|
|
|
|
- `AGENTS.md`
|
|
- `fiona-settings.json`
|
|
- `README.md`
|
|
- `table_of_contents.md`
|
|
|
|
Core directories (every file under them is core):
|
|
|
|
- `wiki/`
|
|
- `ingest/`
|
|
- `personal/`
|
|
- `raw/`
|
|
- `utils/`
|
|
- `constellation/`
|
|
|
|
Read-only zones, including the zone directory itself:
|
|
|
|
- `personal/`
|
|
- `raw/`
|
|
|
|
`wiki/` is copied by `wiki init` and then left to the LLM.
|
|
`wiki update` does not add, replace, or delete any path under
|
|
`wiki/`. The only file it writes there is an update file for a
|
|
template page that is newer than the wiki's copy (*7.1.2.11.*).
|
|
|
|
`wiki update` adds missing core files and directories. It never
|
|
overwrites an existing markdown file: when the template carries a
|
|
newer version, it writes `<stem>-update.md` beside the file and
|
|
leaves the owner's file as it is. A differing
|
|
`fiona-settings.json` is the same kind of conflict: the owner's
|
|
file stays, and the template is written as
|
|
`fiona-settings-update.json` (*7.1.2.3.*). A confirmed update
|
|
that changes or reports anything also writes `corefiles.json`
|
|
at the wiki root (*7.1.2.14.*). That file is not part of the
|
|
template. Only `wiki update` writes it. A glob in
|
|
`fiona.wiki.update.ignore_files` is not updated (*7.1.2.15.*).
|
|
The shipped list is `corefiles.json`. It never deletes user files. It
|
|
never writes inside a read-only zone, so a missing file under
|
|
`personal/` or `raw/` is not added back. It
|
|
never writes a path matched by the repository `.gitignore` or by
|
|
the wiki's own `.gitignore`.
|
|
|
|
After the copy, `wiki init` creates a git repository and writes
|
|
`.gitignore`. A new file starts with the fiona-assistant repository
|
|
`.gitignore` (the same patterns, including `__pycache__/`, `*.pyc`,
|
|
and `*fiona.log`). When this package is not the source checkout,
|
|
that text is the packaged `default.gitignore`. The file adds
|
|
`personal/` when that pattern is missing.
|
|
An existing `.gitignore` keeps every line it already has; missing
|
|
patterns from that set are appended. Init then stages everything
|
|
and commits `Initial llm-wiki commit`. Template paths that match
|
|
the repository `.gitignore` are not copied. `personal/` and `raw/`
|
|
are still copied. `personal/` stays untracked. If git init fails,
|
|
the copy still counts as success and stdout adds
|
|
`Note: Git repository initialization failed: <error>`.
|
|
|
|
- *3.2.1.* An empty directory becomes the tree above, with dates
|
|
substituted, and a git commit that does not track `personal/`.
|
|
- *acceptance*: `personal/` is absent from `git ls-files` after
|
|
`wiki init`.
|
|
- *3.2.2.* A second `wiki init` raises `FileExistsError` whose
|
|
message names each conflicting indicator, and writes nothing.
|
|
- *acceptance*: An existing `README.md` alone is enough to refuse,
|
|
and that file's bytes are unchanged.
|
|
|
|
### 3.3. Deployed Wiki Rules
|
|
|
|
> Status: Implemented
|
|
|
|
Only the template tree is deployed. A wiki does not know this spec,
|
|
the root `AGENTS.md`, `scripts/`, or `tests/`. The template
|
|
`AGENTS.md` is the entry point: it says what the wiki is and loads
|
|
the Fiona skill, which loads `llm-wiki-mod.md`. The product rule
|
|
"never commit without permission" (project agent rule 2.4) covers
|
|
this repository only.
|
|
|
|
- *3.3.1.* No markdown file in the template references a file or
|
|
rule outside the template: `SPEC`, `project agent rule`, or
|
|
`scripts/run_`.
|
|
- *acceptance*: A scan of every template `*.md` finds none of those
|
|
strings.
|
|
- *3.3.2.* The template `AGENTS.md` links the Fiona skill and
|
|
`llm-wiki-mod.md`.
|
|
- *acceptance*: Both relative links appear in that file.
|
|
- *3.3.3.* A deployed wiki commits with
|
|
`fiona-assistant git save --description TEXT` after each ingest
|
|
cycle and each housekeeping pass. `clean`, `undo`, and `restore`
|
|
run only when the user asks.
|
|
- *acceptance*: The template `AGENTS.md` and the Fiona skill both
|
|
contain `git save`, and neither says to commit only when asked.
|
|
- *3.3.4.* The template `wiki/log.md` holds one seed entry dated
|
|
`DATE_PLACEHOLDER`.
|
|
- *acceptance*: The log body has exactly one bullet.
|
|
|
|
---
|
|
|
|
## 4. Command Catalog
|
|
|
|
> Status: Implemented
|
|
|
|
One command tree. Optional `[directory]` defaults to `.` today.
|
|
Wiki-root discovery (section 7.20) is not built. An invalid
|
|
directory prints `Error: '<path>' is not a valid directory.` on
|
|
stderr and exits 1, unless a command below states a different
|
|
message.
|
|
|
|
`fiona-assistant` with no command prints help and exits 0.
|
|
`fiona-assistant git` with no subcommand prints git help and exits 1.
|
|
`fiona-assistant wiki` with no subcommand prints wiki help and
|
|
exits 1.
|
|
|
|
### 4.1. librarian
|
|
|
|
`fiona-assistant librarian` and `librarian` with no subcommand list
|
|
the current directory.
|
|
|
|
`list [directory]` stdout:
|
|
|
|
```text
|
|
Found <N> files in '<directory>':
|
|
- <path>
|
|
```
|
|
|
|
`mdoctor list` uses the same listing with the prefix
|
|
`MDoctor: Found <N> files in '<directory>':`.
|
|
|
|
#### 4.1.1. librarian toc
|
|
|
|
`toc [directory] [flags]` writes `table_of_contents.md` in the
|
|
target folder. Flags: `--recursive-toc`, `--non-recursive`, and
|
|
repeatable `--exclude REGEX`. Depth comes from section 5.1.
|
|
|
|
A path that is a file or missing exits 1 on stderr and names
|
|
`librarian toc file <path>`. A folder with no markdown prints
|
|
`no markdown files found` on stdout and writes nothing. In
|
|
`--recursive-toc` mode that message is a log line at INFO instead.
|
|
|
|
`toc file <path>` writes an in-file TOC in that markdown file.
|
|
`toc file *` does that for every markdown file under the wiki root
|
|
found by section 7.3.1. `--sectionized` forces section numbering for
|
|
that run. Status lines:
|
|
|
|
```text
|
|
wrote toc in <path>
|
|
toc already up to date in <path>
|
|
skipped <path> (<reason>)
|
|
generated toc for <N> file(s), skipped <M>
|
|
```
|
|
|
|
The summary line is bulk mode only. A file with no H1 is
|
|
`skipped <path> (no H1 heading)` and is not rewritten.
|
|
|
|
#### 4.1.2. librarian move
|
|
|
|
`NOT_IMPLEMENTED` for the whole command. Section 7.3.3 is the
|
|
contract.
|
|
|
|
`move <source> <target> [--dry-run]` moves a file or a folder and
|
|
leaves no dead link behind. Each moved markdown file gets its
|
|
relative links rewritten, so every link that resolved before the
|
|
move still resolves from the new place. Every other markdown file in
|
|
the wiki that links to a moved path gets that link rewritten to the
|
|
new path.
|
|
|
|
A `<target>` that is an existing folder receives `<source>` under
|
|
its own name, the same as `mv`. Any other `<target>` is the new
|
|
path. Missing parent folders are created.
|
|
|
|
Stdout, paths with `/` separators:
|
|
|
|
```text
|
|
moved <source> -> <target>
|
|
rewrote <N> link(s) in <path>
|
|
unresolved <N> link(s) in <path>
|
|
read-only <N> link(s) in <path>
|
|
moved <F> file(s), rewrote <L> link(s) in <R> file(s),
|
|
<U> unresolved, <K> read-only
|
|
```
|
|
|
|
`rewrote`, `unresolved`, and `read-only` lines appear only for a
|
|
file whose count is above 0, sorted by `<path>`. `<path>` is the
|
|
file's path after the move. The summary line is always last and is
|
|
one line, even though it is wrapped above. With `--dry-run`, every line starts with `[dry-run]`
|
|
and one space, and nothing is written. A refusal (*7.3.3.2.*) prints one
|
|
`Error: ...` line on stderr, writes nothing, and exits 1. Success
|
|
exits 0.
|
|
|
|
### 4.2. mdoctor
|
|
|
|
No subcommand means `chklint .`.
|
|
|
|
`chklint [directory]` prints the shared report (section 6.1) with
|
|
header `PyMarkdownlint Violations` and exits 0 even when violations
|
|
exist.
|
|
|
|
`fix [directory]` runs pymarkdown auto-fix in place. Stdout:
|
|
|
|
```text
|
|
PyMarkdownFix: <N> file(s) fixed
|
|
- <path>
|
|
```
|
|
|
|
A pymarkdown API failure prints
|
|
`Error: pymarkdown fix failed: <error>` on stderr and exits 1.
|
|
The command does not skip `personal/` or `raw/`.
|
|
|
|
`NOT_IMPLEMENTED` `chklint` and `fix` also take one markdown file in
|
|
place of `[directory]` (*7.4.4.*). Today a file prints
|
|
`Error: '<path>' is not a valid directory.` and exits 1.
|
|
|
|
### 4.3. fmlint
|
|
|
|
`fiona-assistant fmlint` with no subcommand means `chklint .`.
|
|
`chklint [directory]` prints the shared report with header
|
|
`Frontmatter Lint Violations` and exits 0 even when violations
|
|
exist. A missing directory exits 1.
|
|
|
|
### 4.4. python-ruff
|
|
|
|
No subcommand means `chklint .`. The report header is
|
|
`Ruff Violations`. Exit code is 0 when the ruff process itself
|
|
succeeds, including when violations are listed. `lint(path, fix=True)`
|
|
passes `--fix` to the ruff binary. There is no `fix` subcommand yet.
|
|
|
|
### 4.5. json-lint
|
|
|
|
No subcommand means `chklint .`. Header is `JSONLint Violations`.
|
|
|
|
`chklint [directory]` exits 1 when the report contains violations
|
|
and exits 0 when it reports no violations. This differs from
|
|
mdoctor, fmlint, and python-ruff, which exit 0 either way.
|
|
|
|
`chklint --fix` and `fix [directory]` rewrite indentation and
|
|
trailing commas in place, skip `personal/` and `raw/`, and print:
|
|
|
|
```text
|
|
JSONLintFix: <N> file(s) fixed
|
|
- <path>
|
|
```
|
|
|
|
A missing directory on `fix` prints
|
|
`Error: '<path>' does not exist.` on stderr and exits 1.
|
|
|
|
### 4.6. git
|
|
|
|
Operates on the git repository at the current directory. Timestamps
|
|
are UTC.
|
|
|
|
| Command | Stdout on success |
|
|
| --- | --- |
|
|
| `save --description TEXT` | `Committed: <full sha>` |
|
|
| `snapshot <tag_name> --description TEXT` | `Snapshot created: commit=<sha>, tag=<tag>` |
|
|
| `restore <tag_name>` | `Restored to: <sha>` |
|
|
| `clean` | `Working tree cleaned. All changes discarded.` |
|
|
| `undo` | `Undo to: <sha>` |
|
|
| `status` | `Status: clean` or `Status: dirty`, then `git status` |
|
|
|
|
On failure the tool prints `Error: <message>` and exits 1.
|
|
|
|
Commit message: `<YYYY-MM-DD HH:MM:SS> <description>`.
|
|
`save` runs `git add --all` and commits. A clean tree, including
|
|
no untracked files, raises `Working tree is clean. Nothing to commit.`
|
|
|
|
`snapshot` commits, then creates an annotated tag
|
|
`<YYYY-MM-DD_HH-MM-SS>_<tag_name>` with message
|
|
`Snapshot: <description>`.
|
|
|
|
`restore` requires a clean tree
|
|
(`Working tree is dirty. Use 'clean' to discard current edits first.`).
|
|
It accepts the full tag or the user tag suffix. The latest tag whose
|
|
name equals the argument or ends with `_<argument>` wins. It checks
|
|
that commit's tree into the index and working tree and adds a new
|
|
commit `<timestamp> Restore to <full tag>`. History before that
|
|
commit stays.
|
|
|
|
`clean` requires a dirty tree
|
|
(`Working tree is already clean. Nothing to discard.`), then
|
|
`git reset --hard HEAD` and `git clean -fd`.
|
|
|
|
`undo` requires a clean tree and at least two commits
|
|
(`Cannot undo: only one commit exists in the repository.`), then
|
|
`git reset --hard HEAD~1`.
|
|
|
|
### 4.7. wiki
|
|
|
|
Subcommands of `fiona-assistant wiki`.
|
|
|
|
#### 4.7.1. wiki init
|
|
|
|
`fiona-assistant wiki init [directory]`. Success stdout:
|
|
|
|
```text
|
|
Successfully initialized llm-wiki directory at: <resolved path>
|
|
```
|
|
|
|
Rules are section 3.
|
|
|
|
#### 4.7.2. wiki check-update
|
|
|
|
`fiona-assistant wiki check-update [directory] [--json]`.
|
|
The directory must contain `fiona-settings.json`, else stderr
|
|
`Error: Directory '<path>' is not an llm-wiki directory` and exit 1.
|
|
A missing directory is
|
|
`Error: Directory not found: <path>` and exit 1.
|
|
A successful check exits 0 whether or not updates exist.
|
|
|
|
Human stdout when current:
|
|
|
|
```text
|
|
Wiki is up to date.
|
|
```
|
|
|
|
When something differs, the lines that apply:
|
|
|
|
```text
|
|
Files to update: <N>
|
|
- <relative path> (<current> -> <template>)
|
|
Conflicts (owner copy kept): <N>
|
|
- <relative path> (<current> -> <template>) -> <update file name>
|
|
Directories to add: <N>
|
|
- <relative path>
|
|
Files to add: <N>
|
|
- <relative path>
|
|
|
|
Total: <N> file(s) need attention
|
|
Files skipped (locally modified): <N>
|
|
- <relative path>
|
|
```
|
|
|
|
The version parenthesis appears only when both versions exist.
|
|
When the file was compared by hash instead (*7.1.2.12.*), the
|
|
parenthesis is `(sha256 <current> -> <template>)`, each side the
|
|
first 8 lowercase hex digits of that SHA-256. The
|
|
`Conflicts` block appears only when a conflict exists (*7.1.2.11.*).
|
|
`Total` counts updates, conflicts, and additions.
|
|
`--json` prints one JSON object:
|
|
|
|
```json
|
|
{
|
|
"files_to_update": [
|
|
{
|
|
"path": "<path>",
|
|
"current_version": "<version or null>",
|
|
"template_version": "<version or null>",
|
|
"current_sha256": "<64 hex or null>",
|
|
"template_sha256": "<64 hex or null>",
|
|
"is_settings_file": false
|
|
}
|
|
],
|
|
"conflicts": [
|
|
{
|
|
"path": "<path>",
|
|
"update_path": "<path>",
|
|
"current_version": "<version or null>",
|
|
"template_version": "<version or null>",
|
|
"current_sha256": "<64 hex or null>",
|
|
"template_sha256": "<64 hex or null>"
|
|
}
|
|
],
|
|
"files_to_add": [],
|
|
"directories_to_add": [],
|
|
"files_skipped": [
|
|
{"path": "<path>", "reason": "locally modified"}
|
|
],
|
|
"total": 0,
|
|
"has_updates": false
|
|
}
|
|
```
|
|
|
|
#### 4.7.3. wiki update
|
|
|
|
`fiona-assistant wiki update [directory] [-f|--force] [-y|--yes]
|
|
[--dry-run] [--json]`.
|
|
|
|
Without `--force` and `--yes`, stdout shows the check-update text
|
|
and the prompt `Proceed with update? [y/N]` followed by a space.
|
|
`y` and `yes` proceed.
|
|
Any other answer, and a non-TTY, exit 1 with stderr
|
|
`Error: Update requires confirmation. Use --force or --yes to proceed without prompting.`
|
|
|
|
`--dry-run` prints `Dry run - no files will be modified:` and the
|
|
check-update text, writes nothing, and exits 0.
|
|
|
|
Success stdout:
|
|
|
|
```text
|
|
Update complete: <U> file(s) updated, <A> added, <C> conflict(s), <S> skipped.
|
|
```
|
|
|
|
When `<C>` is not zero, one more line follows:
|
|
|
|
```text
|
|
Merge each <stem>-update<suffix> into its file: run fiona-assistant housekeeping.
|
|
```
|
|
|
|
`--json` suppresses the plan and prints:
|
|
|
|
```json
|
|
{
|
|
"files_updated": [],
|
|
"files_added": [],
|
|
"directories_added": [],
|
|
"conflicts": [{"path": "<path>", "update_path": "<path>"}],
|
|
"files_skipped": [],
|
|
"total": 0
|
|
}
|
|
```
|
|
|
|
A confirmed update that had something to report writes
|
|
`corefiles.json` at the wiki root (*7.1.2.14.*).
|
|
`check-update` and `--dry-run` do not. A pristine wiki still
|
|
writes nothing (*7.1.2.5.*). A path in
|
|
`fiona.wiki.update.ignore_files` is left unchanged (*7.1.2.15.*).
|
|
|
|
Comparison and merge rules are section 7.1.2.
|
|
|
|
#### 4.7.4. wiki sanity
|
|
|
|
`fiona-assistant wiki sanity [directory] [--json] [--non-interactive]`.
|
|
Requires `fiona-settings.json`. Exit 0 when every check is clean,
|
|
exit 1 otherwise.
|
|
|
|
Checks, in this order, one thread each: `markdownlint`,
|
|
`frontmatter_lint`, `python-ruff`, `mdlinter`, `jsonlint`.
|
|
|
|
Human report:
|
|
|
|
```text
|
|
Wiki Sanity Report
|
|
TARGET: <path>
|
|
|
|
<failing check reports, omitted when clean>
|
|
|
|
Results:
|
|
<name padded>: PASS - <N> violations (MM:SS)
|
|
|
|
SANITY PASSED (MM:SS)
|
|
```
|
|
|
|
A failure ends with
|
|
`Sanity Failed - <N> checks need attention:` and one detail line per
|
|
failing check. The pass banner carries the total wall time. The
|
|
failure verdict does not. Per-check times may overlap and need not
|
|
sum to the total. Rows stay in registry order.
|
|
|
|
`--json` skips the progress display and prints:
|
|
|
|
```json
|
|
{
|
|
"target": "<path>",
|
|
"checks": {
|
|
"markdownlint": {
|
|
"result": 0,
|
|
"violation_count": 0,
|
|
"duration_seconds": 0.0,
|
|
"violations": []
|
|
}
|
|
},
|
|
"total_violations": 0,
|
|
"passed": true
|
|
}
|
|
```
|
|
|
|
`result` is 0 for pass and 1 for fail. Progress display is
|
|
section 7.14.
|
|
|
|
#### 4.7.5. wiki graph
|
|
|
|
`fiona-assistant wiki graph [directory] [-o FILE] [--format html|json]
|
|
[--include-pattern GLOB] [--exclude-pattern GLOB]
|
|
[--wikilinks|--no-wikilinks] [--json]
|
|
[--external-links|--no-external-links] [-f|--force]
|
|
[--serve [--host HOST] [--port PORT] [--no-open]]`.
|
|
|
|
Defaults: output `wiki_graph.html` in the current directory, format
|
|
`html`, include `**/*.md`, wikilinks on, external-link nodes off.
|
|
`--json` selects JSON format. Behavior is section 7.1.4.
|
|
A path in `fiona.wiki.graph.ignore_files` is not a node
|
|
(*7.1.4.20.*). The shipped pattern `*/table_of_contents.md`
|
|
also matches `table_of_contents.md` at the scan root.
|
|
`NOT_IMPLEMENTED` `--serve` runs a local live view instead of
|
|
writing once (*7.1.4.19.*). `--host`, `--port`, and `--no-open`
|
|
are accepted only with `--serve`. `NOT_IMPLEMENTED` `-f/--force`
|
|
overwrites an existing output file without the prompt of
|
|
*7.1.4.5.9.* (*7.1.4.5.9.1.*).
|
|
|
|
#### 4.7.6. wiki ingest
|
|
|
|
The contract is section 7.1.5.
|
|
|
|
```text
|
|
fiona-assistant wiki ingest <source> [--wiki DIR] [-r|--raw PATH]
|
|
[-t|--target PATH] [--origin URL] [--raw-only] [-f|--force]
|
|
[--dry-run] [--json]
|
|
fiona-assistant wiki ingest --pending [--wiki DIR] [--json]
|
|
```
|
|
|
|
`<source>` is an `http://` or `https://` URL, or a local file path.
|
|
`--wiki` is the wiki root and defaults to `.`. It must contain
|
|
`fiona-settings.json`, else stderr
|
|
`Error: Directory '<path>' is not an llm-wiki directory` and exit 1.
|
|
`<source>` and `--pending` are mutually exclusive, and one of them is
|
|
required. argparse reports a violation and exits 2.
|
|
`NOT_IMPLEMENTED` `--origin URL` names the URL a local file was saved
|
|
from (*7.1.5.2.9.*).
|
|
|
|
Human stdout on success, the lines that apply, in this order:
|
|
|
|
```text
|
|
Fetched: <url> (<media type>, <N> bytes)
|
|
Archived: <raw path>
|
|
Archived: <raw path> (already archived, sha256 <first 12 hex>)
|
|
Duplicate: <page path> already cites this source.
|
|
Wrote: <page path> (converter <name>, conversion <complete|partial>)
|
|
Warning: <text>
|
|
Next: refine <page path>, then set status: refined.
|
|
```
|
|
|
|
`Fetched:` appears only for a URL. One `Warning:` line per warning.
|
|
With no converter, or a failed conversion, the raw copy stays and
|
|
stdout ends with:
|
|
|
|
```text
|
|
No converter for <media type>. Convert <raw path> by hand.
|
|
```
|
|
|
|
`--pending` stdout:
|
|
|
|
```text
|
|
Raw pages: <N>
|
|
- <page path> (<first sources entry>)
|
|
```
|
|
|
|
`--json` prints one object and nothing else on stdout. Paths are
|
|
relative to the wiki root and use `/`:
|
|
|
|
```json
|
|
{
|
|
"source": "<argument as given>",
|
|
"origin": "<url, or the resolved file path>",
|
|
"origin_kind": "url",
|
|
"media_type": "application/pdf",
|
|
"raw_path": "raw/papers/<slug>.pdf",
|
|
"raw_reused": false,
|
|
"sha256": "<64 lowercase hex>",
|
|
"duplicate_page": null,
|
|
"page_path": "wiki/sources/<slug>.md",
|
|
"converter": "pdf",
|
|
"conversion": "complete",
|
|
"status": "raw",
|
|
"warnings": [],
|
|
"dry_run": false
|
|
}
|
|
```
|
|
|
|
`origin_kind` is `url` or `file`. With no page, `page_path` and
|
|
`converter` are `null` and `conversion` is `none`. `duplicate_page`
|
|
is the page that already cites the matching raw copy, else `null`
|
|
(*7.1.5.3.5.*). `--pending --json`
|
|
prints `{"pending": [{"page_path": ..., "sources": [...],
|
|
"origin": ...}], "total": <N>}`.
|
|
|
|
| Exit | Meaning |
|
|
| --- | --- |
|
|
| 0 | Raw copy archived. A page was written, `--raw-only` was set, or a page already cites the same bytes. |
|
|
| 1 | Error on stderr. Nothing written under the wiki. |
|
|
| 2 | argparse usage error. |
|
|
| 3 | Raw copy archived. No page: no converter, or conversion failed. |
|
|
| 4 | `NOT_IMPLEMENTED` The URL needs a login (*7.1.5.2.8.*). Error on stderr. Nothing written under the wiki. |
|
|
|
|
Errors, each on stderr with exit 1:
|
|
|
|
```text
|
|
Error: Source not found: <path>
|
|
Error: Source not found: <path> (moved to <raw path> by an earlier ingest)
|
|
Error: Source is a directory: <path>
|
|
Error: Unsupported URL scheme: <scheme>
|
|
Error: Download failed: <url>: <reason>
|
|
Error: Download exceeds <N> bytes: <url>
|
|
Error: --raw must stay inside raw/: <value>
|
|
Error: --raw cannot be used on a file already under raw/: <path>
|
|
Error: --target must stay inside wiki/: <value>
|
|
Error: --target cannot replace <path>
|
|
Error: Page exists: <path>. Use --force to replace it.
|
|
Error: --origin needs a local file: <source>
|
|
Error: --origin must be an http or https URL: <value>
|
|
```
|
|
|
|
`NOT_IMPLEMENTED` A URL that needs a login prints two lines on stderr
|
|
and exits 4 (*7.1.5.2.8.*):
|
|
|
|
```text
|
|
Error: Authentication required: <url>: <reason>
|
|
Next: save it into ingest/, then run: wiki ingest <file> --origin <url>
|
|
```
|
|
|
|
The `Next:` line names the command without the `fiona-assistant`
|
|
prefix so it fits one line. The agent fills `<file>` with the path
|
|
it saved (*7.1.5.14.*).
|
|
|
|
A failed conversion prints `Error: Conversion failed: <converter>:
|
|
<reason>` on stderr and exits 3, not 1, because the raw copy stays.
|
|
`NOT_IMPLEMENTED` The `(moved to ...)` form is *7.1.5.2.7.*.
|
|
|
|
### 4.8. housekeeping
|
|
|
|
`fiona-assistant housekeeping [directory] [--json]`. One call runs
|
|
the wiki maintenance routine and tells an LLM what to fix. The
|
|
directory must contain `fiona-settings.json`, else stderr
|
|
`Error: Directory '<path>' is not an llm-wiki directory` and exit 1.
|
|
A missing directory is `Error: Directory not found: <path>` and
|
|
exit 1. The steps and their rules are section 7.2.
|
|
|
|
Human stdout:
|
|
|
|
```text
|
|
Housekeeping Report
|
|
TARGET: <path>
|
|
|
|
[<STATUS>] <step>: <summary>
|
|
- <detail>
|
|
|
|
Next steps:
|
|
- <action>
|
|
|
|
HOUSEKEEPING PASSED
|
|
```
|
|
|
|
One `[<STATUS>]` block per step, in step order. `<STATUS>` is
|
|
`PASS`, `WARN`, or `FAIL`. Detail lines appear only when the step
|
|
has details. `Next steps:` appears only when a step has an action.
|
|
The last line is `HOUSEKEEPING PASSED`, or
|
|
`HOUSEKEEPING FAILED - <N> step(s) need attention: <names>` with
|
|
the failing step names joined by a comma and a space.
|
|
|
|
`--json` prints one object and nothing else on stdout. Paths are
|
|
relative to the wiki root and use `/`:
|
|
|
|
```json
|
|
{
|
|
"target": "<path>",
|
|
"steps": [
|
|
{
|
|
"name": "merge",
|
|
"status": "FAIL",
|
|
"summary": "1 update file(s) wait for a merge",
|
|
"details": ["AGENTS-update.md -> AGENTS.md"],
|
|
"next": ["Merge AGENTS-update.md into AGENTS.md ..."]
|
|
}
|
|
],
|
|
"passed": false
|
|
}
|
|
```
|
|
|
|
| Exit | Meaning |
|
|
| --- | --- |
|
|
| 0 | No step is `FAIL`. `WARN` steps may exist. |
|
|
| 1 | A step is `FAIL`, or the directory is not a wiki. |
|
|
|
|
---
|
|
|
|
### 4.9. Repository runners
|
|
|
|
`python scripts/run_tests.py` runs pytest with `--color=no` and a
|
|
temporary JUnit XML report (`junit_family=xunit1`). The child
|
|
process receives the repository `src` directory on `PYTHONPATH`,
|
|
joined with `os.pathsep`.
|
|
|
|
`python scripts/run_sanity.py [--non-interactive]` runs, in order:
|
|
`pytests`, `markdownlint`, `ruff`, `frontmatter_lint`,
|
|
`pycustomlint`, `jsonlint`. Same progress display, result table, and
|
|
verdict shape as wiki sanity, with label `Running Sanity` and an
|
|
extra untimed `coverage:` row under pytests. Details are section 7.14.
|
|
|
|
## 5. Settings Schema
|
|
|
|
Two files. `fiona-settings.json` ships in the template and marks a
|
|
wiki root. `scripts/settings_dev.json` is repository tooling only and is
|
|
not copied into a wiki.
|
|
|
|
### 5.1. fiona-settings.json
|
|
|
|
Absent file: pymarkdown uses its own defaults; other tools use the
|
|
defaults in the table. A typo in a pymarkdown rule id or property
|
|
name is accepted and has no effect. Confirm a new pymarkdown setting
|
|
by its effect on a scan. TOC file settings reject unknown keys and
|
|
bad types with exit 1 and the key name.
|
|
|
|
Shipped document (logical content; `wiki init` copies the file):
|
|
|
|
```json
|
|
{
|
|
"markdown": {
|
|
"pymarkdown": {
|
|
"extensions": ["front-matter"],
|
|
"respect_gitignore": true,
|
|
"md025": {"front_matter_title": "no-match"},
|
|
"md013": {"line_length": 100, "code_blocks": false, "tables": false},
|
|
"md060": {"enabled": false}
|
|
},
|
|
"frontmatter": {
|
|
"ignore_rules": [],
|
|
"ignore_file": [],
|
|
"respect_gitignore": true
|
|
},
|
|
"toc_generator": {
|
|
"max_depth_from_file": 3,
|
|
"max_depth_from_dir": null,
|
|
"file": {
|
|
"min_heading": 2,
|
|
"max_heading": 6,
|
|
"sectionize": false,
|
|
"ignore_files": [],
|
|
"*/rules.md": {"max_heading": 2}
|
|
}
|
|
}
|
|
},
|
|
"json": {
|
|
"indent": "space",
|
|
"indent_len": 2,
|
|
"trailing_commas": false,
|
|
"require_key_order": false,
|
|
"check_duplicate_keys": true,
|
|
"extensions": [".json"]
|
|
},
|
|
"python": {"ruff": {}},
|
|
"fiona": {
|
|
"log_level": "INFO",
|
|
"file_log_level": "INFO",
|
|
"file_log_location": "fiona.log",
|
|
"llm-wiki": {"ignore_files": []},
|
|
"wiki": {
|
|
"graph": {
|
|
"show_external_links": false,
|
|
"subtitle": {"enabled": true},
|
|
"show_legend": true,
|
|
"theme": "night-sky",
|
|
"show_arrows": false,
|
|
"show_source": true,
|
|
"show_glow": true,
|
|
"show_link_card": true,
|
|
"first_fit_ms": 12000,
|
|
"hover_link_tail_chars": 12,
|
|
"simplify_external_links": false,
|
|
"consolidate_external_links": false,
|
|
"consolidate_external_links_depth": 0,
|
|
"consolidate_external_links_max_char": 8,
|
|
"ignore_files": ["*/table_of_contents.md"]
|
|
},
|
|
"update": {
|
|
"ignore_files": ["corefiles.json"]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
| Key | Type | Default when absent | Consumer |
|
|
| --- | --- | --- | --- |
|
|
| `markdown.pymarkdown.extensions` | list of strings | none enabled | mdoctor |
|
|
| `markdown.pymarkdown.respect_gitignore` | bool | false | mdoctor |
|
|
| `markdown.pymarkdown.<rule>.<prop>` | bool, int, or str | pymarkdown default | mdoctor |
|
|
| `markdown.pymarkdown.path_rule_exemptions` | list of `{path, rules}` | `[]` | wiki sanity markdownlint |
|
|
| `markdown.frontmatter.required_fields` | list of strings | `["name", "version"]` | fmlint |
|
|
| `markdown.frontmatter.type_requirements` | object of lists | `{}` | fmlint |
|
|
| `markdown.frontmatter.enabled` | bool | true | fmlint |
|
|
| `markdown.frontmatter.ignore_rules` | list of codes | `[]` | fmlint |
|
|
| `markdown.frontmatter.ignore_file` | list of globs | `[]` | fmlint |
|
|
| `markdown.frontmatter.respect_gitignore` | bool | true | fmlint |
|
|
| `markdown.toc_generator.max_depth_from_file` | int | unlimited (`0` or absent) | folder TOC |
|
|
| `markdown.toc_generator.max_depth_from_dir` | int or null | unlimited | folder TOC |
|
|
| `markdown.toc_generator.file.min_heading` | int `>= 2` | 2 | in-file TOC |
|
|
| `markdown.toc_generator.file.max_heading` | int `>= min` | 6 | in-file TOC |
|
|
| `markdown.toc_generator.file.sectionize` | bool | false | in-file TOC |
|
|
| `markdown.toc_generator.file.ignore_files` | list of globs | `[]` | in-file TOC |
|
|
| `markdown.toc_generator.file.<glob>` | object of file keys | none | per-file override |
|
|
| `json.indent` | `"space"` or `"tab"` | `"space"` | json-lint |
|
|
| `json.indent_len` | int | 2 | json-lint |
|
|
| `json.trailing_commas` | bool | false | json-lint |
|
|
| `json.require_key_order` | bool | false | json-lint |
|
|
| `json.check_duplicate_keys` | bool | true | json-lint |
|
|
| `json.extensions` | list of strings | `[".json"]` | json-lint |
|
|
| `json.max_line_length` | int or absent | no line-length check | json-lint |
|
|
| `python.ruff` | object | `{}` | reserved, unused |
|
|
| `fiona.log_level` | level name | `WARNING` | logging |
|
|
| `fiona.file_log_level` | level name | `fiona.log_level` | file log |
|
|
| `fiona.file_log_location` | string or absent | no file log | file log |
|
|
| `fiona.llm-wiki.ignore_files` | list of globs | copy all | wiki init, update |
|
|
| `fiona.wiki.update.ignore_files` | list of globs | `[]` | wiki update, housekeeping merge |
|
|
| `fiona.wiki.graph.wikilinks_enabled` | bool | true | wiki graph |
|
|
| `fiona.wiki.graph.max_files` | int | 5000 | wiki graph |
|
|
| `fiona.wiki.graph.show_external_links` | bool | false | wiki graph toggle |
|
|
| `fiona.wiki.graph.subtitle.enabled` | bool | true | wiki graph toggle |
|
|
| `fiona.wiki.graph.show_legend` | bool | true | wiki graph toggle |
|
|
| `fiona.wiki.graph.show_arrows` | bool | true | wiki graph toggle |
|
|
| `fiona.wiki.graph.show_source` | bool | true | wiki graph toggle |
|
|
| `fiona.wiki.graph.show_glow` | bool | true | wiki graph |
|
|
| `fiona.wiki.graph.show_link_card` | bool | true | wiki graph |
|
|
| `fiona.wiki.graph.first_fit_ms` | number | 12000 | wiki graph |
|
|
| `fiona.wiki.graph.hover_link_tail_chars` | int | 12 | wiki graph |
|
|
| `fiona.wiki.graph.theme` | string | `night-sky` | wiki graph |
|
|
| `fiona.wiki.graph.simplify_external_links` | bool | false | wiki graph |
|
|
| `fiona.wiki.graph.consolidate_external_links` | bool | false | wiki graph |
|
|
| `fiona.wiki.graph.consolidate_external_links_depth` | int | follow the bool | wiki graph |
|
|
| `fiona.wiki.graph.consolidate_external_links_max_char` | int | 8 | wiki graph |
|
|
| `fiona.wiki.graph.ignore_files` | list of globs | `[]` | wiki graph |
|
|
| `fiona.wiki.graph.node_importance_min_point` | int `>= 0` | 0 | wiki graph |
|
|
| `fiona.wiki.graph.directory_nodes.min_files` | int `>= 0` | 0 (off) | wiki graph |
|
|
| `fiona.wiki.graph.directory_nodes.outside_depth` | int `>= 0` | 1 | wiki graph |
|
|
| `fiona.wiki.graph.serve.host` | string | `127.0.0.1` | wiki graph `--serve` |
|
|
| `fiona.wiki.graph.serve.port` | int `0..65535` | 8765 | wiki graph `--serve` |
|
|
| `fiona.wiki.graph.serve.poll_seconds` | number `> 0` | 1 | wiki graph `--serve` |
|
|
| `fiona.wiki.graph.serve.debounce_ms` | int `>= 0` | 300 | wiki graph `--serve` |
|
|
| `fiona.wiki.ingest.timeout_seconds` | number `> 0` | 30 | wiki ingest |
|
|
| `fiona.wiki.ingest.max_download_bytes` | int `> 0` | 52428800 | wiki ingest |
|
|
| `fiona.wiki.ingest.user_agent` | string | `fiona-assistant/<package version>` | wiki ingest |
|
|
| `fiona.wiki.ingest.default_target` | string | `sources` | wiki ingest |
|
|
| `fiona.wiki.ingest.raw_categories` | object of strings | `{}` | wiki ingest |
|
|
| `fiona.wiki.ingest.login_url_patterns` | list of regex strings | see below | wiki ingest |
|
|
|
|
Bool pymarkdown properties use `set_boolean_property`, ints use
|
|
`set_integer_property`, strings use `set_string_property`, applied as
|
|
`plugins.<rule_id>.<property>`. `"enabled": false` inside a rule map
|
|
disables that rule.
|
|
|
|
`markdown-tables` stays off. pymarkdownlnt 0.9.40 raises
|
|
`BadTokenizationError` on list lines that contain pipes, which breaks
|
|
sanity on generated contents files. Until that is fixed, table rules
|
|
MD055, MD056, MD058, and MD060 do not run. MD060 is also disabled
|
|
because pymarkdown's delimiter-width check disagrees with Node
|
|
markdownlint compact tables. Project tables use compact style:
|
|
one space around cell text, including the delimiter row
|
|
(`| --- | ------- |`).
|
|
|
|
Graph colors, sizes, and shapes come from section 5.3. If
|
|
`node_colors`, `node_radius`, or `hover` is still present here, those
|
|
keys override the `colorful` theme only. Hover radius is
|
|
`min(current * size_multiplier, max)` and returns to the pre-hover
|
|
radius when the pointer leaves. `subtitle.enabled` starts the
|
|
Show Subtitles toggle. The
|
|
label is the frontmatter `name` when that field is a non-empty
|
|
string, otherwise the filename. `show_legend` starts the Show
|
|
Legend toggle. The default is on. The card sits at the upper
|
|
right of the graph. It groups shape, node size and color, and
|
|
edge color, so those three are not one mixed list.
|
|
The graph command starts from the graph section of the packaged
|
|
`fiona-settings.json`. A `fiona-settings.json` found by walking up
|
|
from the scanned directory replaces the keys it sets. A missing
|
|
file still uses that packaged section.
|
|
`show_arrows` starts the Show Arrows toggle. The default is on.
|
|
`show_source` starts the Show Sources toggle. The default is on.
|
|
It draws each frontmatter `source` value as its own node.
|
|
`show_glow` draws the glow on a hovered node, a clicked node,
|
|
and a search match. The default is on. `show_link_card` draws
|
|
the hover card beside the node. The default is on. A click
|
|
shows the frontmatter card at the upper left, including when
|
|
`show_link_card` is false. `first_fit_ms` is the first node's
|
|
fit, in milliseconds. The shipped value, and the value when
|
|
the key is absent or not a positive number, is 12000. See all,
|
|
the last node, and a theme change stay at 4200 ms.
|
|
`hover_link_tail_chars` is how many characters of a URL's last
|
|
path segment the hover card keeps. The shipped value, and the
|
|
value when the key is absent or not a positive number, is 12.
|
|
The arrowhead sits on the target end of a visible edge, so the
|
|
source file points at the target file. Turning the toggle off
|
|
removes the arrowheads. `theme` names the theme the page opens
|
|
on. The shipped value is `night-sky`. An unknown name also opens
|
|
`night-sky`. `simplify_external_links` shortens an external
|
|
subtitle to the host. The default is off.
|
|
`consolidate_external_links` draws one node per host, named
|
|
`host/*`, when depth is absent. The default of the bool is off.
|
|
`consolidate_external_links_depth` is how many levels to keep.
|
|
`0` leaves each URL alone. `1` is `host/*`. `2` is `host/a/*`.
|
|
`3` is `host/a/b/*`. A missing depth follows the bool: true is
|
|
`1`, false is `0`. A path part longer than
|
|
`consolidate_external_links_max_char` is `...`. That limit is
|
|
`8` when absent or not a positive integer. Turning depth above
|
|
`0` also shortens the subtitle to the host.
|
|
`node_importance_min_point` is how many different files must
|
|
point at a node before the HTML graph draws it. The count is
|
|
`unique_points` (*7.1.4.2.3.*). `0` turns the filter off and
|
|
every node is drawn. A missing key, or a value that is not an
|
|
integer of 0 or more, is `0`. A positive integer is *7.1.4.16.*
|
|
and is not built yet. The packaged `fiona-settings.json` omits
|
|
the key until that requirement is built. Absent means `0`.
|
|
`directory_nodes.min_files` is how many markdown pages a
|
|
directory must hold directly before the HTML graph draws it as
|
|
one directory node (*7.1.4.18.*). `0` turns consolidation off.
|
|
`directory_nodes.outside_depth` is how many hops past a
|
|
directory's own pages its view draws (*7.1.4.18.5.*).
|
|
`serve.host`, `serve.port`, `serve.poll_seconds`, and
|
|
`serve.debounce_ms` are the `--serve` defaults (*7.1.4.19.*).
|
|
A CLI flag wins over these keys. A value that is absent, of the
|
|
wrong type, or out of range uses the default in the table.
|
|
Both are not built yet. The packaged `fiona-settings.json`
|
|
omits `directory_nodes` and `serve` until they are built.
|
|
|
|
`fiona.wiki.ingest` is read by section 7.1.5. `default_target` is a
|
|
directory relative to `wiki/`. `raw_categories` maps a converter
|
|
name to a directory under `raw/`, such as `{"text": "transcripts"}`.
|
|
A value that is absent, of the wrong type, or out of range uses the
|
|
default in the table. The packaged `fiona-settings.json` omits
|
|
`fiona.wiki.ingest`. Absent keys use the defaults in the table.
|
|
`login_url_patterns` is read by *7.1.5.2.8.3.* and is not built
|
|
yet. A list replaces the default list, it does not extend it. An
|
|
entry that does not compile is skipped with one warning. The
|
|
default list:
|
|
|
|
```json
|
|
[
|
|
"/(login|logon|signin|sign-in)\\b",
|
|
"/(sso|saml2?|oauth2?|adfs|cas)(/|$)",
|
|
"login\\.microsoftonline\\.com",
|
|
"j_security_check"
|
|
]
|
|
```
|
|
|
|
A mapping key under `toc_generator.file` whose name is not one of
|
|
`min_heading`, `max_heading`, `sectionize`, `ignore_files` is an
|
|
fnmatch override against the file path relative to the wiki root.
|
|
|
|
On `wiki update`, a differing `fiona-settings.json` is not
|
|
rewritten (*7.1.2.3.*). The template is written beside it as
|
|
`fiona-settings-update.json`. The owner merges that file: start
|
|
from the template object. For each user key, if both sides are
|
|
objects, merge recursively; otherwise the user value replaces the
|
|
template value. New template keys that the user does not set are
|
|
kept. Write the result as JSON with indent 2 and a trailing
|
|
newline, then delete the update file.
|
|
|
|
- *5.1.1.* Changing `md013.line_length` changes the next mdoctor scan
|
|
with no code change.
|
|
- *acceptance*: A line between the old and new limit is clean after
|
|
the settings change and reported before it.
|
|
- *5.1.2.* `wiki update` leaves `fiona-settings.json` byte-for-byte
|
|
unchanged and writes the template as `fiona-settings-update.json`.
|
|
- *acceptance*: `markdown.pymarkdown.md013.line_length` set to 120
|
|
is still 120 in the owner file after `wiki update --yes`, and
|
|
`fiona-settings-update.json` is the template copy.
|
|
|
|
### 5.2. settings_dev.json
|
|
|
|
```json
|
|
{
|
|
"pytests": {"required-coverage": 96},
|
|
"progress": {"interval": 3, "expected-seconds": 30}
|
|
}
|
|
```
|
|
|
|
| Key | Default when absent | Meaning |
|
|
| --- | --- | --- |
|
|
| `pytests.required-coverage` | (required for sanity) | Minimum total coverage percent |
|
|
| `progress.interval` | 3 | Seconds between non-interactive dots |
|
|
| `progress.expected-seconds` | 30 | Bar scale for a running check |
|
|
|
|
- *5.2.1.* Coverage below `required-coverage` fails sanity when every
|
|
test passed.
|
|
- *acceptance*: Raising the number in this file flips a green run
|
|
to failed with no code change.
|
|
|
|
### 5.3. constellation themes
|
|
|
|
Themes for the wiki graph live in `constellation/`. Each theme is
|
|
one JSON file. The file name without `.json` is the theme name.
|
|
`colorful` is complete and listed first. The other names follow
|
|
in alphabetical order. Any other theme may omit keys. A missing
|
|
key is taken from `colorful`. A `dark` object on a theme is the
|
|
same kind of partial theme, used when the page is in dark mode.
|
|
|
|
Each file has a `version` string (semver). Bump it when that
|
|
theme changes. `version` is not a color key. A theme that omits
|
|
it does not take `colorful`'s version.
|
|
|
|
`colorful` is a warm paper field. Nodes run from warm gray
|
|
through amber to crimson. Edges use that same warm ramp, darker
|
|
so a thin line stays visible. Hover, click, and buttons are one
|
|
teal, the complement of the amber. Pages are circles and
|
|
external links are clouds. `night-sky` draws
|
|
circle pages and star external links on a dark field.
|
|
`grayscale` runs charcoal to near-black on a light field, with
|
|
a dark stroke so the pale end still stands off the canvas. Its
|
|
`dark` object is a gray night on a charcoal field, not black.
|
|
Sparse nodes stay medium gray. A mid link count is white-yellow.
|
|
Many incoming links are a light yellow-orange. `dark` is a blue-slate night. Nodes
|
|
run from slate through white-yellow to a light gold. The page and the canvas
|
|
stay blue-gray, not black. `professional` is a light boardroom
|
|
palette: pale blue nodes, a soft blue stroke, light blue
|
|
buttons, and a quiet radius.
|
|
|
|
| Key | Meaning |
|
|
| --- | --- |
|
|
| `background` | Page background |
|
|
| `graph_background` | Graph canvas background |
|
|
| `font_family` | Font for the page |
|
|
| `text_color` | Text on the page and cards |
|
|
| `node_colors.low/mid/high` | Node color from few to many incoming links |
|
|
| `node_stroke` | Outline around a node, or `none` |
|
|
| `node_radius.min/max` | Smallest and largest node radius, in em. 16 is 1em. Fractions are kept |
|
|
| `subtitle.min_radius` | Label cutoff, a fraction of `node_radius.max` from 0 to 1 |
|
|
| `subtitle.padding` | Em between the shape and the label, where 16 is 1em. Smaller sits closer |
|
|
| `subtitle.size` | Label font size in em, where 16 is 1em |
|
|
| `subtitle.on_top` | When true, labels paint above every node |
|
|
| `subtitle.outline` | When true, a stroke keeps the label readable over nodes |
|
|
| `subtitle.outline_color` | Stroke color for that outline |
|
|
| `subtitle.outline_width` | Stroke width in em, where 16 is 1em |
|
|
| `edge_colors.low/mid/high` | Edge color from one link to many |
|
|
| `edge_width` | Edge thickness in em, where 16 is 1em. `1` when absent or not positive |
|
|
| `highlight.node_color` | Node color while the pointer is over it |
|
|
| `highlight.edge_color` | Edge color while a node is highlighted |
|
|
| `highlight.size_multiplier` | Radius multiplier while highlighted |
|
|
| `clicked.node_color` | Node color after a click |
|
|
| `buttons.background/text/hover/disabled` | Button colors |
|
|
| `card.background/border` | Cards, legend, and the control bar |
|
|
| `shapes.page` | `circle`, `cloud`, `star`, or `folder` |
|
|
| `shapes.external` | Shape for an external-link node |
|
|
| `shapes.directory` | Shape for a directory node (*7.1.4.18.*). `folder` in `colorful` |
|
|
| `subtitle_color` | Label under a large node |
|
|
| `version` | Semver of this theme file. Not a color key |
|
|
|
|
`subtitle.padding` is 12 when absent. `subtitle.size` is 11 when
|
|
absent or below 1. `subtitle.on_top` and `subtitle.outline` are
|
|
false when absent. `subtitle.outline_width` is 3 when absent.
|
|
`subtitle.outline_color` uses `graph_background` when absent.
|
|
A theme that sets only some of these keeps the rest from
|
|
`colorful`.
|
|
|
|
- *5.3.1.* A theme that omits `font_family` uses the `colorful`
|
|
theme font. The HTML theme control lists `colorful`, `night-sky`,
|
|
`grayscale`, `dark`, and `professional`.
|
|
- *acceptance*: The resolved grayscale theme font is the colorful
|
|
theme font. The graph HTML contains a theme select and those names.
|
|
- *5.3.2.* Subtitle gap, size, stacking, and outline come from the
|
|
active theme's `subtitle` object.
|
|
- *acceptance*: The resolved grayscale subtitle padding, size,
|
|
`on_top`, and outline match `colorful`. The graph HTML reads
|
|
those keys from the theme.
|
|
- *5.3.3.* Each theme is one JSON file in `constellation/`. The
|
|
name is the file stem. `colorful` is first in the catalog. The
|
|
other names follow in alphabetical order. Each file has a
|
|
`version` string. Filling a theme does not copy `version` from
|
|
`colorful`, in the loader or in the page script.
|
|
- *acceptance*: The template has `constellation/colorful.json`,
|
|
`constellation/night-sky.json`, `constellation/grayscale.json`,
|
|
`constellation/dark.json`, and `constellation/professional.json`,
|
|
and no `constellation-settings.json`. Each resolved packaged
|
|
theme `version` equals the `version` in its file. A theme
|
|
object that omits `version` has no `version` after it is
|
|
filled from `colorful`.
|
|
- *5.3.4.* A `constellation/` directory at the wiki, or in a
|
|
parent, overlays the packaged themes. Each JSON file there
|
|
overlays the theme of the same name. `constellation-settings.json`
|
|
in that same directory overlays those files. The nearest
|
|
directory that has either source wins. A theme file that is
|
|
not a JSON object is skipped.
|
|
- *acceptance*: `constellation/night-sky.json` setting
|
|
`background` changes that key and keeps the packaged page
|
|
shape. A `constellation-settings.json` beside it replaces
|
|
`background` with its own value. A file of `{` does not
|
|
remove `colorful`.
|
|
|
|
---
|
|
|
|
## 6. Violation Rule Catalog
|
|
|
|
### 6.1. Shared Report
|
|
|
|
> Status: Implemented
|
|
|
|
Custom linters and the ruff wrapper print this shape. Sanity runners
|
|
omit the report for a check with zero violations; the `Results:` row
|
|
still shows the count.
|
|
|
|
```text
|
|
<Linter Name> Violations
|
|
TARGET: <path>
|
|
|
|
- DEFINITIONS:
|
|
- <code>: <rule_name>, <description>
|
|
|
|
- FILE: <path>
|
|
- <message>
|
|
|
|
- SUMMARY:
|
|
- Total files with violations: <N>
|
|
- Total violations: <N>
|
|
```
|
|
|
|
A clean standalone run that still prints a report uses
|
|
`- No violations found.` in place of the FILE section and still
|
|
prints DEFINITIONS and SUMMARY. Headers:
|
|
|
|
| Tool | Header name |
|
|
| --- | --- |
|
|
| mdoctor | PyMarkdownlint |
|
|
| fmlint | Frontmatter Lint |
|
|
| python-ruff | Ruff |
|
|
| pycustomlint | Python Custom Lint |
|
|
| mdlinter | Markdown Lint |
|
|
| json-lint | JSONLint |
|
|
|
|
Frontmatter messages omit a `line:column` prefix. Frontmatter is
|
|
always at the start of the file.
|
|
|
|
- *6.1.1.* A report with one violation contains TARGET, DEFINITIONS,
|
|
one FILE entry, and SUMMARY, in that order.
|
|
- *acceptance*: The three custom linters and ruff use this order.
|
|
|
|
### 6.2. Frontmatter Fields and FML Codes
|
|
|
|
> Status: Implemented
|
|
|
|
Required on every markdown file: `name`, `version`. Required on every
|
|
page under `wiki/`: `confidence`, `llm`, `contested`, and
|
|
`contradictions`. The Fiona skill teaches how to set those four.
|
|
fmlint enforces `name` and `version` only. All other fields are
|
|
optional.
|
|
|
|
| Field | Required | Rule |
|
|
| --- | --- | --- |
|
|
| `name` | yes | Document name. Wiki pages use lowercase and hyphens. |
|
|
| `version` | yes | Semver. Bump when the page is edited. |
|
|
| `title` | no | Human title. |
|
|
| `created` | no | `YYYY-MM-DD` |
|
|
| `updated` | no | `YYYY-MM-DD`. Bump when the page is edited. |
|
|
| `type` | no | `entity`, `concept`, `comparison`, `query`, `summary`, `toc` |
|
|
| `tags` | no | Taxonomy tags from `wiki/schema.md` |
|
|
| `sources` | no | Paths such as `raw/articles/source-name.md` |
|
|
| `confidence` | wiki pages | `trusted`, `high`, `medium`, `low`, or `zero`. The agent sets it on every page under `wiki/`. fmlint does not enforce it. |
|
|
| `llm` | wiki pages | `model` is required. `reasoning` is set when the model reports a level. The agent sets it on every page under `wiki/`. fmlint does not enforce it. |
|
|
| `contested` | wiki pages | `true` when a claim on the page is disputed, otherwise `false`. The agent sets it on every page under `wiki/`. fmlint does not enforce it. |
|
|
| `contradictions` | wiki pages | Other page slugs that disagree. `[]` when none. The agent sets it on every page under `wiki/`. fmlint does not enforce it. |
|
|
| `description` | SKILL.md only | Required on a file whose name is `SKILL.md`. |
|
|
| `status` | no | `raw` or `refined`. `wiki ingest` writes `raw`. Only the agent sets `refined` (*7.1.5.10.*). |
|
|
| `ingest` | no | Map of provenance written by `wiki ingest`: origin, media type, SHA-256, size, converter, conversion, warnings (*7.1.5.5.3.*). |
|
|
| `source_meta` | no | Map of metadata read from the source, such as author and dates. `{}` when none. |
|
|
|
|
`markdown.frontmatter.type_requirements` maps a `type` value to the
|
|
field list required for that type, in addition to the global list.
|
|
|
|
| Code | Name | Message | Fails when | Passes when |
|
|
| --- | --- | --- | --- | --- |
|
|
| FML001 | Frontmatter Parse Error | `FML001 - <error>` | YAML between `---` markers is invalid | YAML parses |
|
|
| FML002 | Missing Frontmatter | `FML002` | No frontmatter block | A block is present |
|
|
| FML003 | Missing Name Field | `FML003` | `name` absent | `name` present |
|
|
| FML004 | Missing Description | `FML004` | `SKILL.md` has no `description` | `description` present |
|
|
| FML005 | Invalid Skill Name | `FML005` | `SKILL.md` `name` fails `^[a-z0-9-]+$` | `name: python-tdd` |
|
|
| FML006 | Skill Name Mismatch | `FML006 - (expected: <folder>, got: <name>)` | `name` differs from the parent folder | folder `python-tdd` and `name: python-tdd` |
|
|
| FML007 | Missing Version Field | `FML007` | `version` absent | `version` present |
|
|
|
|
`SKILL.md` is case-sensitive. FML005 and FML006 apply only to that
|
|
filename. Scan skips `.gitignore` matches when
|
|
`respect_gitignore` is true, `ignore_file` globs, and read-only
|
|
zones.
|
|
|
|
- *6.2.1.* A markdown file with `name` and `version` and no `title`
|
|
produces no FML violation.
|
|
- *acceptance*: fmlint on that file reports no violations.
|
|
- *6.2.2.* `SKILL.md` in `python-tdd/` with `name: Python TDD` is
|
|
FML005, and `name: other-name` is FML006.
|
|
- *acceptance*: The report code matches the case above.
|
|
- *6.2.3.* Every page under `wiki/` includes `confidence`, `llm`,
|
|
`contested`, and `contradictions`. The Fiona skill requires them
|
|
and states how to choose each value. `confidence` is `trusted`
|
|
(the user stated it, or it is copied from a cited source),
|
|
`high` (more than one source agrees, or one source and no known
|
|
contradiction), `medium` (one source, or a synthesis the agent
|
|
is reasonably sure of), `low` (inference, thin evidence, or an
|
|
open gap), or `zero` (unknown, or not checked). `llm` is a list
|
|
whose `model` entry is required: the model that last edited the
|
|
page, or `none` when no model wrote it. `reasoning` is included
|
|
when that model reports a level. `contested` is `true` when any
|
|
claim on the page is disputed by a source or another page, and
|
|
`false` otherwise. `contradictions` lists the `name` of each
|
|
page that disagrees, and is `[]` when none do. fmlint enforces
|
|
`name` and `version` only.
|
|
- *acceptance*: Fiona skill section 15 marks those four fields
|
|
required on wiki pages and states the choice rules above.
|
|
|
|
### 6.3. MDL Codes
|
|
|
|
> Status: Implemented
|
|
|
|
mdlinter scans `.md` files. Fenced code, inline code, links whose
|
|
target starts with `http://`, `https://`, `mailto:`, or `#`, and any
|
|
target containing `://`, are ignored. Sources under `personal/` and
|
|
`raw/` are skipped. Targets inside those zones are still resolved.
|
|
|
|
| Code | Name | Message | Fails when | Passes when |
|
|
| --- | --- | --- | --- | --- |
|
|
| MDL001 | Dead Link | `line <n>: <target> does not exist` | A relative `](target)` does not resolve to a file | The relative file exists |
|
|
| MDL002 | Dead Wiki Link | `line <n>: [[<target>]] does not resolve to any page` | A `[[wikilink]]` matches no markdown stem | A stem matches, case-insensitive |
|
|
| MDL003 | Wrapped Table Row | `line <n>: table row continues onto the next line; keep each row on one line` | A line starts with `\|`, does not end with `\|`, and the next line is not blank and not a new row. A following line that is only `\|` counts as a continuation | The row is one line, or the next line is blank or a new row, or the split is inside a fence |
|
|
|
|
Wikilink syntax: `[[page]]`, `[[page#heading]]`, `[[page|alias]]`.
|
|
The lookup key is the text before `#` or `|`, compared to markdown
|
|
file stems in the wiki, case-insensitive. Markdown links resolve
|
|
against the linking file's directory.
|
|
|
|
- *6.3.1.* A link to a deleted file is MDL001 with file, line, and
|
|
target. A `[[missing]]` wikilink is MDL002.
|
|
- *acceptance*: Both codes appear in one wiki that has one of each.
|
|
- *6.3.2.* A table row split across two lines is MDL003 on the opening
|
|
line. The same split inside a fence is clean.
|
|
- *acceptance*: Only the unfenced split is reported.
|
|
|
|
### 6.4. PYC Codes
|
|
|
|
> Status: Implemented
|
|
|
|
pycustomlint scans `.py` files with the AST, skips `.gitignore`
|
|
matches and read-only zones, and accepts `--json` for a JSON list of
|
|
violation objects.
|
|
|
|
| Code | Name | Severity | Message | Fails when | Passes when |
|
|
| --- | --- | --- | --- | --- | --- |
|
|
| PYC001 | Inline Import | error | `PYC001 - Inline import at line <n>: <stmt>` | `import` or `from` is indented inside a function, class, `try`, or an `if` that follows a function or class | Every import is at column 0, or it sits in a module-level `if` before the first function or class |
|
|
| PYC002 | Import After Code | warning | `PYC002 - Import after code at line <n>: <stmt>` | A module-level import follows a non-import statement | Imports are grouped before other module statements |
|
|
| PYC003 | Late Declaration | error | `PYC003 - Declaration after first function or class at line <n>: <stmt>` | A module-level assignment is at or after the first function or class in the module body | Every module-level assignment is before that first function or class |
|
|
|
|
- *6.4.1.* `import os` inside a function is PYC001.
|
|
- *acceptance*: The report names the file, the line, and `import os`.
|
|
- *6.4.2.* An import inside a module-level `if`, `elif`, or `else`
|
|
that appears before the first function or class is not PYC001.
|
|
The same import after that first function or class is PYC001.
|
|
A `try` does not get this exception. An import inside a function
|
|
or class is PYC001.
|
|
- *acceptance*: `if os.name == "nt": import msvcrt` before any
|
|
function is clean. The same `if` after a function is PYC001
|
|
and the message names `import msvcrt`.
|
|
- *acceptance*: `try: import os` before any function is PYC001.
|
|
- *6.4.3.* A module-level assignment at or after the first function
|
|
or class statement in the module body is PYC003. An assignment
|
|
nested inside a function or class is not PYC003. An assignment
|
|
before that first function or class, including one inside a
|
|
module-level `if`, is not PYC003.
|
|
- *acceptance*: `SHORT_HASH_LENGTH = 8` after a function is PYC003
|
|
and the message names `SHORT_HASH_LENGTH = 8`.
|
|
- *acceptance*: The same assignment before any function or class
|
|
is clean.
|
|
- *acceptance*: `value = 1` inside a function, and `VALUE = 1`
|
|
inside a class, are clean.
|
|
- *acceptance*: `VALUE = 1` inside a module-level `if` before any
|
|
function is clean. The same `if` after a function is PYC003.
|
|
|
|
### 6.5. JSN Codes
|
|
|
|
> Status: Implemented
|
|
|
|
json-lint parses with the standard `json` module. Style checks use
|
|
section 5.1. Duplicate-key detection uses a parse hook so the last
|
|
value does not hide the earlier key.
|
|
|
|
| Code | Name | Severity | Fails when | Passes when |
|
|
| --- | --- | --- | --- | --- |
|
|
| JSN001 | JSON Syntax Error | error | The file is not JSON, including a trailing comma when that is a syntax error | `json.loads` succeeds |
|
|
| JSN002 | Indentation Error | error | Indent differs from `indent` / `indent_len` | Indent matches |
|
|
| JSN003 | Trailing Comma | error | A trailing comma is present and `trailing_commas` is false | No trailing comma, or the setting allows them |
|
|
| JSN004 | Duplicate Keys | error | An object repeats a key and `check_duplicate_keys` is true | Keys are unique |
|
|
| JSN005 | Key Order | warning | `require_key_order` is true and keys are not sorted | Keys are sorted, or the setting is false |
|
|
| JSN006 | Line Too Long | warning | `max_line_length` is set and a line exceeds it | The key is absent, or every line fits |
|
|
|
|
- *6.5.1.* A file with a missing comma reports JSN001 with line and
|
|
column. A file with duplicate keys reports JSN004.
|
|
- *acceptance*: Both cases name the file and the code.
|
|
- *6.5.2.* Four-space indent against `indent_len: 2` is JSN002.
|
|
`json-lint fix` rewrites it to two spaces.
|
|
- *acceptance*: The fixed file is clean on the next `chklint`.
|
|
|
|
### 6.6. Base Linter Contract
|
|
|
|
> Status: Implemented
|
|
|
|
- *6.6.1.* Shared behavior for fmlint, pycustomlint, mdlinter, and
|
|
json-lint.
|
|
- *6.6.1.1.* Recursive file collection by extension.
|
|
- *6.6.1.2.* `.gitignore` and read-only-zone skipping.
|
|
- *6.6.1.3.* Settings load from `fiona-settings.json`.
|
|
- *6.6.1.4.* A violation record has filepath, line, column, code,
|
|
message, rule id, and severity, plus a dict form for JSON.
|
|
- *6.6.1.5.* Report assembly follows section 6.1. The header is the
|
|
linter name.
|
|
- *6.6.1.6.* The CLI is `chklint [directory]` and defaults to `.`.
|
|
- *6.6.2.* A new rule is a new checker used by one linter. It does
|
|
not require an edit to the shared collector. Codes are unique:
|
|
`FML`, `MDL`, `PYC`, `JSN`, plus pymarkdown `MD` codes and ruff
|
|
codes.
|
|
- *acceptance*: Existing FML and PYC tests still pass after a new
|
|
MDL checker is registered.
|
|
|
|
---
|
|
|
|
## 7. Feature Requirements
|
|
|
|
### 7.1. wiki
|
|
|
|
Requirements for the `wiki` subcommands.
|
|
|
|
#### 7.1.1. wiki init
|
|
|
|
> Status: Partially Implemented
|
|
|
|
Implements sections 3 and 4.7.1.
|
|
|
|
- *7.1.1.1.* Copy is 1:1 aside from `DATE_PLACEHOLDER`,
|
|
`ignore_files`, and paths matched by the repository `.gitignore`.
|
|
- *acceptance*: A pattern in `ignore_files` omits those paths and
|
|
copies every other template file.
|
|
- *acceptance*: A template file matched by the repository
|
|
`.gitignore` is not copied.
|
|
- *7.1.1.2.* Copied markdown keeps `version` and receives
|
|
`created` / `updated` dates where the template used the placeholder.
|
|
- *acceptance*: After init, those fields match `YYYY-MM-DD` for
|
|
the UTC day of the run.
|
|
- *7.1.1.3.* `wiki init` puts or appends `.gitignore` from the
|
|
repository `.gitignore`, plus `personal/` when that pattern is
|
|
missing. Lines already in the file stay.
|
|
- *acceptance*: After init, `.gitignore` contains `__pycache__/`
|
|
and `personal/`. A line `keep-me` written before init is still
|
|
in the file.
|
|
- *7.1.1.4.* `NOT_IMPLEMENTED` The CLI turns the refusal of *3.2.2.*
|
|
into one stderr line and exit 1 (*1.2.7.*):
|
|
`Error: Directory '<path>' already contains llm-wiki files:
|
|
<names>. Run fiona-assistant wiki update to refresh the template.`
|
|
The line is one line, even though it is wrapped here. `<names>` is
|
|
the comma-separated list from the `FileExistsError` message.
|
|
- *acceptance*: A second `wiki init` exits 1, stderr is exactly that
|
|
line, and no file changes.
|
|
|
|
#### 7.1.2. wiki update
|
|
|
|
> Status: Implemented
|
|
|
|
Implements sections 4.7.2 and 4.7.3.
|
|
|
|
- *7.1.2.1.* Markdown core files are compared as follows. An existing
|
|
markdown file is never overwritten.
|
|
- *7.1.2.1.1.* Both sides have `version`, and the template version is
|
|
greater: conflict (*7.1.2.11.*). Numeric dotted versions compare
|
|
segment by segment (`1.2.0` vs `1.10.0`). Any other string
|
|
compares as text.
|
|
- *7.1.2.1.2.* The wiki version is greater, or the versions are equal
|
|
and the body differs after expanding `DATE_PLACEHOLDER` in the
|
|
template: skip the file and report `locally modified`.
|
|
- *7.1.2.1.3.* Either version is missing: compare SHA-256
|
|
(*7.1.2.12.*). Equal hashes need nothing. Different hashes are a
|
|
conflict.
|
|
- *7.1.2.2.* Other core files compare SHA-256 (*7.1.2.12.*) and update
|
|
when the hash differs.
|
|
- *7.1.2.3.* `fiona-settings.json` uses the hash comparison of
|
|
*7.1.2.12.*. Different hashes are a conflict (*7.1.2.11.*). The
|
|
owner's file stays. The template is written as
|
|
`fiona-settings-update.json`. The tool does not merge the two.
|
|
- *acceptance*: After `wiki update --yes`, the owner settings
|
|
bytes are unchanged and `fiona-settings-update.json` matches
|
|
the template.
|
|
- *7.1.2.4.* Missing template directories outside `wiki/` and outside
|
|
read-only zones are created when they hold a file that update may
|
|
add. User files that are not in the template stay. Read-only zones
|
|
are never in the update set.
|
|
- *7.1.2.5.* A pristine init reports `Wiki is up to date.` and
|
|
`wiki update` writes nothing.
|
|
- *acceptance*: A second check-update still reports up to date.
|
|
- *7.1.2.6.* An older core markdown file is listed as a conflict.
|
|
After a confirmed update its bytes are unchanged and its update
|
|
file matches the template. A locally modified file is listed
|
|
under skipped and is not overwritten.
|
|
- *acceptance*: The modified file's bytes are unchanged after
|
|
`wiki update --yes`.
|
|
- *acceptance*: An `AGENTS.md` at `1.0.0` against a template at
|
|
`2.0.0` keeps its bytes after `wiki update --yes`, and
|
|
`AGENTS-update.md` holds the expanded template text.
|
|
- *7.1.2.7.* `--dry-run` prints the plan and leaves the tree unchanged.
|
|
- *acceptance*: File hashes before and after a dry run match.
|
|
- *7.1.2.8.* `--force` and `--yes` skip the prompt, including on a
|
|
non-TTY.
|
|
- *acceptance*: The command exits 0 on a pipe when `--yes` is set
|
|
and updates exist.
|
|
- *7.1.2.9.* A path under `wiki/` is never added, overwritten, or
|
|
listed as a change, an addition, a skipped file, or a directory to
|
|
add. A template page under `wiki/` that is newer than the wiki's
|
|
copy (*7.1.2.1.1.* or *7.1.2.1.3.*) is listed as a conflict, the same
|
|
as any other markdown core file.
|
|
- *acceptance*: A `wiki/index.md` older than the template keeps
|
|
its bytes after `wiki update --yes`, and `wiki/index-update.md`
|
|
holds the expanded template text.
|
|
- *acceptance*: A `wiki/index.md` newer than the template, or
|
|
missing, is absent from the check-update report.
|
|
- *7.1.2.10.* Update skips a path matched by the repository
|
|
`.gitignore` or by the wiki's `.gitignore`. Comment lines and
|
|
blank lines are not patterns. A pattern with no slash matches any
|
|
path segment. A pattern that ends with `/` matches that directory
|
|
and everything under it.
|
|
- *acceptance*: A template file
|
|
`utils/scripts/__pycache__/example.pyc` is absent from the
|
|
check-update report.
|
|
- *acceptance*: When the wiki `.gitignore` lists `example.py`, a
|
|
drifted `utils/scripts/example.py` is left unchanged and is
|
|
absent from the report.
|
|
- *7.1.2.11.* A conflict means the template copy and the owner's
|
|
copy both exist and the tool must not overwrite the owner. That
|
|
is a newer markdown file (*7.1.2.1.*) or a differing
|
|
`fiona-settings.json` (*7.1.2.3.*). Update yields to the owner:
|
|
it keeps the file and writes the template beside it. The LLM
|
|
merges the two during housekeeping (*7.2.*).
|
|
- *7.1.2.11.1.* The update file sits in the same directory. Its name
|
|
is the file stem, then `-update`, then the suffix:
|
|
`AGENTS.md` gives `AGENTS-update.md`, `wiki/schema.md` gives
|
|
`wiki/schema-update.md`, and `fiona-settings.json` gives
|
|
`fiona-settings-update.json`. Relative links in the template still
|
|
resolve from that directory.
|
|
- *acceptance*: A conflict on `wiki/schema.md` writes
|
|
`wiki/schema-update.md` and `corefiles.json`, and no
|
|
other file.
|
|
- *7.1.2.11.2.* The update file holds the template text with
|
|
`DATE_PLACEHOLDER` expanded, as `wiki init` would write it. An
|
|
existing update file is replaced with the current template
|
|
text.
|
|
- *acceptance*: A second `wiki update --yes` leaves one update
|
|
file, with the same bytes as the first run on the same day.
|
|
- *7.1.2.11.3.* Only a confirmed update writes an update file.
|
|
`check-update` and `--dry-run` list the conflict and write
|
|
nothing.
|
|
- *acceptance*: No `*-update.md` exists after
|
|
`wiki update --dry-run`.
|
|
- *7.1.2.11.4.* A conflict clears when the owner's `version` is at
|
|
least the template's version. The merge sets that version and
|
|
deletes the update file.
|
|
- *acceptance*: After `AGENTS.md` is set to the template version
|
|
with the owner's body, check-update lists no conflict for it.
|
|
- *7.1.2.12.* A file without version control, meaning a markdown file
|
|
with no `version` on either side or any non-markdown file, is
|
|
compared by SHA-256. The wiki side and the template side both use
|
|
the text hash in *7.1.2.13.*. The template side expands
|
|
`DATE_PLACEHOLDER` before that hash. A binary file hashes its raw
|
|
bytes.
|
|
- *7.1.2.12.1.* Equal hashes: the file is up to date and is not
|
|
listed.
|
|
- *acceptance*: A freshly initialized wiki reports
|
|
`Wiki is up to date.` even when a template file holds
|
|
`DATE_PLACEHOLDER`.
|
|
- *7.1.2.12.2.* Different hashes: the report line shows both hashes
|
|
where a version pair would go, shortened to 8 hex digits:
|
|
`(sha256 1a2b3c4d -> 9f8e7d6c)`. `--json` carries the full
|
|
64-digit `current_sha256` and `template_sha256`. A file compared
|
|
by version has `null` in both.
|
|
- *acceptance*: A drifted `utils/scripts/example.py` is listed as
|
|
`utils/scripts/example.py (sha256 <8 hex> -> <8 hex>)`, and a
|
|
markdown file with no `version` that differs is a conflict with
|
|
the same parenthesis.
|
|
- *7.1.2.13.* The SHA-256 of a UTF-8 text file is the digest of that
|
|
text after every `\r\n` and every lone `\r` is folded to `\n`,
|
|
then encoded as UTF-8. Two text files that differ only in line
|
|
endings are equal. A file that is not valid UTF-8 hashes its raw
|
|
bytes, with no folding.
|
|
- *acceptance*: `fiona-settings.json` saved with CRLF against a
|
|
template saved with LF, with the same characters, is not a
|
|
conflict and is absent from the check-update report.
|
|
- *7.1.2.14.* A confirmed `wiki update` that had a file to update,
|
|
add, or conflict writes `corefiles.json` at the wiki root.
|
|
`check-update` and `--dry-run` do not write it. A pristine wiki
|
|
writes nothing (*7.1.2.5.*). The file is not part of the template
|
|
and is never itself an update, a conflict, an addition, or a
|
|
merge candidate (*7.2.3.*). Only this command writes it. It is
|
|
one JSON object:
|
|
|
|
```json
|
|
{
|
|
"files": {
|
|
"<relative posix path>": {
|
|
"version": "<owner version or null>",
|
|
"template_version": "<template version or null>",
|
|
"sha256": "<64 hex>",
|
|
"template_sha256": "<64 hex>",
|
|
"date": "YYYY-MM-DD"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Each entry is a core file the update compared, using that file's
|
|
bytes after the update. `date` is the UTC day the entry was
|
|
written. A later check does not list a core file while its owner
|
|
version, template version, owner hash, and template hash still
|
|
match that entry, including a second update on the same day. The
|
|
date does not expire the entry. A missing record, or a record that
|
|
is not this object, acknowledges nothing. When the owner file or
|
|
the template changes, the file is listed again.
|
|
- *acceptance*: After `wiki update --yes` on a drifted
|
|
`fiona-settings.json`, `corefiles.json` names
|
|
`fiona-settings.json` with `version` null, both 64-digit hashes,
|
|
and the UTC date. A second check-update does not list that file.
|
|
Editing the owner file lists it again.
|
|
- *acceptance*: `wiki update --dry-run` does not create
|
|
`corefiles.json`.
|
|
- *7.1.2.15.* `fiona.wiki.update.ignore_files` is a list of glob
|
|
patterns in the wiki's `fiona-settings.json`. A path that matches
|
|
is not an update, a conflict, an addition, or a merge candidate.
|
|
`wiki update` does not copy or replace it. A missing key is an
|
|
empty list. A value that is not a list of strings raises
|
|
`ValueError` with the text
|
|
`fiona.wiki.update.ignore_files must be a list of strings`.
|
|
The shipped list is `corefiles.json`. That path is the record in
|
|
*7.1.2.14.* and stays out of the candidate set even when the list
|
|
omits it. A confirmed update still writes the record.
|
|
- *acceptance*: A drifted `utils/scripts/example.py` whose path is
|
|
in the list is absent from check-update and unchanged after
|
|
`wiki update --yes`.
|
|
- *acceptance*: A template `corefiles.json` that differs from the
|
|
wiki copy is absent from check-update, and `wiki update --yes`
|
|
leaves the wiki copy unchanged when nothing else needs an update.
|
|
- *acceptance*: A non-list `ignore_files` makes check-update raise
|
|
`ValueError` with that text.
|
|
|
|
#### 7.1.3. wiki sanity
|
|
|
|
> Status: Implemented
|
|
|
|
Implements section 4.7.4. Markdown lint uses the wiki's own
|
|
`fiona-settings.json`. Dead links are MDL001 and MDL002 inside
|
|
mdlinter, not a separate check. New checks are added by registering
|
|
another `(name, runner)` pair in the wiki sanity list.
|
|
|
|
- *7.1.3.1.* A clean wiki exits 0. Any violation exits 1.
|
|
- *acceptance*: One dead link makes `passed` false in `--json`
|
|
and produces a non-zero exit.
|
|
- *7.1.3.2.* `--json` output contains only JSON.
|
|
- *acceptance*: The stdout of a `--json` run parses with
|
|
`json.loads` and contains no ANSI sequences.
|
|
- *7.1.3.3.* Human output uses the progress display in *7.14.2.*
|
|
with label `Running Wiki Sanity`.
|
|
- *acceptance*: A piped run prints that label and dots, and no
|
|
ANSI escapes.
|
|
- *7.1.3.4.* `markdown.pymarkdown.path_rule_exemptions` omits
|
|
matching markdownlint hits from wiki sanity only. Each entry has
|
|
`path` (glob relative to the wiki root) and `rules` (pymarkdown
|
|
rule ids). Other sanity checks still scan those paths.
|
|
- *acceptance*: A file under `tasks/` with an MD013 violation and
|
|
`{"path": "tasks/**", "rules": ["MD013"]}` in settings reports
|
|
zero markdownlint violations; without the entry it fails.
|
|
|
|
#### 7.1.4. wiki graph
|
|
|
|
> Status: Partially Implemented
|
|
|
|
*7.1.4.16.*, *7.1.4.17.*, *7.1.4.18.*, and *7.1.4.19.* are
|
|
planned. Every other requirement in this section is built.
|
|
|
|
The renderer is D3 v7, loaded from `https://d3js.org/d3.v7.min.js`,
|
|
with the visualization embedded in one HTML file. vis.js is not
|
|
used.
|
|
|
|
- *7.1.4.1.* The parser extracts links as follows.
|
|
- *7.1.4.1.1.* Markdown links `[text](path)`, including `./` and
|
|
`../`, resolve against the source file and are stored relative
|
|
to the wiki root. A path without an extension is treated as
|
|
`.md`.
|
|
- *7.1.4.1.2.* Wikilinks `[[file]]` and `[[dir/file]]` are parsed
|
|
when wikilinks are enabled.
|
|
- *7.1.4.1.3.* External URLs (`http://`, `https://`, `ftp://`, and
|
|
any `://`) are recorded. The HTML graph includes each URL as
|
|
a node. JSON lists them in metadata only with
|
|
`--external-links` or `external_links` in settings.
|
|
- *7.1.4.1.4.* A link whose target file is missing is recorded as
|
|
broken and does not abort the run.
|
|
- *7.1.4.2.* The graph is built as follows.
|
|
- *7.1.4.2.1.* A node is a markdown file. An edge is one or more
|
|
links from source to target. Weight is the link count. Each
|
|
markdown node stores that file's full frontmatter object.
|
|
- *7.1.4.2.2.* An orphan has in-degree 0 and out-degree 0.
|
|
- *7.1.4.2.3.* Each node stores `unique_points`, `total_points`,
|
|
and `link_score`. `unique_points` is how many different files
|
|
point at the node. `total_points` is the sum of those weights.
|
|
One source with weight w scores `2 * (1 - 0.5^w)`.
|
|
Ten links from one file score less than two files that each
|
|
link once.
|
|
- *7.1.4.3.* `--format json` and `--json` emit this object.
|
|
|
|
```json
|
|
{
|
|
"metadata": {
|
|
"wiki_root": "<path>",
|
|
"file_count": 1,
|
|
"link_count": 0,
|
|
"generated_at": "<UTC ISO-8601>"
|
|
},
|
|
"nodes": [
|
|
{
|
|
"id": "wiki/a.md",
|
|
"name": "a.md",
|
|
"in_degree": 0,
|
|
"out_degree": 0,
|
|
"unique_points": 0,
|
|
"total_points": 0,
|
|
"link_score": 0,
|
|
"frontmatter": {"name": "A", "version": "1.0.0"}
|
|
}
|
|
],
|
|
"edges": [
|
|
{"source": "wiki/a.md", "target": "wiki/b.md", "weight": 2}
|
|
],
|
|
"orphans": ["wiki/a.md"]
|
|
}
|
|
```
|
|
|
|
- *7.1.4.4.* When external URLs were collected, `metadata` also has
|
|
`external_links_count` (total occurrences) and `external_links`
|
|
(unique URL strings).
|
|
- *7.1.4.5.* HTML behavior.
|
|
- *7.1.4.5.1.* One self-contained file. The layout fills the
|
|
viewport and follows resize. The page script is
|
|
`templates/graph_physics.js`. Generation pastes that file
|
|
into the page. The saved HTML does not request it separately.
|
|
- *acceptance*: Generated HTML contains `function frameNodes`
|
|
and does not contain the paste marker.
|
|
- *7.1.4.5.2.* Node radius scales with `link_score` from `min` to
|
|
`max` in section 5.3. `min` and `max` are lengths in em, where
|
|
16 is 1em, and may be fractions. The drawn radius keeps that
|
|
fraction, so the sizes between min and max are not snapped to
|
|
a whole unit. Color interpolates low to mid over the
|
|
first half of `link_score` divided by the max score, then
|
|
mid to high. A score of 0 uses the low color and the min
|
|
radius. Two files that each link once outrank one file that
|
|
links ten times.
|
|
- *7.1.4.5.3.* Edge thickness is `edge_width` from the active
|
|
theme, in em, where 16 is 1em. It is 1 when that key is absent or not a
|
|
positive number. Changing the theme uses that theme's width.
|
|
Color runs from gray (weight 1) through yellow to red
|
|
(maximum weight).
|
|
- *7.1.4.5.4.* Hover: the node uses the hover color and the capped
|
|
larger radius. Connected edges use the hover edge color.
|
|
Other visible edges dim. Leaving the node restores its previous
|
|
radius and color and restores the dimmed edges. Hover does not
|
|
clear a click highlight. While both are active, edges of the
|
|
hovered node and edges of the clicked node stay highlighted.
|
|
Hover fixes the node in place the same way a click does.
|
|
Leaving the node releases it, unless that node is the clicked
|
|
one. A card appears beside the node. A line with an arrow
|
|
points from the card to the node, the same way an edge points
|
|
at its target. The card has two lines. The first line is the
|
|
node name. The second line is
|
|
`unique points:total points:in:out`. A page name stays as
|
|
the filename. An `http` or `https` link drops the scheme.
|
|
A path with more than one segment shows `host/.../last`.
|
|
When that last segment is longer than
|
|
`hover_link_tail_chars`, the card shows `host/...` plus the
|
|
last that many characters, with no slash before them. Width
|
|
follows the longer line. Height follows the two lines.
|
|
The card is a body in the layout. A link keeps it near the
|
|
node. It repels other nodes, and they repel it. The card stays
|
|
inside the graph. Leaving the node hides that card. It does
|
|
not hide a click card.
|
|
Hover uses that glow when `show_glow` is true or absent.
|
|
`show_glow` false draws no glow on hover, click, or a search
|
|
match. The hover card is shown when `show_link_card` is true
|
|
or absent. `show_link_card` false draws no hover card. It
|
|
does not change the click card.
|
|
- *7.1.4.5.5.* Single click toggles a persistent highlight and
|
|
shows that file's frontmatter card at the upper left of the
|
|
graph. This card is not the hover card. It lists every
|
|
frontmatter field and its value, then unique points, total
|
|
points, in-degree, and out-degree. A file with no frontmatter
|
|
shows an empty field list. Another node replaces the card.
|
|
Empty space hides it. Hover does not. `show_link_card` does
|
|
not hide it.
|
|
The clicked node carries the same glow as a search match
|
|
when `show_glow` is true or absent. Clearing the highlight
|
|
removes that glow. A node that is still a search match keeps
|
|
the glow when glow is on.
|
|
The clicked node is fixed where it is. No other node is fixed.
|
|
Connected nodes keep moving under the simulation, so their
|
|
edges follow. Clearing the highlight releases the node.
|
|
Dragging that node leaves it fixed at the drop point.
|
|
- *7.1.4.5.6.* Double click opens the markdown file. A double
|
|
click on a directory node opens that directory's view
|
|
instead (*7.1.4.18.4.*).
|
|
- *7.1.4.5.7.* Search highlights matching nodes in light blue at
|
|
1.5 times radius (capped at max), shows neighbors in gray, and
|
|
hides nodes that are not reachable from a match. A match also
|
|
carries a glow when `show_glow` is true or absent. Neighbors
|
|
and a cleared search do not. `show_glow` false draws no glow.
|
|
- *7.1.4.5.8.* A checkbox toggle labeled "Show Orphan Nodes" shows
|
|
or hides orphans. It starts checked. The label does not change.
|
|
- *7.1.4.5.9.* Overwrite of an existing output file asks for
|
|
confirmation on a TTY.
|
|
- *7.1.4.5.9.1.* `NOT_IMPLEMENTED` `-f/--force` skips that
|
|
prompt. Without a TTY and without `--force`, the file is still
|
|
overwritten and one `WARNING` record names it. The housekeeping
|
|
graph step (*7.2.6.*) passes `--force`, because
|
|
`wiki_graph.html` is the one file it owns (*7.2.7.*), and logs
|
|
that overwrite at `INFO`.
|
|
- *acceptance*: Housekeeping on a wiki that already has
|
|
`wiki_graph.html`, at `LOG_LEVEL=WARNING`, writes no
|
|
`Overwriting` line on stderr.
|
|
- *7.1.4.5.10.* Files above `max_files` are skipped after a warning.
|
|
The first `max_files` files are drawn.
|
|
- *7.1.4.5.11.* The run completes in under 10 seconds for 1000 files
|
|
on a current developer machine.
|
|
- *7.1.4.5.12.* A checkbox toggle labeled "Show External Links"
|
|
shows or hides external-link nodes and their edges. It starts
|
|
checked when `show_external_links` in section 5.1 is true, and
|
|
unchecked when that key is absent or false. The label does not
|
|
change. When it starts checked, those nodes and edges are drawn
|
|
on the opening fade. A click is not required. An external node
|
|
is the text cloud character. A page node stays a circle.
|
|
- *7.1.4.5.13.* A checkbox toggle labeled "Show Subtitles" shows or
|
|
hides labels under nodes. It starts checked when
|
|
`subtitle.enabled` in section 5.1 is true or absent, and
|
|
unchecked when that key is false. The label does not change.
|
|
A node shows its label when its drawn radius is at least
|
|
`subtitle.min_radius` from the active theme times that theme's
|
|
`node_radius.max`. The fraction is 0.5 when absent, and a
|
|
value outside 0 to 1 is clamped. Changing the theme uses that
|
|
theme's fraction and max. The text is the frontmatter `name`
|
|
when that field is a non-empty string, otherwise the filename.
|
|
The label sits under the shape. The gap is `subtitle.padding`
|
|
em from the active theme, where 16 is 1em, 12 when absent. A
|
|
smaller number sits closer. The font size is `subtitle.size`
|
|
on that same scale, 11 when absent or below 1. `subtitle.on_top`
|
|
true draws every label after the nodes, so another node does
|
|
not cover it. Absent or false keeps the label inside its node.
|
|
`subtitle.outline` true strokes the letters with
|
|
`subtitle.outline_width` on that same scale, 3
|
|
when absent, in `subtitle.outline_color`. Absent color uses
|
|
`graph_background`. Absent or false draws no stroke. Changing
|
|
the theme applies that theme's subtitle settings.
|
|
- *7.1.4.5.14.* A checkbox toggle labeled "Show Legend" shows or
|
|
hides a card at the upper right of the graph. It starts
|
|
checked when `show_legend` in section 5.1 is true or absent,
|
|
and unchecked when that key is false. The label does not
|
|
change. The card has three groups. Shape shows a page and an
|
|
external link in the text color. Nodes are growing circles:
|
|
few, some, and many files pointing here. Edges are lines:
|
|
linked once, a few times, and many times. Node circles use
|
|
the theme node colors. Edge lines use the theme edge colors.
|
|
- *7.1.4.5.15.* A select labeled "Theme" lists every theme from
|
|
section 5.3 and starts on `theme` from section 5.1.
|
|
That name is `night-sky` when the key is absent. Choosing a
|
|
theme updates the page background, font, buttons, cards, node
|
|
colors, node sizes, edge colors, highlight colors, the
|
|
clicked-node color, and node shapes. Choosing a theme also
|
|
runs the See all fit, so the shown nodes are centered again.
|
|
The grayscale dark-mode switch runs that fit too. The theme
|
|
applied at load does not. A missing theme key uses
|
|
`colorful`.
|
|
`grayscale` uses its `dark` object when the page is in dark mode.
|
|
- *7.1.4.5.16.* A checkbox toggle labeled "Show Arrows" draws or
|
|
hides an arrowhead on each visible edge. It starts checked
|
|
when `show_arrows` in section 5.1 is true or absent, and
|
|
unchecked when that key is false. The label does not change.
|
|
The arrowhead meets the target shape. The source file points
|
|
at the target file. The arrow uses that edge's color. A
|
|
hidden edge has no arrowhead.
|
|
- *7.1.4.5.17.* `simplify_external_links` sets an external node's
|
|
subtitle to its host. A leading `www.` is dropped. The node
|
|
stays one per URL. `consolidate_external_links` merges URLs
|
|
with the same host into one node whose id is `host/*` when
|
|
`consolidate_external_links_depth` is absent. Depth `0`
|
|
leaves each URL alone, even when the bool is true. Depth
|
|
`1` is `host/*`. Depth `2` is `host/a/*`. Depth `3` is
|
|
`host/a/b/*`. A leading `www.` is dropped from the host.
|
|
A path part longer than
|
|
`consolidate_external_links_max_char` characters is `...`.
|
|
The limit is 8 when absent. A part of that length stays.
|
|
Size and color follow `link_score` in section 7.1.4.2.3.
|
|
When depth is above 0, simplify is on even if its key is
|
|
false. The bool defaults to false. A double click opens
|
|
`https://host/` plus the real path parts before the first
|
|
`...`, with a trailing slash.
|
|
- *acceptance*: `https://www.domain.com/abc` and
|
|
`https://domain.com/acd` from two files become one node
|
|
`domain.com/*` with subtitle `domain.com` and `link_score`
|
|
2 when consolidate is true and depth is absent. With only
|
|
simplify, both URLs remain and each subtitle is
|
|
`domain.com`. Depth `2` merges
|
|
`https://domain.com/abc/one` and
|
|
`https://domain.com/abc/two` into `domain.com/abc/*` and
|
|
leaves `https://domain.com/zzz/one` as `domain.com/zzz/*`.
|
|
Depth `3` names `https://domain.com/a/b/c` as
|
|
`domain.com/a/b/*` and opens `https://domain.com/a/b/`.
|
|
`https://domain.com/abcdefghij/b/c` at depth `3` is
|
|
`domain.com/.../b/*`.
|
|
- *7.1.4.5.18.* A refresh puts every toggle, the theme select, and
|
|
the search box back to the values in section 5.1. The browser's
|
|
saved form state does not keep the previous choice.
|
|
- *acceptance*: The page sets each control again on `pageshow`.
|
|
- *7.1.4.5.19.* A checkbox toggle labeled "Show Sources" shows or
|
|
hides nodes built from the frontmatter `source` key, and the
|
|
edges from the page to those nodes. It starts checked when
|
|
`show_source` in section 5.1 is true or absent, and unchecked
|
|
when that key is false. The label does not change. A string
|
|
is one node. A list of strings is one node per entry. A blank
|
|
string is skipped. Any other value is skipped. The node is a
|
|
cloud, the same shape as an external link. Its id is that
|
|
string. Two pages with the same string share one node. A
|
|
double click opens the string when it contains `://`.
|
|
- *acceptance*: `source: https://example.com/rules.md` adds
|
|
one cloud node and one edge from that page. A list of two
|
|
URLs adds two nodes. `source: 12` adds none.
|
|
- *7.1.4.6.* Layout constants live at the top of the HTML template so
|
|
they can be tuned without hunting through the script.
|
|
|
|
| Constant | Value | Role |
|
|
| --- | --- | --- |
|
|
| `LINK_STRENGTH` | 0.4 | Spring along an edge; neighbors follow a dragged node |
|
|
| `BASE_LINK_DISTANCE` | 80 | Rest length of an edge |
|
|
| `LINK_DISTANCE_PER_DEGREE` | 0 | Distance does not grow with degree |
|
|
| `CHARGE_STRENGTH` | -140 | Repulsion that spreads the cloud |
|
|
| `CHARGE_DISTANCE_MIN` | 24 | Near distance for repulsion |
|
|
| `CHARGE_DISTANCE_MAX` | 500 | Far cutoff for repulsion |
|
|
| `NODE_ATTRACTION_STRENGTH` | 0 | Extra attraction is off |
|
|
| `CENTER_STRENGTH` | 1 | Each tick places the centroid on the viewport center |
|
|
| `GRAVITY_STRENGTH` | 0.05 | Per-node pull toward the viewport center |
|
|
| `COLLIDE_STRENGTH` | 1 | Overlap prevention |
|
|
| `BASE_COLLIDE_RADIUS` | 6 | Gap around the hitbox |
|
|
| `HITBOX_SCALE` | 1.25 | Collision circle versus the drawn radius |
|
|
| `EDGE_PADDING_MULTIPLIER` | 0 | Degree does not add collide radius |
|
|
| `CANVAS_MARGIN` | 56 | Border band that pushes a node inward |
|
|
| `CANVAS_MARGIN_STRENGTH` | 0.35 | Strength of that inward push |
|
|
| `ALPHA_DECAY` | 0.018 | How fast the simulation cools |
|
|
| `ALPHA_MIN` | 0.001 | Floor before the layout rests |
|
|
| `VELOCITY_DECAY` | 0.22 | Low damping so motion glides |
|
|
| `FLOAT_AMPLITUDE` | 2 | Idle drift, in em where 16 is 1em |
|
|
| `FLOAT_SPEED` | 0.0007 | How fast that drift cycles |
|
|
|
|
- *7.1.4.7.* Nodes start in a small disk around the viewport center
|
|
and expand under repulsion. They fade in by a breadth-first walk
|
|
from the highest in-degree node, highest connectivity first.
|
|
`d3.forceCenter` keeps the centroid on the viewport center.
|
|
`d3.forceX` and `d3.forceY` add center gravity so outliers drift
|
|
back. Both targets follow resize. The tick does not write a
|
|
canvas-edge clamp back into node coordinates. A margin force adds
|
|
inward velocity only while a node is inside `CANVAS_MARGIN` of the
|
|
border, and it skips a node that is pinned. The hitbox is
|
|
`HITBOX_SCALE` times the drawn radius, plus a fixed gap, so a
|
|
large node keeps clear of its neighbors. Each tick then pushes
|
|
any covering pair apart until the drawn circles, plus that gap,
|
|
no longer cover. An edge's rest length
|
|
is at least the two hitboxes added together. Degree does not
|
|
add padding, so a
|
|
hub is not shoved into the border. An unpinned node then drifts
|
|
at most `FLOAT_AMPLITUDE` on that em scale, and its edges
|
|
follow, so it looks like it is floating. A pinned or dragged node
|
|
does not drift. The view eases toward every shown node
|
|
over about 4200 ms, with 4em of padding. That is the same
|
|
fit as the button labeled "See all". The first node uses
|
|
`first_fit_ms` instead, so that opening zoom is slower.
|
|
It runs when the first node fades in, and again when the
|
|
last node fades in. A fit that is already moving keeps its
|
|
speed and aims at the new bounds. A second See all does not
|
|
stop the zoom.
|
|
It also runs when the theme changes.
|
|
The scale can go below 0.15 when that is what fits the
|
|
cloud, and it stays at most 6.
|
|
Hidden external links and hidden orphans are left out. A
|
|
later wheel zoom,
|
|
background drag, or node drag keeps the view the user set.
|
|
Wheel zoom and background drag pan the view. Dragging a node
|
|
reheats the simulation; releasing it lets the springs and
|
|
gravity settle.
|
|
- *7.1.4.8.* The output path must be writable. A failure is an error
|
|
message and a non-zero exit.
|
|
- *7.1.4.9.* A wiki with one relative link and one wikilink produces
|
|
two edges, and a file with neither is listed in `orphans`.
|
|
- *acceptance*: The JSON `edges` weights match the link counts.
|
|
- *7.1.4.10.* `--no-wikilinks` drops wikilink edges and keeps markdown
|
|
links.
|
|
- *acceptance*: The wikilink target is absent from `edges`.
|
|
- *7.1.4.11.* Opening the HTML in a current browser shows the graph,
|
|
hover, click highlight, search, and the orphan toggle.
|
|
- *acceptance*: A node hover changes its color to the hover color
|
|
and restores the previous radius on exit.
|
|
- *7.1.4.12.* JSON and HTML each store `frontmatter` on every markdown
|
|
node: every YAML field from that file. Missing or invalid YAML is
|
|
`{}` and does not abort the run.
|
|
- *acceptance*: A file with `name`, `version`, and `updated` has
|
|
those three keys on its node. A file with no frontmatter has
|
|
`{}`. The click card lists every frontmatter field. The
|
|
hover card has two lines: the node name, then unique
|
|
points, total points, in-degree, and out-degree.
|
|
- *7.1.4.13.* Repeated links from one file add less to size and
|
|
color than links from different files.
|
|
- *acceptance*: One source with weight 10 has `unique_points` 1,
|
|
`total_points` 10, and `link_score` below 2. Two sources with
|
|
weight 1 have `link_score` 2. HTML radius and color divide
|
|
`link_score` by `maxLinkScore`. The hover card's second line
|
|
includes the unique-point count and the total-point count.
|
|
- *7.1.4.14.* The Show Arrows toggle follows `show_arrows`.
|
|
- *acceptance*: The checkbox is checked when the key is absent
|
|
or true, and unchecked when it is false. The HTML defines
|
|
marker `edge-arrow` and sets `marker-end` from that toggle.
|
|
- *7.1.4.15.* Graph settings start from the packaged
|
|
`fiona-settings.json` graph section. A file found by walking up
|
|
from the scanned directory replaces the keys it sets. A missing
|
|
file still uses the packaged section.
|
|
- *acceptance*: A scan whose directory and parents have no
|
|
`fiona-settings.json` still applies `show_legend`,
|
|
`show_arrows`, and `simplify_external_links` from the
|
|
packaged file.
|
|
- *7.1.4.16.* `NOT_IMPLEMENTED` The HTML graph draws a node when
|
|
enough files point at it, or when a parent of that node is
|
|
clicked. The setting is `node_importance_min_point` in
|
|
section 5.1.
|
|
- *7.1.4.16.1.* `NOT_IMPLEMENTED` The count is `unique_points`
|
|
from *7.1.4.2.3.*: how many different files point at the
|
|
node. A node is below the cutoff when `unique_points` is
|
|
less than the setting. A node at the cutoff is drawn.
|
|
- *acceptance*: The setting is 5. A node with
|
|
`unique_points` 4 is absent from the opening draw. A
|
|
node with `unique_points` 5 is drawn.
|
|
- *7.1.4.16.2.* `NOT_IMPLEMENTED` `0` disables the filter. A
|
|
missing key, or a value that is not an integer of 0 or
|
|
more, is the same as `0`. Every node is drawn, still
|
|
subject to the Show Orphans, Show External Links, and
|
|
Show Sources toggles.
|
|
- *acceptance*: With the key absent or set to `0`, a node
|
|
with `unique_points` 0 is drawn when those toggles
|
|
allow it.
|
|
- *7.1.4.16.3.* `NOT_IMPLEMENTED` A parent of a node is any
|
|
node with an edge whose target is that node. While a
|
|
parent is the clicked node, each direct target is drawn
|
|
even when it is below the cutoff. The clicked node stays
|
|
drawn for as long as it stays clicked, including when it
|
|
is below the cutoff. Clearing the click hides a
|
|
below-cutoff node again. Clicking a node draws its
|
|
direct targets only, not the targets of those targets.
|
|
- *acceptance*: The setting is 5. Page A has
|
|
`unique_points` 5 and links to B. B has
|
|
`unique_points` 1 and links to C. C has
|
|
`unique_points` 1. The opening draw shows A and hides
|
|
B and C. Clicking A draws B and leaves C hidden.
|
|
Clicking B keeps B drawn and draws C. Clicking empty
|
|
space hides B and C. A node with no incoming edge
|
|
stays hidden, because it has no parent to click.
|
|
- *7.1.4.16.4.* `NOT_IMPLEMENTED` An edge is drawn only when
|
|
both of its nodes are drawn. See all, the opening fit,
|
|
and search use that same drawn set. A hidden node is
|
|
left out of the fit, as *7.1.4.7.* does for a hidden
|
|
orphan, and it is not a search match.
|
|
- *acceptance*: An edge into a below-cutoff node is
|
|
absent until a click draws that node. Searching for
|
|
its name does not draw it.
|
|
- *7.1.4.16.5.* `NOT_IMPLEMENTED` Show Orphans, Show External
|
|
Links, and Show Sources still apply. A node hidden by
|
|
one of those toggles stays hidden when it meets the
|
|
cutoff or when its parent is clicked.
|
|
- *acceptance*: With Show External Links unchecked,
|
|
clicking a page does not draw the external node it
|
|
links to.
|
|
- *7.1.4.16.6.* `NOT_IMPLEMENTED` `--format json` and `--json`
|
|
still list every node and edge. The filter changes the
|
|
HTML draw only.
|
|
- *acceptance*: JSON includes a node whose
|
|
`unique_points` is below the setting.
|
|
- *7.1.4.20.* `fiona.wiki.graph.ignore_files` is a list of glob
|
|
patterns in the wiki's `fiona-settings.json`. A markdown file
|
|
whose path relative to the scan root matches is not a graph
|
|
node. HTML and JSON both omit it. An edge that would start or
|
|
end at that file is omitted. The file does not count toward
|
|
`max_files` and is not an orphan. A missing key is an empty
|
|
list. A value that is not a list of strings raises `ValueError`
|
|
with the text `fiona.wiki.graph.ignore_files must be a list of strings`.
|
|
A pattern that starts with `*/` also matches that same path at
|
|
the scan root. The shipped list is `*/table_of_contents.md`.
|
|
A table of contents links to every page, so leaving it in the
|
|
graph hides the real links.
|
|
- *acceptance*: With that pattern, `table_of_contents.md` at the
|
|
scan root and `docs/table_of_contents.md` are absent from JSON
|
|
`nodes`, `edges`, and `orphans`. `a.md` and `b.md` stay.
|
|
An edge from the table of contents to `a.md` is absent. An
|
|
edge from `a.md` to the table of contents is absent.
|
|
- *acceptance*: `max_files` is 1. The directory holds
|
|
`table_of_contents.md` and `kept.md`. JSON `file_count` is 1
|
|
and the only node is `kept.md`.
|
|
- *acceptance*: A non-list `ignore_files` makes `wiki graph`
|
|
raise `ValueError` with that text.
|
|
- *7.1.4.17.* `NOT_IMPLEMENTED` A markdown file that `.gitignore`
|
|
matches is not a graph node. HTML and JSON both omit it. An
|
|
edge that would start or end at that file is omitted. The file
|
|
does not count toward `max_files`. Housekeeping's orphan list
|
|
(*7.2.6.*) uses this graph, so an omitted file is not an
|
|
orphan. There is no setting that turns this off.
|
|
- *acceptance*: The scanned `.gitignore` lists `secret.md`.
|
|
`secret.md` is absent from JSON `nodes`, `edges`, and
|
|
`orphans`. A page that links to it has no edge to it.
|
|
- *acceptance*: `max_files` is 1. The directory holds `kept.md`
|
|
and a gitignored `secret.md`. JSON `file_count` is 1 and the
|
|
only node is `kept.md`.
|
|
- *7.1.4.17.1.* `NOT_IMPLEMENTED` Patterns are the non-comment,
|
|
non-blank lines of the `.gitignore` in the scanned directory,
|
|
and of each `.gitignore` in a subdirectory. A subdirectory
|
|
file's patterns match only paths under that subdirectory.
|
|
Matching is *7.1.2.10.*: a pattern with no slash matches any
|
|
path segment, and a pattern that ends with `/` matches that
|
|
directory and everything under it.
|
|
- *acceptance*: The scanned `.gitignore` lists `drafts/`.
|
|
`drafts/note.md` is absent. `wiki/note.md` is present.
|
|
- *acceptance*: A line `# secret.md` does not omit
|
|
`secret.md`.
|
|
- *acceptance*: `notes/.gitignore` lists `scratch.md`.
|
|
`notes/scratch.md` is absent. `scratch.md` at the scan
|
|
root stays.
|
|
- *7.1.4.17.2.* `NOT_IMPLEMENTED` A path under `.git/` is omitted
|
|
even when no pattern names it.
|
|
- *acceptance*: With no `.gitignore`, a markdown file under
|
|
the scan root stays, and a markdown file under `.git/` is
|
|
absent.
|
|
- *7.1.4.18.* `NOT_IMPLEMENTED` The HTML graph draws a crowded
|
|
directory as one directory node. That keeps a large wiki
|
|
readable and cuts the number of shapes the page simulates. The
|
|
settings are `directory_nodes.min_files` and
|
|
`directory_nodes.outside_depth` in section 5.1.
|
|
- *7.1.4.18.1.* `NOT_IMPLEMENTED` A directory collapses when it
|
|
holds at least `min_files` markdown pages directly. Pages in
|
|
its subdirectories do not count toward that number. The scan
|
|
root never collapses. `0`, a missing key, or a value that is
|
|
not an integer of 0 or more turns collapsing off, and the
|
|
graph is drawn as it is today.
|
|
- *acceptance*: `min_files` is 3. `wiki/concepts/` holds
|
|
`a.md`, `b.md`, and `c.md`. The opening draw has one node
|
|
`wiki/concepts/` and no node for those three pages.
|
|
- *acceptance*: `min_files` is 3. `wiki/notes/` holds two
|
|
pages directly and `wiki/notes/deep/` holds three. The
|
|
opening draw shows both pages of `wiki/notes/` and one node
|
|
`wiki/notes/deep/`.
|
|
- *acceptance*: With the key absent or `0`, no directory node
|
|
is drawn.
|
|
- *7.1.4.18.2.* `NOT_IMPLEMENTED` A directory node stands for
|
|
every page under that directory, including its
|
|
subdirectories. Its id is the directory path relative to the
|
|
scan root with a trailing `/`. Its name and subtitle are the
|
|
last path segment with a trailing `/`. A collapsed directory
|
|
inside another collapsed directory is drawn only in its
|
|
parent's view (*7.1.4.18.4.*).
|
|
- *acceptance*: The node for `wiki/concepts/` has id
|
|
`wiki/concepts/` and subtitle `concepts/`.
|
|
- *7.1.4.18.3.* `NOT_IMPLEMENTED` A directory node acts like a
|
|
page node. The edges of its pages are drawn on it.
|
|
- *7.1.4.18.3.1.* `NOT_IMPLEMENTED` An edge between a page
|
|
inside and a node outside is redrawn between the directory
|
|
node and that outside node. Edges that land on the same pair
|
|
merge into one edge whose weight is the sum. An edge with
|
|
both ends inside the directory is not drawn in the view that
|
|
shows the directory node.
|
|
- *acceptance*: `wiki/index.md` links once to `a.md` and
|
|
twice to `b.md`, both inside collapsed `wiki/concepts/`.
|
|
One edge runs from `wiki/index.md` to `wiki/concepts/`
|
|
with weight 3. A link from `a.md` to `b.md` draws no edge
|
|
in that view.
|
|
- *7.1.4.18.3.2.* `NOT_IMPLEMENTED` `unique_points`,
|
|
`total_points`, `link_score`, in-degree, and out-degree are
|
|
computed from those merged edges with the rule in
|
|
*7.1.4.2.3.*. Size and color follow `link_score` like any
|
|
node.
|
|
- *acceptance*: In the example above, the directory node
|
|
has `unique_points` 1, `total_points` 3, and in-degree 1.
|
|
- *7.1.4.18.3.3.* `NOT_IMPLEMENTED` Hover, click highlight,
|
|
drag, glow, the hover card, and the toggles behave as for a
|
|
page. The click card shows `path`, `pages` (every page under
|
|
the directory), then unique points, total points, in-degree,
|
|
and out-degree. It has no frontmatter fields.
|
|
- *7.1.4.18.3.4.* `NOT_IMPLEMENTED` Search matches a directory
|
|
node when its name matches, or when the name of a page
|
|
under it matches. External links and `source` nodes of its
|
|
pages hang from the directory node.
|
|
- *acceptance*: Searching `a.md` while `wiki/concepts/` is
|
|
collapsed highlights the `wiki/concepts/` node.
|
|
- *7.1.4.18.3.5.* `NOT_IMPLEMENTED` The shape is
|
|
`shapes.directory` from the active theme (section 5.3). The
|
|
Legend's shape group adds a directory entry while a
|
|
directory node is drawn.
|
|
- *acceptance*: A theme that sets `shapes.directory` to
|
|
`star` draws the directory node as a star. A theme that
|
|
omits the key uses `folder` from `colorful`.
|
|
- *7.1.4.18.4.* `NOT_IMPLEMENTED` Double click on a directory
|
|
node opens that directory's view. The other nodes leave the
|
|
page. The view draws the pages that sit directly in the
|
|
directory, one directory node for each collapsed
|
|
subdirectory, the pages of each subdirectory that does not
|
|
collapse, and the edges between them. A collapsed
|
|
subdirectory follows *7.1.4.18.1.* inside the view. The view
|
|
runs the See all fit from *7.1.4.7.*.
|
|
- *acceptance*: Double click on `wiki/concepts/` draws
|
|
`a.md`, `b.md`, and `c.md`, the edge from `a.md` to
|
|
`b.md`, and no node that sits outside `wiki/concepts/`
|
|
except the ones in *7.1.4.18.5.*.
|
|
- *7.1.4.18.5.* `NOT_IMPLEMENTED` A node outside the directory
|
|
is drawn in its view when a page inside points at it, within
|
|
`outside_depth` hops along the direction of the edges. Depth
|
|
`1` draws the direct targets. Depth `2` also draws their
|
|
targets. Depth `0` draws no outside node. An outside node is
|
|
drawn the way the opening draw shows it: as its page, or as
|
|
the collapsed directory that holds it. Outside nodes and
|
|
their edges are drawn at half opacity. A node that only
|
|
points into the directory is not drawn.
|
|
- *acceptance*: `outside_depth` is 1. `a.md` links to
|
|
`wiki/index.md`, and `wiki/index.md` links to
|
|
`wiki/log.md`. The `wiki/concepts/` view draws
|
|
`wiki/index.md` at half opacity and does not draw
|
|
`wiki/log.md`. With depth 2 it draws both.
|
|
- *acceptance*: `outside_depth` is 1 and `a.md` links to
|
|
`wiki/other/x.md`, inside collapsed `wiki/other/`. The view
|
|
draws the node `wiki/other/` and no node `wiki/other/x.md`.
|
|
- *7.1.4.18.6.* `NOT_IMPLEMENTED` A bar at the upper center of
|
|
the graph names the open view as a path, for example
|
|
`All / wiki / concepts`. Each segment opens that view. A
|
|
button labeled "Back" opens the parent view and is disabled
|
|
on the opening view. The open view is kept in the page hash
|
|
as `#dir=<id>`, so browser back and forward move between
|
|
views and a reload reopens the same view. A hash that names
|
|
no collapsed directory opens the opening view.
|
|
- *acceptance*: From `wiki/concepts/`, Back draws the
|
|
opening view again with the `wiki/concepts/` node.
|
|
- *7.1.4.18.7.* `NOT_IMPLEMENTED` Show Orphans, Show External
|
|
Links, Show Sources, search, See all, and
|
|
`node_importance_min_point` (*7.1.4.16.*) apply to the nodes
|
|
of the open view only. A double click on an outside page in
|
|
a view opens that file. A double click on an outside
|
|
directory node opens that directory's view.
|
|
- *7.1.4.18.8.* `NOT_IMPLEMENTED` `--format json` and `--json`
|
|
list every page and edge, as today. Collapsing changes the
|
|
HTML draw only.
|
|
- *acceptance*: JSON with `min_files` 3 still lists `a.md`,
|
|
`b.md`, and `c.md` as nodes.
|
|
- *7.1.4.19.* `NOT_IMPLEMENTED` `wiki graph --serve` is a light
|
|
live view. It builds the graph, serves it over local HTTP, and
|
|
rebuilds when a scanned file changes. The open page then shows
|
|
the change without a manual refresh. It uses the standard
|
|
library only and adds no dependency.
|
|
- *7.1.4.19.1.* `NOT_IMPLEMENTED` The server binds `--host`
|
|
(default `serve.host`, `127.0.0.1`) and `--port` (default
|
|
`serve.port`, 8765). Port `0` picks a free port. On start it
|
|
prints
|
|
`Serving wiki graph at http://<host>:<port>/ (Ctrl+C to stop)`
|
|
on stdout and opens that URL in the default browser, unless
|
|
`--no-open` is set. Ctrl+C stops it, prints `Stopped`, and
|
|
exits 0. A host other than `127.0.0.1`, `localhost`, or `::1`
|
|
prints a warning on stderr that the wiki is readable from the
|
|
network.
|
|
- *acceptance*: `--serve --port 0 --no-open` prints the URL
|
|
with a port above 0, and opens no browser.
|
|
- *acceptance*: A port already in use prints an error on
|
|
stderr and exits 1.
|
|
- *7.1.4.19.2.* `NOT_IMPLEMENTED` `--serve` with `--format json`
|
|
or `--json` is a usage error with exit 2. `--host`, `--port`,
|
|
or `--no-open` without `--serve` is the same usage error.
|
|
`--serve` writes no file unless `-o` is given. Then the server
|
|
writes that file after every build.
|
|
- *7.1.4.19.3.* `NOT_IMPLEMENTED` The server answers `GET` only.
|
|
Another method is 405.
|
|
- *7.1.4.19.3.1.* `NOT_IMPLEMENTED` `/` is the current graph
|
|
HTML: the same page as *7.1.4.5.*, plus the live script in
|
|
*7.1.4.19.5.*.
|
|
- *7.1.4.19.3.2.* `NOT_IMPLEMENTED` `/graph.json` is the
|
|
current JSON from *7.1.4.3.* with `metadata.build` added.
|
|
`build` starts at 1 and adds 1 on every rebuild.
|
|
- *7.1.4.19.3.3.* `NOT_IMPLEMENTED` `/events` is a
|
|
Server-Sent Events stream. After each rebuild it sends
|
|
`event: graph` with `data: {"build": <n>}`. A comment line
|
|
goes out every 15 seconds so the connection stays open.
|
|
- *7.1.4.19.3.4.* `NOT_IMPLEMENTED` `/files/<path>` returns a
|
|
markdown file under the scan root as `text/plain;
|
|
charset=utf-8`. A path that leaves the scan root, a path
|
|
that is not a scanned file, or a missing file is 404. In a
|
|
served page, a double click on a page opens this URL in
|
|
place of `file://`.
|
|
- *acceptance*: `/files/../secret.md` and
|
|
`/files/%2e%2e/secret.md` are 404.
|
|
- *7.1.4.19.3.5.* `NOT_IMPLEMENTED` Any other path is 404.
|
|
- *7.1.4.19.4.* `NOT_IMPLEMENTED` The watcher polls every
|
|
`serve.poll_seconds` seconds. It compares the set of matched
|
|
markdown files and each file's modification time and size.
|
|
The match is the include and exclude patterns and the
|
|
`.gitignore` rule in *7.1.4.17.*. The nearest
|
|
`fiona-settings.json` and the `constellation/` files from
|
|
section 5.3 are watched too. A change waits `serve.debounce_ms`
|
|
with no newer change, then one rebuild runs. Several saves
|
|
in that window cause one rebuild.
|
|
- *acceptance*: Adding `new.md` that links to `a.md` makes
|
|
the next `/graph.json` list node `new.md` and an edge to
|
|
`a.md`, and `/events` sends one `graph` event.
|
|
- *acceptance*: Deleting a page removes its node and its
|
|
edges from the next `/graph.json`.
|
|
- *acceptance*: Saving three files within the debounce window
|
|
raises `build` by 1.
|
|
- *7.1.4.19.5.* `NOT_IMPLEMENTED` The served page listens on
|
|
`/events`. On a `graph` event whose `build` is newer than its
|
|
own, it saves its state to `sessionStorage`, then reloads.
|
|
The state is the zoom transform, each drawn node's position
|
|
by id, the clicked node, the open directory view, every
|
|
toggle, the theme, and the search text. After the reload the
|
|
page restores that state once, places known nodes where they
|
|
were, and fades in only the new nodes. The restore replaces
|
|
the reset in *7.1.4.5.18.* for that one load. A manual
|
|
refresh still resets the controls.
|
|
- *acceptance*: After a rebuild adds `new.md`, an existing
|
|
node keeps its position within 1em, the theme and toggles
|
|
are unchanged, and `new.md` fades in.
|
|
- *7.1.4.19.6.* `NOT_IMPLEMENTED` A rebuild that fails keeps the
|
|
last good graph. The error prints on stderr and the server
|
|
keeps running. No `graph` event is sent.
|
|
- *7.1.4.19.7.* `NOT_IMPLEMENTED` A file written without
|
|
`--serve` contains no live script and opens no connection.
|
|
- *acceptance*: Plain `wiki graph` output does not contain
|
|
`EventSource`.
|
|
- *7.1.4.19.8.* `NOT_IMPLEMENTED` The server makes no outbound
|
|
call. It reads only files under the scan root and the
|
|
settings and theme files of section 5.
|
|
|
|
#### 7.1.5. wiki ingest
|
|
|
|
> Status: Partially Implemented
|
|
|
|
Implements section 4.7.6. An ingest has two phases. The tool runs
|
|
the first: it turns one source into a file on disk, archives a copy
|
|
under `raw/`, converts that copy to markdown, and writes a wiki page
|
|
with `status: raw`. The agent runs the second: it refines the page
|
|
and sets `status: refined`. The Fiona skill teaches the second
|
|
phase and when to convert a source by hand. The pattern both phases
|
|
follow is section 3.1 of
|
|
[llm-wiki-mod](./src/fiona/data/llm_wiki/utils/skills/fiona/llm-wiki-mod.md).
|
|
|
|
The tool runs five stages in order. Each stage has one base class
|
|
and one job. Class names are in Appendix D.
|
|
|
|
| Stage | Input | Output |
|
|
| --- | --- | --- |
|
|
| Fetch | the `<source>` argument | a local file, its origin, and its media type |
|
|
| Archive | that local file | a file under `raw/` |
|
|
| Convert | the archived file | markdown body, title, metadata, conversion state, warnings |
|
|
| Write | the conversion | one page under `wiki/` with `status: raw` |
|
|
| Record | the run | one line appended to `wiki/log.md` |
|
|
|
|
Shipped converters, in registration order:
|
|
|
|
| Converter | Media types | Extensions | Raw category | Library |
|
|
| --- | --- | --- | --- | --- |
|
|
| `markdown` | `text/markdown` | `.md`, `.markdown` | `articles` | none |
|
|
| `text` | `text/plain` | `.txt`, `.text`, `.log` | `articles` | none |
|
|
| `html` | `text/html`, `application/xhtml+xml` | `.html`, `.htm`, `.xhtml` | `articles` | `beautifulsoup4`, `markdownify` |
|
|
| `pdf` | `application/pdf` | `.pdf` | `papers` | `pypdf` |
|
|
|
|
A source that no converter claims is archived under `raw/other/`.
|
|
Pure-Python libraries are preferred so the install stays light.
|
|
Candidates for later formats are in Appendix C.
|
|
|
|
- *7.1.5.1.* The pipeline is built from parts.
|
|
- *7.1.5.1.1.* The pipeline receives its fetchers, converter
|
|
registry, archiver, page writer, and log writer as constructor
|
|
arguments. It holds no format-specific code.
|
|
- *7.1.5.1.2.* A converter declares its name, media types,
|
|
extensions, and raw category as class attributes, and implements
|
|
one `convert` method. Adding a format is one new converter class,
|
|
one registration, and its tests.
|
|
- *acceptance*: A test registers a converter for
|
|
`application/x-test` and ingests a `.xtest` file into a page.
|
|
No pipeline, archiver, or writer module changes.
|
|
- *7.1.5.1.3.* A fetcher declares whether it accepts an argument and
|
|
returns the fetched source. The URL fetcher takes its opener as
|
|
an argument.
|
|
- *acceptance*: Tests run the URL path with a fake opener and
|
|
make no network call.
|
|
- *7.1.5.2.* Fetch.
|
|
- *7.1.5.2.1.* An argument that starts with `http://` or `https://`
|
|
is a URL. Any other `<scheme>://` prefix exits 1 with
|
|
`Error: Unsupported URL scheme: <scheme>`. Anything else is a
|
|
local path. A missing path and a directory each exit 1 with the
|
|
message from section 4.7.6.
|
|
- *acceptance*: `ftp://host/a.pdf` exits 1 and writes nothing.
|
|
- *7.1.5.2.2.* A URL is downloaded with `urllib.request` into
|
|
`ingest/<slug>.<ext>`. Redirects are followed. Proxies come from
|
|
the environment. The request sends `user_agent` and waits up to
|
|
`timeout_seconds`. No credentials are sent.
|
|
- *7.1.5.2.3.* A non-2xx response or a network error exits 1 with
|
|
`Error: Download failed: <url>: <reason>`. A response that needs
|
|
a login is *7.1.5.2.8.* instead. The reason names the
|
|
HTTP status when there is one. A body larger than
|
|
`max_download_bytes` stops the download and exits 1. In each
|
|
case the partial file is deleted and `ingest/` is unchanged.
|
|
- *acceptance*: A fake opener that answers 404 exits 1, the
|
|
message contains `404`, and `ingest/` has the same files as
|
|
before.
|
|
- *7.1.5.2.4.* The media type is the response `Content-Type`
|
|
without parameters. A `charset` parameter is kept as a decoding
|
|
hint. With no usable header, the extension decides, using the
|
|
converter table and then `mimetypes`. With no known extension,
|
|
the first 1024 bytes decide: `%PDF-` is `application/pdf`,
|
|
`<!doctype html` or `<html` in any case is `text/html`, valid
|
|
UTF-8 with no NUL byte is `text/plain`. Anything else is
|
|
`application/octet-stream`.
|
|
- *acceptance*: A local file named `notes` whose bytes start with
|
|
`%PDF-` is converted by `pdf`.
|
|
- *7.1.5.2.5.* A downloaded file takes the first extension of the
|
|
converter that claims its media type, then the URL path suffix,
|
|
then `.bin`.
|
|
- *acceptance*: `https://host/report` served as
|
|
`application/pdf` is saved as `ingest/report.pdf`.
|
|
- *7.1.5.2.6.* The slug of a URL is built from its last three
|
|
non-empty path segments, each without its extension, with a
|
|
segment named `index` dropped. An empty path uses the host. The
|
|
slug of a file is its stem. The text is lowercased, each run of
|
|
characters outside `[a-z0-9]` becomes one `-`, leading and
|
|
trailing `-` are trimmed, and the result is cut to 80
|
|
characters. An empty result is `source`.
|
|
- *acceptance*:
|
|
`https://host/job/performance_TL153/2108/console` has the slug
|
|
`performance-tl153-2108-console`.
|
|
- *7.1.5.2.7.* `NOT_IMPLEMENTED` A local path is resolved against
|
|
the current directory. `./`, `.\`, and either separator are
|
|
accepted on Windows. A missing path inside the wiki's `ingest/`
|
|
is often a file an earlier run already moved (*7.1.5.3.6.*).
|
|
When a file with the same name exists under `raw/`, the error
|
|
adds one space and `(moved to <raw path> by an earlier ingest)`.
|
|
Several matches use the first in sorted path order.
|
|
- *acceptance*: Ingesting `./ingest/a.md` and then
|
|
`.\ingest\a.md` exits 0, then exits 1 with
|
|
`Error: Source not found: .\ingest\a.md (moved to
|
|
raw/articles/a.md by an earlier ingest)` on one line.
|
|
- *7.1.5.2.8.* `NOT_IMPLEMENTED` A URL that needs a login is an
|
|
error, not a source. The tool cannot log in, so it stops and
|
|
leaves the next step to the agent (*7.1.5.14.*). A login page
|
|
must never reach `raw/` as if it were the source.
|
|
- *7.1.5.2.8.1.* The tool sends no credentials, cookies, or
|
|
tokens, and reads none from the environment or from a file
|
|
(*7.1.5.2.2.*). It never asks for a password.
|
|
- *7.1.5.2.8.2.* Three checks run in this order. The first hit
|
|
decides the reason:
|
|
|
|
| Check | Hit when | Reason text |
|
|
| --- | --- | --- |
|
|
| Status | The response is HTTP 401 or 403 | `<status> <phrase>`, e.g. `401 Unauthorized` |
|
|
| Redirect | The final URL differs from the requested URL and its host and path match one of `login_url_patterns` | `redirected to <final url>` |
|
|
| Login form | The media type is `text/html` and the body has an `<input>` whose `type` is `password`, in any case or quoting | `login page at <final url>` |
|
|
|
|
- *7.1.5.2.8.3.* `login_url_patterns` is a list of regular
|
|
expressions, searched case-insensitively in
|
|
`<host><path>` of the final URL. The query string is not
|
|
searched. The default list is in section 5.1. The redirect
|
|
check is skipped when there was no redirect, so a user can
|
|
still ingest a page whose own path holds `login`.
|
|
- *7.1.5.2.8.4.* On a hit, stderr has the `Error:
|
|
Authentication required` line and the `Next:` line from
|
|
section 4.7.6, and the command exits 4. The body is read
|
|
within `max_download_bytes` and then dropped. Nothing is
|
|
written under `ingest/`, `raw/`, or `wiki/`, and `wiki/log.md`
|
|
is unchanged. The URL in both lines is the one the user
|
|
passed, with userinfo removed.
|
|
- *7.1.5.2.8.5.* `--dry-run` runs the same checks and exits 4
|
|
the same way. `--json` prints nothing on stdout
|
|
(*7.1.5.13.*).
|
|
- *7.1.5.2.8.6.* An HTML page that passes the three checks but
|
|
has no text still converts as `partial` with the warning in
|
|
*7.1.5.4.5.5.*. Login detection does not replace that warning.
|
|
- *acceptance*: A fake opener that answers 401 exits 4. Stderr
|
|
has `Authentication required`, `401`, and the `Next:` line.
|
|
`ingest/`, `raw/`, `wiki/`, and `wiki/log.md` have the same
|
|
hashes as before.
|
|
- *acceptance*: A fake opener that redirects
|
|
`https://host/job/x/console` to
|
|
`https://host/login?from=%2Fjob%2Fx%2Fconsole` and answers 200
|
|
exits 4 with the reason `redirected to https://host/login?...`.
|
|
- *acceptance*: A fake opener that answers 200 with an HTML form
|
|
holding `<INPUT TYPE='password'>` exits 4 with the reason
|
|
`login page at <url>`.
|
|
- *acceptance*: A 200 HTML page at `https://host/docs/login`,
|
|
with no redirect and no password input, is ingested with
|
|
exit 0.
|
|
- *acceptance*: A 404 still exits 1 with `Download failed`.
|
|
- *7.1.5.2.9.* `NOT_IMPLEMENTED` `--origin URL` records where a
|
|
local file came from, so a source saved by hand after
|
|
*7.1.5.2.8.* keeps its real provenance.
|
|
- *7.1.5.2.9.1.* `--origin` is valid only when `<source>` is a
|
|
local path. With a URL source it exits 1 with
|
|
`Error: --origin needs a local file: <source>`. A value that
|
|
is not `http://` or `https://` exits 1 with
|
|
`Error: --origin must be an http or https URL: <value>`. Both
|
|
checks run before fetch.
|
|
- *7.1.5.2.9.2.* The file is fetched, archived, and moved as any
|
|
local file. The page frontmatter then has `ingest.origin` set
|
|
to the URL without userinfo, `origin_kind: url`, and
|
|
`final_url: null`. The `html` converter uses that URL to make
|
|
relative links absolute (*7.1.5.4.5.3.*). The log bullet names
|
|
that URL (*7.1.5.6.*). JSON `origin` and `origin_kind` match
|
|
the frontmatter.
|
|
- *7.1.5.2.9.3.* The slug and default raw name still come from
|
|
the file stem (*7.1.5.2.6.*), so the name the agent chose in
|
|
`ingest/` is kept.
|
|
- *acceptance*: `wiki ingest ingest/console.html --origin
|
|
https://host/job/x/console` exits 0. The page has
|
|
`origin: https://host/job/x/console`, `origin_kind: url`,
|
|
and a relative link `/a` in the file becomes
|
|
`https://host/a`. No network call is made.
|
|
- *acceptance*: `wiki ingest https://host/a --origin
|
|
https://host/b` exits 1 and makes no network call.
|
|
- *7.1.5.3.* Archive.
|
|
- *7.1.5.3.1.* The default raw path is
|
|
`raw/<category>/<slug>.<ext>`. The category is the raw category
|
|
of the converter that claims the media type, unless
|
|
`raw_categories` maps that converter name to another directory.
|
|
With no converter it is `other`.
|
|
- *acceptance*: A PDF is archived at `raw/papers/<slug>.pdf`.
|
|
- *7.1.5.3.2.* `-r/--raw PATH` is relative to `raw/`. A leading
|
|
`raw/` is accepted. A value that ends in an extension names the
|
|
file. Any other value is a directory, and the file inside it is
|
|
`<slug>.<ext>`. Missing directories are created. After resolving
|
|
`..` and symlinks the path must be inside `raw/`, else exit 1
|
|
with nothing written.
|
|
- *acceptance*: `-r jenkins/tl153` archives an HTML source at
|
|
`raw/jenkins/tl153/<slug>.html`. `-r ../wiki/x` exits 1.
|
|
- *7.1.5.3.3.* Before writing, the archive computes the SHA-256 of
|
|
the fetched file and compares it with every file under `raw/`.
|
|
Only files of the same byte size are hashed. A match anywhere
|
|
under `raw/`, at any name or directory, is reused: nothing is
|
|
written under `raw/`, `raw_path` is the matching file, and
|
|
`raw_reused` is true. When `-r` names another place, the match
|
|
still wins and adds the warning
|
|
`already archived at <raw path>; --raw ignored`. Several matches
|
|
use the first in sorted path order.
|
|
- *acceptance*: Ingesting one file twice, the second time under
|
|
another name and with `-r other/dir`, leaves one raw copy and
|
|
the second run reports `raw_reused: true` and that copy's path.
|
|
- *acceptance*: A URL whose body matches an archived file reuses
|
|
that file.
|
|
- *7.1.5.3.4.* The archive never overwrites, renames, or deletes a
|
|
file under `raw/`. When the raw path holds a file with different
|
|
bytes, the first free name of `<slug>-2.<ext>`,
|
|
`<slug>-3.<ext>`, and so on is used.
|
|
- *acceptance*: Ingesting a changed file of the same name creates
|
|
`<slug>-2.<ext>` and the first copy is unchanged.
|
|
- *7.1.5.3.5.* When the raw copy is reused and a page under `wiki/`
|
|
already has that raw path in `sources` or that hash in
|
|
`ingest.sha256`, the run is a duplicate. It prints the
|
|
`Duplicate:` line, sets `duplicate_page`, writes no page and no
|
|
log line, and exits 0. `--force` converts again and replaces
|
|
that page, at its current path unless `-t` is set.
|
|
- *acceptance*: A second ingest of the same bytes exits 0,
|
|
reports the first page as `duplicate_page`, and leaves every
|
|
file under the wiki unchanged apart from the moved source.
|
|
- *7.1.5.3.6.* A file inside the wiki's `ingest/` directory is moved
|
|
into `raw/`, so it leaves the drop zone. A downloaded file is in
|
|
`ingest/`, so it is moved too. When the raw copy is reused, the
|
|
`ingest/` file is deleted, because `raw/` already holds its
|
|
bytes. Any other file is copied and the original is left as it
|
|
was. A file already under `raw/` is not archived again, and
|
|
conversion reads it in place. `-r` on such a file exits 1.
|
|
- *acceptance*: After ingesting `ingest/a.pdf`, that path is gone
|
|
and `raw/papers/a.pdf` holds its bytes. A file ingested from
|
|
`personal/` is copied and the original is unchanged.
|
|
- *7.1.5.4.* Convert.
|
|
- *7.1.5.4.1.* The registry picks the first converter, in
|
|
registration order, whose media types contain the detected type,
|
|
then the first whose extensions contain the file extension. With
|
|
no match, no page is written, stdout ends with the
|
|
`No converter` line, and the command exits 3.
|
|
- *acceptance*: A `.xyz` file of random bytes exits 3 and is
|
|
archived under `raw/other/`.
|
|
- *7.1.5.4.2.* A converter returns the markdown body, a title or
|
|
none, a metadata map, a conversion state of `complete` or
|
|
`partial`, and a list of warnings. A converter that cannot read
|
|
a file it claims, such as a corrupt or encrypted PDF, raises a
|
|
conversion error. The command then writes no page, prints
|
|
`Error: Conversion failed: <converter>: <reason>` on stderr, and
|
|
exits 3. The raw copy stays.
|
|
- *acceptance*: An encrypted PDF exits 3, its raw copy exists,
|
|
and no page is written.
|
|
- *7.1.5.4.3.* `markdown` keeps the body byte for byte. A
|
|
frontmatter block in the source moves into `source_meta`. The
|
|
title is that frontmatter `title`, then the first H1, then the
|
|
slug.
|
|
- *7.1.5.4.4.* `text` decodes UTF-8 with any BOM stripped, then the
|
|
charset hint, then `cp1252`. A fallback decode adds a warning.
|
|
The body is one fenced `text` block whose fence is longer than
|
|
any backtick run in the text. The title is the slug.
|
|
- *7.1.5.4.5.* `html` parses with beautifulsoup4 and converts with
|
|
markdownify.
|
|
- *7.1.5.4.5.1.* The title is `<title>`, then `og:title`, then the
|
|
first `<h1>`. `source_meta` holds `lang`, the canonical link,
|
|
and the `description`, `author`, `og:*`, and
|
|
`article:published_time` meta values that are present.
|
|
- *7.1.5.4.5.2.* The content root is the first `<article>`, then
|
|
`<main>`, then `<body>`. `script`, `style`, `noscript`,
|
|
`template`, and comments are dropped.
|
|
- *7.1.5.4.5.3.* A relative `href` or `src` is made absolute
|
|
against `<base href>`, then the origin URL. A local file has no
|
|
origin URL, so its relative links are left to *7.1.5.5.4.*.
|
|
- *7.1.5.4.5.4.* Headings are ATX. Bullets use `-`. `<pre>` becomes
|
|
a fenced block. A table becomes a pipe table.
|
|
- *7.1.5.4.5.5.* A body with no text outside markup is `partial`
|
|
with the warning `no text content; the page may need
|
|
JavaScript or a login`.
|
|
- *acceptance*: For a fixture with a nav, an article, a script,
|
|
a relative link `/a`, and a table, fetched from
|
|
`https://host/p`, the body holds the article text, no script
|
|
text, the link `https://host/a`, and a pipe table.
|
|
- *7.1.5.4.6.* `pdf` reads with pypdf.
|
|
- *7.1.5.4.6.1.* `source_meta` holds the page count and the
|
|
`/Title`, `/Author`, `/Subject`, `/Keywords`, `/CreationDate`,
|
|
and `/ModDate` values that are present. Dates are ISO 8601.
|
|
The title is the metadata title, then the slug.
|
|
- *7.1.5.4.6.2.* With an outline, each outline entry becomes a
|
|
heading before the text of its destination page. The level is
|
|
the entry depth plus 2, capped at 6. Without one, each page
|
|
starts with `## Page <n>`.
|
|
- *7.1.5.4.6.3.* A page with no text adds the warning
|
|
`page <n> has no text layer` and makes the result `partial`.
|
|
When every page is empty, the one warning is
|
|
`no text layer; the PDF may be scanned`.
|
|
- *acceptance*: A two-page fixture with no outline has
|
|
`## Page 1` and `## Page 2`. A fixture with one blank page is
|
|
`partial` and names that page.
|
|
- *7.1.5.5.* Write.
|
|
- *7.1.5.5.1.* The default page is
|
|
`wiki/<default_target>/<slug>.md`. `-t/--target PATH` is
|
|
relative to `wiki/`, and a leading `wiki/` is accepted. A value
|
|
that ends in `.md` names the page. Any other value is a
|
|
directory, and the page inside it is `<slug>.md`. After
|
|
resolving `..` and symlinks the path must be inside `wiki/`, and
|
|
it must not be `wiki/index.md`, `wiki/log.md`, `wiki/schema.md`,
|
|
or a file under `wiki/logs/`. Else exit 1.
|
|
- *acceptance*: `-t concepts` writes
|
|
`wiki/concepts/<slug>.md`. `-t ../raw/x.md` and `-t index.md`
|
|
exit 1.
|
|
- *7.1.5.5.2.* An existing page exits 1 unless `--force` is set. The
|
|
check runs before fetch, so a refusal writes nothing and
|
|
downloads nothing.
|
|
- *acceptance*: A second run on the same source exits 1 and the
|
|
page bytes are unchanged. With `--force` the page is replaced.
|
|
- *7.1.5.5.3.* The page frontmatter is complete when the run ends.
|
|
Every key below is present, in this order, even when its value
|
|
is empty: `[]`, `{}`, or `null`. `sources` is the raw path
|
|
relative to the wiki root. `title` is the converter title, else
|
|
the slug. `ingest` is filled from the fetch, archive, and convert
|
|
stages, so its values are facts about this run.
|
|
|
|
```yaml
|
|
---
|
|
name: <slug>
|
|
version: 0.1.0
|
|
title: <title>
|
|
created: <YYYY-MM-DD>
|
|
updated: <YYYY-MM-DD>
|
|
type: summary
|
|
status: raw
|
|
sources: [<raw path>]
|
|
source_meta: {<key>: <value>}
|
|
ingest:
|
|
origin: <url, or resolved file path>
|
|
origin_kind: <url or file>
|
|
final_url: <url after redirects, or null>
|
|
fetched: <UTC time, YYYY-MM-DDTHH:MM:SSZ>
|
|
media_type: <media type>
|
|
size_bytes: <int>
|
|
sha256: <64 lowercase hex of the raw copy>
|
|
raw_reused: <true or false>
|
|
converter: <converter name>
|
|
conversion: <complete or partial>
|
|
warnings: [<text>]
|
|
tool: fiona-assistant <package version>
|
|
tags: []
|
|
confidence: zero
|
|
llm:
|
|
- model: none
|
|
contested: false
|
|
contradictions: []
|
|
---
|
|
```
|
|
|
|
- *acceptance*: For each fixture, the page frontmatter parses
|
|
and has every key above. `ingest.sha256` equals the SHA-256 of
|
|
the file at `sources[0]`, and `ingest.warnings` equals the
|
|
JSON `warnings`.
|
|
|
|
- *7.1.5.5.4.* The body is `# <title>`, a blank line, the line
|
|
`Raw ingest of [<raw file name>](<link to the raw file>). Not
|
|
refined yet.`, a blank line, and the converted body. When the
|
|
converted body has an H1, every heading in it moves down one
|
|
level, and H6 stays H6. A relative link or image that does not
|
|
resolve from the page is replaced by its text, and the count is
|
|
one warning.
|
|
- *acceptance*: The page has exactly one H1, and mdlinter
|
|
reports no MDL001 for it.
|
|
- *7.1.5.5.5.* After writing, the writer runs the pymarkdown fixer
|
|
on that page only. Violations still left add the warning
|
|
`<N> markdown style violations left`.
|
|
- *acceptance*: Every fixture page passes fmlint with no
|
|
violation.
|
|
- *7.1.5.5.6.* `NOT_IMPLEMENTED` When violations are left, human
|
|
stdout prints, before the `Next: refine` line,
|
|
`Next: fix the style violations listed by fiona-assistant mdoctor chklint <page path>.`
|
|
The page must pass `wiki sanity` before it is refined. The JSON
|
|
`warnings` text does not change.
|
|
- *acceptance*: An HTML fixture with a bare URL that the fixer
|
|
leaves prints that line with the page path. A clean fixture
|
|
does not.
|
|
- *7.1.5.6.* Record. A run that archives appends one
|
|
bullet to `wiki/log.md` and bumps that file's patch `version` and
|
|
`updated`. The bullet is
|
|
`- <YYYY-MM-DD> - ingest <origin> -> <raw path> -> <page path> (raw)`,
|
|
or ends `-> <raw path> (no page)` when no page was written. The tool
|
|
does not edit `wiki/index.md` or any other page.
|
|
- *acceptance*: One successful run adds one line to `log.md` and
|
|
leaves `index.md` unchanged.
|
|
- *7.1.5.6.1.* `NOT_IMPLEMENTED` The bullet passes the wiki's own
|
|
markdownlint. The origin is a code span, so a URL is not a bare
|
|
URL (MD034). Each `->` part starts a continuation line indented
|
|
two spaces, so no line is longer than the bullet's longest path:
|
|
|
|
```text
|
|
- <YYYY-MM-DD> - ingest `<origin>`
|
|
-> <raw path>
|
|
-> <page path> (raw)
|
|
```
|
|
|
|
The bullet still counts as one entry for *3.3.4.* and the first
|
|
acceptance above.
|
|
- *acceptance*: After ingesting a URL of 90 characters into a
|
|
fresh wiki, `wiki sanity` reports no markdownlint hit in
|
|
`wiki/log.md`.
|
|
- *7.1.5.7.* `--raw-only` runs fetch, archive, and
|
|
record, and skips convert and write. It exits 0.
|
|
- *acceptance*: No page is written and `conversion` is `none`.
|
|
- *7.1.5.8.* `--dry-run` fetches into a temporary
|
|
directory outside the wiki, so a URL is still downloaded. It
|
|
prints `Dry run - no files will be modified:` and then the lines
|
|
a real run would print. It writes nothing under the wiki, does not
|
|
move the source, and deletes the temporary directory. `dry_run` is
|
|
true in JSON.
|
|
- *acceptance*: File hashes under the wiki match before and after
|
|
a dry run, and an `ingest/` source is still in place.
|
|
- *7.1.5.8.1.* `NOT_IMPLEMENTED` A dry run logs no record that says
|
|
a file was archived, moved, or written. Those records read
|
|
`dry run: would archive <raw path>` and
|
|
`dry run: would write <page path>`.
|
|
- *acceptance*: A dry run at `LOG_LEVEL=INFO` has `would archive`
|
|
on stderr and no line that starts with `archived` after the
|
|
logger name.
|
|
- *7.1.5.9.* `--pending` lists every page under
|
|
`wiki/` whose frontmatter `status` is `raw`, sorted by path, and
|
|
exits 0.
|
|
- *acceptance*: A page changed to `status: refined` is no longer
|
|
listed.
|
|
- *7.1.5.10.* Refinement is the agent's phase. The
|
|
tool never writes `status: refined`. A page is refined when
|
|
`status` is `refined`, `version` is `1.0.0` or later, the quality
|
|
signals are set and `llm.model` is not `none`, the page is listed
|
|
in `wiki/index.md`, and `wiki/log.md` has a line for the
|
|
refinement.
|
|
- *acceptance*: Fiona skill section 18 lists those conditions and
|
|
the steps that meet them, and says when to convert a source by
|
|
hand.
|
|
- *7.1.5.11.* The agent checks the frontmatter the
|
|
tool wrote before it refines the body. It may correct `title`,
|
|
`type`, `tags`, `source_meta`, `ingest.conversion`, and
|
|
`ingest.warnings` when the source shows they are wrong. It does
|
|
not hand-edit `ingest.sha256`, `ingest.size_bytes`,
|
|
`ingest.fetched`, `ingest.media_type`, or `ingest.tool`. When one
|
|
of those is wrong, it re-runs the ingest with `--force` or reports
|
|
the fault to the user.
|
|
- *acceptance*: Fiona skill section 18 has a metadata check step
|
|
that names the fields it may edit and the fields it may not.
|
|
- *7.1.5.12.* Archived sources do not fail
|
|
`wiki sanity`. Every check skips `raw/`.
|
|
- *acceptance*: After ingesting a markdown file with style
|
|
violations, `wiki sanity` reports none from its raw copy.
|
|
- *7.1.5.13.* `--json` prints only the JSON object
|
|
from section 4.7.6 on stdout, on exit 0 and on exit 3. On exit 1
|
|
and exit 4, stdout is empty.
|
|
- *acceptance*: The stdout of an exit 3 run parses with
|
|
`json.loads` and has `conversion: none`.
|
|
- *7.1.5.14.* `NOT_IMPLEMENTED` On exit 4 the agent gets the source
|
|
another way, or asks the user for it. The Fiona skill teaches
|
|
this in section 18.3. The steps, in order:
|
|
- *7.1.5.14.1.* Use a tool the agent already has that can read
|
|
that host with the user's access, such as an MCP server for
|
|
Confluence, SharePoint, Jenkins, or a ticket system, or an
|
|
authenticated API script in the wiki's `utils/scripts/`. Save
|
|
what it returns into `ingest/<slug>.<ext>` in its own format:
|
|
HTML as `.html`, a PDF as `.pdf`, plain text as `.txt` or `.md`.
|
|
Then run the `Next:` command with that file and
|
|
`--origin <url>`.
|
|
- *7.1.5.14.2.* When no tool has access, stop and ask the user to
|
|
save the page into `ingest/`, for example with the browser's
|
|
Save As or Print to PDF. Name the URL, the folder, and the
|
|
command the agent will run once the file is there. Do not
|
|
retry the URL.
|
|
- *7.1.5.14.3.* Never ask the user for a password, token, or
|
|
cookie. Never put credentials in a URL, a command line, a
|
|
page, or the log.
|
|
- *7.1.5.14.4.* A tool that returns a summary instead of the
|
|
source is not the source. Archive the content the tool
|
|
returned as it is, and say in `ingest.warnings` that it came
|
|
through that tool.
|
|
- *acceptance*: Fiona skill section 6.7 lists exit 4. Section 18.3
|
|
has the three cases above (a tool with access, ask the user,
|
|
never ask for credentials) and shows the `--origin` command.
|
|
|
|
### 7.2. housekeeping
|
|
|
|
> Status: Partially Implemented
|
|
|
|
Implements section 4.8. Housekeeping is the routine an LLM runs to
|
|
maintain a wiki. It gathers what needs work; the LLM does the work.
|
|
Paths in details and actions are relative to the wiki root.
|
|
|
|
- *7.2.1.* The steps run in this order: `check-update`, `merge`,
|
|
`ingest`, `sanity`, `graph`. Every step runs, even after a step
|
|
fails.
|
|
- *acceptance*: A wiki with a pending update file and a dead link
|
|
reports `merge` and `sanity` as `FAIL`, and `graph` still runs.
|
|
- *7.2.2.* `check-update` runs the check of *7.1.2.* and writes
|
|
nothing. It is `PASS` when nothing needs an update, a conflict, or
|
|
an addition, and `WARN` otherwise. Its summary is
|
|
`<U> update(s), <C> conflict(s), <A> addition(s)`. Each item is a
|
|
detail line: `update: <path>`, `conflict: <path> -> <update path>`,
|
|
or `add: <path>`. The action is
|
|
`Ask the user, then run fiona-assistant wiki update --dry-run and wiki update --yes.`
|
|
- *acceptance*: An `AGENTS.md` older than the template gives
|
|
`WARN` and the detail `conflict: AGENTS.md -> AGENTS-update.md`,
|
|
and no update file is written.
|
|
- *7.2.3.* `merge` lists every file whose stem ends with `-update`
|
|
outside `personal/`, `raw/`, and `.git/`, except a path matched by
|
|
`fiona.wiki.update.ignore_files` and except `corefiles.json`
|
|
(*7.1.2.15.*). That includes `fiona-settings-update.json`. It is
|
|
`FAIL` when one such file exists.
|
|
Each detail is `<update path> -> <target path>`, followed by a
|
|
space and `(target missing)` when the target is absent. The target
|
|
is the same name without `-update`. The action per file is:
|
|
|
|
```text
|
|
Merge <update path> into <target path>: keep the owner's edits, take the template changes, set version to at least <template version>, then delete <update path>.
|
|
```
|
|
|
|
`<template version>` is the update file's `version`, or the text
|
|
`the template version` when it has none.
|
|
- *acceptance*: A `wiki/schema-update.md` at `1.5.0` gives the
|
|
detail `wiki/schema-update.md -> wiki/schema.md` and an action
|
|
that names `1.5.0`.
|
|
- *acceptance*: `fiona-settings-update.json` gives the detail
|
|
`fiona-settings-update.json -> fiona-settings.json`.
|
|
- *acceptance*: `corefiles.json` at the wiki root is not a detail.
|
|
A `notes-update.md` whose path is in `ignore_files` is not a
|
|
detail.
|
|
- *7.2.4.* `ingest` lists the markdown pages under `wiki/` whose
|
|
frontmatter has `status: raw`. It is `WARN` when one exists, with
|
|
one detail per page and the action
|
|
`Refine each raw page, then set status: refined.`
|
|
- *acceptance*: A page with `status: raw` is `WARN` and named in
|
|
the details. With none, the step is `PASS`.
|
|
- *7.2.5.* `sanity` runs the checks of *7.1.3.* without the progress
|
|
display. It is `FAIL` when any check has a violation. The summary
|
|
is `<N> violation(s) in <K> check(s)`. The details are each
|
|
failing check's report lines. The action is
|
|
`Fix the violations, then run fiona-assistant housekeeping again.`
|
|
- *acceptance*: One dead link makes the step `FAIL` and a detail
|
|
line names the file.
|
|
- *7.2.5.1.* `NOT_IMPLEMENTED` The details are one line per
|
|
violation, `<check>: <path>:<line>:<column> <code> <message>`,
|
|
with the path relative to the wiki root. Report headers,
|
|
DEFINITIONS, and SUMMARY lines are not details. The full report
|
|
stays with `wiki sanity`.
|
|
- *acceptance*: A wiki with three MD013 hits and one dead link
|
|
gives exactly four `sanity` detail lines.
|
|
- *7.2.6.* `graph` writes `<wiki root>/wiki_graph.html` with the
|
|
defaults of section 4.7.5 and the wiki's graph settings. The summary
|
|
is `wrote wiki_graph.html (<nodes> nodes, <edges> edges)`. A
|
|
markdown page under `wiki/` with no inbound and no outbound link is
|
|
an orphan: the step is `WARN`, one detail `orphan: <path>` per
|
|
page, and the action
|
|
`Link each orphan page from a related page or from wiki/index.md.`
|
|
Update files are not orphans. An error writing the graph is `FAIL`
|
|
with the error as the detail.
|
|
- *acceptance*: After the run `wiki_graph.html` exists at the wiki
|
|
root, and an unlinked page under `wiki/` is named as an orphan.
|
|
- *7.2.7.* Housekeeping writes only `wiki_graph.html`. It does not
|
|
run `wiki update`, merge an update file, or fix a violation.
|
|
- *acceptance*: Apart from `wiki_graph.html`, every file hash is
|
|
unchanged after a run.
|
|
- *7.2.8.* `Next steps:` lists each step's actions in step order,
|
|
without repeats. `passed` is true when no step is `FAIL`.
|
|
- *acceptance*: `--json` stdout parses with `json.loads`, and a
|
|
wiki with only `WARN` steps has `passed: true` and exit 0.
|
|
|
|
---
|
|
|
|
### 7.3. librarian
|
|
|
|
Requirements for `librarian`.
|
|
|
|
#### 7.3.1. Folder and In-file TOC
|
|
|
|
> Status: Implemented
|
|
|
|
- *7.3.1.1.* Folder TOC (`librarian toc`).
|
|
- *7.3.1.1.1.* `table_of_contents.md` has frontmatter `name`,
|
|
`version`, `title`, `created`, `updated`, and `type: toc`.
|
|
`name` is `Table of Contents` and `version` is `1.0.0`.
|
|
- *7.3.1.1.2.* Entries list subfolders, file links, and `v<version>`,
|
|
or `v0.0.0` when version is missing or unreadable.
|
|
- *7.3.1.1.3.* Headers become nested GitHub-style anchor links.
|
|
Fenced code is not scanned. Files are read as `utf-8-sig` so a
|
|
BOM does not hide the first header.
|
|
- *7.3.1.1.4.* Skip the TOC file itself, hidden folders, and
|
|
`--exclude` matches.
|
|
- *7.3.1.1.5.* `--recursive-toc` writes a TOC in every subfolder.
|
|
- *7.3.1.1.6.* `--non-recursive` lists only markdown files directly
|
|
in the folder, ignoring `max_depth_from_dir`.
|
|
- *7.3.1.1.7.* `max_depth_from_file` is the deepest header level
|
|
(3 lists h1 through h3). `max_depth_from_dir` is how many
|
|
subfolder levels to include (`1` includes the folder and its
|
|
children). Absent or `0` means unlimited. Any other non-integer
|
|
or negative value exits 1 and names the key.
|
|
- *7.3.1.2.* In-file TOC (`librarian toc file`).
|
|
- *7.3.1.2.1.* Insert directly under the H1, after frontmatter. The
|
|
H1 itself is not listed. Headings inside fences are not listed.
|
|
A `#` inside frontmatter is not the H1.
|
|
- *7.3.1.2.2.* The block is wrapped in these markers. `sectionized`
|
|
is present only when numbering was on.
|
|
|
|
```html
|
|
<!-- toc-start: headings 2..6 -->
|
|
<!-- toc-end -->
|
|
```
|
|
|
|
Example with numbering:
|
|
`<!-- toc-start: sectionized, headings 2..6 -->`.
|
|
- *7.3.1.2.3.* On regeneration, marker parameters win, then
|
|
`--sectionized`, then settings. A `sectionized` marker keeps
|
|
numbering. The recorded heading range stays, so a later settings
|
|
change does not retarget an existing TOC. A bare
|
|
`<!-- toc-start -->` has no parameters and is upgraded on
|
|
regeneration. An invalid range is ignored with a warning.
|
|
- *7.3.1.2.4.* The block is replaced in place. A current block is
|
|
reported unchanged and not rewritten. Blank lines separate the
|
|
markers from the list so MD032 passes.
|
|
- *7.3.1.2.5.* When `sectionize` is true, entries and heading lines
|
|
gain numbers such as `1.1.`. Existing numbers are stripped before
|
|
renumbering so anchors still match.
|
|
- *7.3.1.2.6.* `toc file *` root is the nearest `fiona-settings.json`,
|
|
current directory first, then subdirectories. Skip hidden folders
|
|
and `build/`, `dist/`, `__pycache__/`, `*.egg-info/`. If none is
|
|
found, use the current directory.
|
|
- *7.3.1.2.7.* `toc file *` skips read-only zones, hidden files and
|
|
folders, `table_of_contents.md`, and `ignore_files` matches.
|
|
- *7.3.1.2.8.* `toc file <explicit path>` inside `personal/` or
|
|
`raw/`, or a missing or non-markdown path, exits 1 and writes
|
|
nothing.
|
|
- *7.3.1.3.* Two markdown files in a folder produce one TOC that
|
|
lists both versions and their headers, and omits a header that
|
|
sits inside a fence.
|
|
- *acceptance*: The fence header's anchor is absent.
|
|
- *7.3.1.4.* `max_depth_from_file: 3` omits `####` and lists `###`.
|
|
Absent or `0` lists every level.
|
|
- *acceptance*: The `####` line is absent only in the capped run.
|
|
- *7.3.1.5.* `max_depth_from_dir: 1` omits files two levels down.
|
|
Absent or `0` lists them. `--non-recursive` lists only the
|
|
folder's own files.
|
|
- *acceptance*: The deep file is absent in both limited modes.
|
|
- *7.3.1.6.* Re-running `toc file` replaces the marked block and
|
|
does not append a second TOC.
|
|
- *acceptance*: The file contains one `toc-start` marker.
|
|
- *7.3.1.7.* `min_heading: 3` omits `##`. `max_heading: 3` omits
|
|
`####`.
|
|
- *acceptance*: Each setting hides only its out-of-range level.
|
|
- *7.3.1.8.* An `ignore_files` match is not modified by `toc file *`.
|
|
An explicit path in `personal/` exits 1 and is unchanged.
|
|
- *acceptance*: Both files keep their original bytes.
|
|
|
|
#### 7.3.2. Non-markdown Version Tags
|
|
|
|
> Status: Planned
|
|
|
|
- *7.3.2.1.* `NOT_IMPLEMENTED` Python and shell files gain a version
|
|
tag. The tag format is undecided (Appendix C). Do not implement
|
|
until that format is chosen.
|
|
- *acceptance*: The format is written in this spec before any
|
|
scanner ships.
|
|
|
|
#### 7.3.3. Move with Link Rewrite
|
|
|
|
> Status: Planned
|
|
|
|
Implements section 4.1.2. In this section a link is one of:
|
|
|
|
- an inline link `[text](path)`
|
|
- an image ``
|
|
- a reference definition `[id]: path` at the start of a line,
|
|
after up to three spaces
|
|
|
|
A relative link holds a path relative to the folder of the file
|
|
that contains it. A markdown file is a file whose suffix is `.md`,
|
|
in any case.
|
|
|
|
- *7.3.3.1.* `NOT_IMPLEMENTED` Source and target.
|
|
- *7.3.3.1.1.* `<source>` is an existing file or folder.
|
|
- *7.3.3.1.2.* When `<target>` is an existing folder, the final
|
|
path is `<target>/<source name>`. Otherwise `<target>` is the
|
|
final path. Missing parent folders of the final path are
|
|
created.
|
|
- *7.3.3.1.3.* A folder source moves every file under it,
|
|
hidden files included, and keeps their layout relative to the
|
|
folder. The emptied source folder is removed.
|
|
- *acceptance*: `move wiki/a.md wiki/concepts/` with an existing
|
|
`wiki/concepts/` creates `wiki/concepts/a.md` and removes
|
|
`wiki/a.md`.
|
|
- *acceptance*: `move wiki/a.md wiki/new/b.md` with no `wiki/new/`
|
|
creates `wiki/new/` and `wiki/new/b.md`.
|
|
- *acceptance*: `move wiki/notes wiki/archive/notes` moves
|
|
`wiki/notes/x.md` and `wiki/notes/img/p.png` to the same
|
|
relative places, and `wiki/notes/` no longer exists.
|
|
- *7.3.3.2.* `NOT_IMPLEMENTED` Refusals. Each prints its line on
|
|
stderr, writes nothing, and exits 1.
|
|
- *7.3.3.2.1.* Missing source:
|
|
`Error: '<source>' does not exist.`
|
|
- *7.3.3.2.2.* The final path exists:
|
|
`Error: '<final path>' already exists.`
|
|
- *7.3.3.2.3.* The final path is the source, or lies inside a
|
|
folder source: `Error: cannot move '<source>' into itself.`
|
|
- *7.3.3.2.4.* The source, or the final path, is inside
|
|
`personal/` or `raw/` of the wiki root (*1.2.3.*). The wiki
|
|
root is found as in *7.3.1.2.6.*:
|
|
`Error: '<path>' is in a read-only zone.`
|
|
- *acceptance*: Each refusal leaves every file under the wiki
|
|
root byte-identical and creates no folder.
|
|
- *7.3.3.3.* `NOT_IMPLEMENTED` Outbound links in each moved
|
|
markdown file are rewritten.
|
|
- *7.3.3.3.1.* The link path is percent-decoded and resolved
|
|
against the file's old folder. When it names an existing file
|
|
or folder, the new path is the relative path from the file's
|
|
new folder to that target's location after the move. A target
|
|
moved in the same run counts at its new location.
|
|
- *7.3.3.3.2.* The new path uses `/`. A `#fragment` and a
|
|
`?query` are kept as written. A link title (`"t"`, `'t'`, or
|
|
`(t)`) is kept. A path written in `<...>` stays in `<...>`. A
|
|
leading `./` is kept when the old path had one. Outside
|
|
`<...>`, a space is written as `%20`.
|
|
- *7.3.3.3.3.* A link whose new text equals the old text is not
|
|
rewritten and not counted.
|
|
- *acceptance*: `wiki/a.md` holds `[b](b.md#x)` and
|
|
``. After `move wiki/a.md wiki/concepts/a.md`
|
|
the file holds `[b](../b.md#x)` and ``,
|
|
and stdout has `rewrote 2 link(s) in wiki/concepts/a.md`.
|
|
- *acceptance*: `wiki/notes/a.md` links `b.md` (a sibling) and
|
|
`../index.md`. After `move wiki/notes wiki/archive/notes`,
|
|
`b.md` is unchanged and `../index.md` becomes
|
|
`../../index.md`.
|
|
- *acceptance*: `[d]: ../docs/My%20File.md` keeps its
|
|
`%20` after the rewrite.
|
|
- *7.3.3.4.* `NOT_IMPLEMENTED` These stay byte-identical:
|
|
- *7.3.3.4.1.* External links: any path with a URI scheme
|
|
(`http:`, `https:`, `mailto:`, or `<scheme>://`).
|
|
- *7.3.3.4.2.* Pure anchors (`#section`) and absolute paths (a
|
|
leading `/`, or a Windows drive letter).
|
|
- *7.3.3.4.3.* Wikilinks `[[page]]`. They resolve by stem, so a
|
|
move that keeps the file name does not change them. A rename is
|
|
*7.3.3.6.3.*
|
|
- *7.3.3.4.4.* Text inside fenced code and inline code, HTML
|
|
tags such as `<a href>` and `<img src>`, and autolinks
|
|
`<https://...>`.
|
|
- *7.3.3.4.5.* Frontmatter. `sources:` paths are relative to the
|
|
wiki root, not to the file.
|
|
- *7.3.3.4.6.* A relative link that did not resolve before the
|
|
move. It is counted on an `unresolved` line.
|
|
- *acceptance*: A moved file that holds one example of each item
|
|
above differs from the original only in its rewritten links,
|
|
and a dead link adds `unresolved 1 link(s) in <path>`.
|
|
- *7.3.3.5.* `NOT_IMPLEMENTED` A file that is moved or rewritten
|
|
keeps everything that is not a rewritten link: encoding, a UTF-8
|
|
BOM, line endings, the final newline, and frontmatter `version`
|
|
and `updated`. A file that is not markdown moves byte for byte.
|
|
- *acceptance*: A CRLF markdown file still has CRLF endings after
|
|
the move, and a moved PNG has the same SHA-256.
|
|
- *7.3.3.6.* `NOT_IMPLEMENTED` Inbound links. The command searches
|
|
the whole wiki for references to the moved paths and points them
|
|
at the new paths, so the move leaves no dead link.
|
|
- *7.3.3.6.1.* The search covers every markdown file under the
|
|
wiki root (*7.3.1.2.6.*) that did not move. Hidden folders
|
|
(`.git` included) are skipped.
|
|
- *7.3.3.6.2.* A relative link in such a file that resolves,
|
|
before the move, to the source, or to any file or folder under
|
|
a folder source, is rewritten to the relative path from that
|
|
file's folder to the new location. *7.3.3.3.2.* and
|
|
*7.3.3.3.3.* apply. *7.3.3.4.* applies, except its last item:
|
|
a link that does not point at a moved path is left alone and
|
|
is not counted as unresolved.
|
|
- *7.3.3.6.3.* When a moved markdown file changes its stem,
|
|
every wikilink in the wiki, in moved files too, whose target
|
|
equals the old stem (case-insensitive, as MDL002 matches) is
|
|
rewritten to the new stem. A `#section` and a `|label` are
|
|
kept. A wikilink is left alone when another markdown file in
|
|
the wiki still has the old stem after the move.
|
|
- *7.3.3.6.4.* A file inside `personal/` or `raw/` is never
|
|
written (*1.2.3.*). Each of its links that point at a moved
|
|
path is counted on a `read-only` line, so the user sees what
|
|
will be dead.
|
|
- *7.3.3.6.5.* A file with nothing to rewrite stays
|
|
byte-identical.
|
|
- *acceptance*: `wiki/index.md` holds `[a](a.md#top)`. After
|
|
`move wiki/a.md wiki/concepts/a.md` it holds
|
|
`[a](concepts/a.md#top)`, and stdout has
|
|
`rewrote 1 link(s) in wiki/index.md`.
|
|
- *acceptance*: `wiki/log.md` links `notes/x.md` and
|
|
`notes/img/p.png`. After `move wiki/notes wiki/archive/notes`
|
|
those links are `archive/notes/x.md` and
|
|
`archive/notes/img/p.png`.
|
|
- *acceptance*: `wiki/index.md` holds `[[a#intro|A page]]`.
|
|
After `move wiki/a.md wiki/b.md` it holds
|
|
`[[b#intro|A page]]`. When `wiki/other/a.md` also exists, the
|
|
wikilink is unchanged.
|
|
- *acceptance*: `raw/n.md` links `../wiki/a.md`. After the move
|
|
`raw/n.md` is byte-identical and stdout has
|
|
`read-only 1 link(s) in raw/n.md`.
|
|
- *acceptance*: After a move with no `read-only` and no
|
|
`unresolved` line, `wiki sanity` reports no MDL001 or MDL002
|
|
that the move caused.
|
|
- *7.3.3.7.* `NOT_IMPLEMENTED` Order of work: check every refusal,
|
|
compute every rewrite, then move and write. `--dry-run` stops
|
|
after the compute step and prints the same lines a real run
|
|
prints, each with the `[dry-run]` prefix. An `OSError` during
|
|
the write step prints `Error: <message>` on stderr and exits 1.
|
|
- *acceptance*: `--dry-run` leaves the tree byte-identical, and
|
|
its stdout equals the stdout of the real run with each line
|
|
prefixed.
|
|
- *7.3.3.8.* `NOT_IMPLEMENTED` The command runs no git operation.
|
|
The wiki records the move with `git save` (*3.3.3.*).
|
|
- *acceptance*: In a git wiki the commit count is the same before
|
|
and after `move`.
|
|
|
|
### 7.4. mdoctor
|
|
|
|
> Status: Partially Implemented
|
|
|
|
Implements section 4.2 and the pymarkdown keys in section 5.1.
|
|
Settings load at construction. The default settings path is the
|
|
packaged template `fiona-settings.json`. Callers, including
|
|
`run_sanity.py`, may pass another path. A missing file means plain
|
|
pymarkdown defaults.
|
|
|
|
- *7.4.1.* `mdoctor fix` repairs an auto-fixable violation such as
|
|
MD047 and lists the path.
|
|
- *acceptance*: The file ends with a newline and stdout contains
|
|
`PyMarkdownFix: 1 file(s) fixed`.
|
|
- *7.4.2.* A tree with nothing to fix prints
|
|
`PyMarkdownFix: 0 file(s) fixed`.
|
|
- *acceptance*: That line is the only stdout line.
|
|
- *7.4.3.* A scan does not ask PyMarkdown to run git. When
|
|
`respect_gitignore` is true and the path is inside the working
|
|
directory, directory names from `.gitignore` and `.git` are
|
|
exclude patterns. A path outside the working directory is still
|
|
skipped by the directory-name check after the scan.
|
|
`respect_gitignore` false still scans those directories, except
|
|
`.git` when the path is inside the working directory.
|
|
- *acceptance*: `scan_path` is called with `respect_gitignore`
|
|
false. Inside the working directory the exclude list contains
|
|
`.git/` and each plain directory name from `.gitignore`.
|
|
- *7.4.4.* `NOT_IMPLEMENTED` `chklint` and `fix` accept one markdown
|
|
file as well as a directory, the same as `fmlint chklint`. A path
|
|
that does not exist prints `Error: '<path>' does not exist.` on
|
|
stderr and exits 1.
|
|
- *acceptance*: `mdoctor fix wiki/sources/a.md` lists only that
|
|
file, and every other file hash is unchanged.
|
|
|
|
### 7.5. fmlint
|
|
|
|
> Status: Implemented
|
|
|
|
Implements sections 4.3 and 6.2.
|
|
|
|
- *7.5.1.* `required_fields` in settings replaces the default pair
|
|
on the next run.
|
|
- *acceptance*: Adding `title` to the list reports a file that
|
|
previously passed.
|
|
- *7.5.2.* `enabled: false` reports no frontmatter violations.
|
|
- *acceptance*: A file with no frontmatter is clean for fmlint.
|
|
- *7.5.3.* A code in `ignore_rules` is omitted. A glob in
|
|
`ignore_file` skips those paths.
|
|
- *acceptance*: `ignore_rules: ["FML002"]` hides a missing-block
|
|
finding and still reports a missing `name`.
|
|
|
|
### 7.6. mdlinter
|
|
|
|
> Status: Implemented
|
|
|
|
Implements section 6.3. Checks run in registry order. A new check
|
|
is a method plus an entry in that registry. `lint_file` resolves
|
|
wikilinks against the file's parent tree. `lint_directory` scans
|
|
the wiki.
|
|
|
|
- *7.6.1.* The report header is `Markdown Lint Violations` and the
|
|
section order matches section 6.1.
|
|
- *acceptance*: DEFINITIONS lists `MDL001: Dead Link` when a dead
|
|
markdown link exists.
|
|
|
|
### 7.7. pycustomlint
|
|
|
|
> Status: Implemented
|
|
|
|
Implements section 6.4. The sanity runner in section 7.14 includes
|
|
it as `pycustomlint`. There is no `fiona-assistant pycustomlint`
|
|
command; the repository runner calls the library.
|
|
|
|
- *7.7.1.* `--json` prints a JSON list of violation objects.
|
|
- *acceptance*: `json.loads` succeeds and each object has `code`.
|
|
|
|
### 7.8. python-ruff
|
|
|
|
> Status: Implemented
|
|
|
|
Implements section 4.4. The sanity check name is `ruff` in
|
|
`run_sanity.py` and `python-ruff` in wiki sanity.
|
|
|
|
- *7.8.1.* An unused import appears in DEFINITIONS, FILE, and
|
|
SUMMARY.
|
|
- *acceptance*: The FILE line contains the ruff rule code.
|
|
- *7.8.2.* `NOT_IMPLEMENTED` A `python-ruff fix` subcommand mirrors
|
|
`mdoctor fix` and calls the existing `fix=True` path.
|
|
- *acceptance*: An auto-fixable ruff finding is gone after the
|
|
command and the path is listed.
|
|
|
|
### 7.9. Shellcheck
|
|
|
|
> Status: Planned
|
|
|
|
- *7.9.1.* `NOT_IMPLEMENTED` Wrap the `shellcheck` binary and print
|
|
the shared report.
|
|
- *acceptance*: A known shellcheck finding shows file, line, and
|
|
rule code.
|
|
- *7.9.2.* `NOT_IMPLEMENTED` A missing binary prints an install hint
|
|
and exits non-zero, with no traceback.
|
|
- *acceptance*: The message names `shellcheck`.
|
|
- *7.9.3.* `NOT_IMPLEMENTED` Wiki sanity runs this check.
|
|
- *acceptance*: The `Results:` table includes the new check name.
|
|
|
|
### 7.10. Markdown Code Blocks
|
|
|
|
> Status: Planned
|
|
|
|
- *7.10.1.* `NOT_IMPLEMENTED` Extract fenced blocks. Lint Python
|
|
blocks with Ruff and shell blocks with shellcheck. Skip other
|
|
languages.
|
|
- *acceptance*: An unused import inside a Python fence is
|
|
reported. Prose outside fences is not.
|
|
- *7.10.2.* `NOT_IMPLEMENTED` The finding names the markdown path,
|
|
the fence index, the line inside the fence, and the markdown
|
|
line number.
|
|
- *acceptance*: The reported markdown line is the fence line that
|
|
contains the import.
|
|
- *7.10.3.* `NOT_IMPLEMENTED` Wiki sanity runs this check.
|
|
- *acceptance*: The check appears in the `Results:` table.
|
|
|
|
### 7.11. json-lint
|
|
|
|
> Status: Implemented
|
|
|
|
Implements sections 4.5 and 6.5. Both sanity runners include it.
|
|
|
|
- *7.11.1.* A JSON violation fails `run_sanity.py` and wiki sanity.
|
|
- *acceptance*: The `Results:` row for `jsonlint` is FAIL and the
|
|
process exit code is non-zero.
|
|
- *7.11.2.* An empty tree of JSON files prints no violations.
|
|
- *acceptance*: The report says no violations were found.
|
|
|
|
### 7.12. Git Operations
|
|
|
|
> Status: Implemented
|
|
|
|
Implements section 4.6. `wiki init` writes the repository
|
|
`.gitignore` patterns and `personal/` (section 7.1.1). Those paths
|
|
are never staged by these commands except where `git add --all`
|
|
respects that ignore file.
|
|
|
|
- *7.12.1.* `save --description note` on a dirty tree creates one
|
|
commit whose message starts with the UTC timestamp and ends with
|
|
`note`.
|
|
- *acceptance*: `git log -1 --format=%s` matches
|
|
`<YYYY-MM-DD HH:MM:SS> note`.
|
|
- *7.12.2.* `snapshot demo --description note` then
|
|
`restore demo` returns file bytes to the tagged tree and adds a
|
|
new commit. The pre-restore history is still reachable.
|
|
- *acceptance*: `git rev-parse HEAD~1` after restore is the
|
|
snapshot commit or a descendant, and the tagged tree matches
|
|
the worktree.
|
|
- *7.12.3.* `undo` and `restore` on a dirty tree exit 1 and mention
|
|
`clean`.
|
|
- *acceptance*: The HEAD sha is unchanged.
|
|
- *7.12.4.* `clean` leaves `git status` empty of changes.
|
|
- *acceptance*: A following `status` prints `Status: clean`.
|
|
|
|
### 7.13. Backup Script
|
|
|
|
> Status: Planned
|
|
|
|
- *7.13.1.* `NOT_IMPLEMENTED` A bundled command writes a restorable
|
|
archive of the wiki.
|
|
- *acceptance*: Restoring that archive reproduces the wiki tree
|
|
from before the archive.
|
|
- *7.13.2.* `NOT_IMPLEMENTED` The archive is stored outside the wiki
|
|
tree.
|
|
- *acceptance*: The archive path is not under the wiki root.
|
|
- *7.13.3.* `NOT_IMPLEMENTED` The filename is
|
|
`YYYYMMDD_hhmm_<description>.zstd`.
|
|
- *acceptance*: The file is valid zstd and the name matches.
|
|
- *7.13.4.* `NOT_IMPLEMENTED` The command asks for a directory and
|
|
uses `~/Downloads` when the user accepts the default.
|
|
- *acceptance*: Pressing Enter writes the archive under
|
|
`~/Downloads`.
|
|
|
|
### 7.14. Test and Sanity Runners
|
|
|
|
> Status: Implemented
|
|
|
|
- *7.14.1.* Shared engine, used by `run_sanity.py` and wiki sanity.
|
|
- *7.14.1.1.* One thread per check. Results stay in registry order.
|
|
- *7.14.1.2.* Summary line `PASS/FAIL - <detail> (MM:SS)`. Minutes
|
|
may exceed 59 and are zero-padded. A missing duration omits the
|
|
suffix.
|
|
- *7.14.1.3.* `SANITY PASSED (MM:SS)` lists check names in registry
|
|
order. `Sanity Failed - N checks need attention:` has no total
|
|
time.
|
|
- *7.14.1.4.* A runner may add an extra result row, such as
|
|
coverage, and per-check failure details.
|
|
- *7.14.1.5.* `progress(name, result)` runs when a check finishes.
|
|
`heartbeat(remaining_names)` runs about every 0.5 seconds. Start
|
|
and finish are logged at INFO.
|
|
- *7.14.2.* Progress display.
|
|
- *7.14.2.1.* On a TTY, one bar per check, redrawn in place with
|
|
cursor-up and line clear. Example:
|
|
`pytests [###------------] 22% 6.5s`.
|
|
- *7.14.2.2.* The bar fills as elapsed / `expected-seconds`, capped
|
|
at 100%.
|
|
- *7.14.2.3.* A finished check shows a full bar and `(MM:SS)`:
|
|
`markdownlint [###############] 100% done (00:07)`.
|
|
- *7.14.2.4.* The final block stays on screen above the reports.
|
|
- *7.14.2.5.* `--non-interactive`, also when stdout is not a TTY,
|
|
prints `Running Sanity` or `Running Wiki Sanity` once, then one
|
|
dot per `interval` seconds. A run shorter than the interval
|
|
prints no dots. A newline ends the line before the report. No
|
|
ANSI.
|
|
- *7.14.2.6.* `--json` on wiki sanity skips the display.
|
|
- *7.14.2.7.* Bar scale and dot interval come from section 5.2.
|
|
- *7.14.3.* `run_tests.py` and the pytests check.
|
|
- *7.14.3.1.* The command includes `--color=no` and JUnit XML
|
|
`junit_family=xunit1`.
|
|
- *7.14.3.2.* Displayed messages prefer regex-parsed pytest output.
|
|
JUnit fills missing fields and replaces empty messages. Match by
|
|
basename plus test name so path spellings do not duplicate a
|
|
failure.
|
|
- *7.14.3.3.* A missing short summary logs a WARNING that names
|
|
`PYTEST_ADDOPTS`, `--no-summary`, and plugin output as likely
|
|
causes.
|
|
- *7.14.3.4.* `LOG_LEVEL=DEBUG` logs the pytest command, return
|
|
code, parsed failures, and raw output when the parse disagrees.
|
|
- *7.14.3.5.* The report nests failures as `Class : test_name`
|
|
under the file, then the assert text, or the exception text when
|
|
there is no assert line.
|
|
- *7.14.3.6.* The `Sanity Failed` verdict lists failing test names
|
|
only. Assert text stays in the report body.
|
|
- *7.14.3.7.* The `coverage:` line is
|
|
`PASS/FAIL - <percent>% (required <percent>%)` and is untimed. A
|
|
shortfall is its own verdict entry.
|
|
- *7.14.4.* `run_sanity.py` check order is `pytests`, `markdownlint`,
|
|
`ruff`, `frontmatter_lint`, `pycustomlint`, `jsonlint`.
|
|
- *7.14.5.* A failing test is listed under its file with the assert
|
|
text when pytest is run with `--no-summary` or `PY_COLORS=1`.
|
|
- *acceptance*: The name and assert still appear.
|
|
- *7.14.6.* A green run prints every `Results:` line with `(MM:SS)`
|
|
and ends with `SANITY PASSED (MM:SS)`.
|
|
- *acceptance*: The banner time is the wall time of the whole run.
|
|
- *7.14.7.* A piped run contains no ANSI and prints one dot per
|
|
interval after the label, unless the run is shorter than the
|
|
interval.
|
|
- *acceptance*: The captured stdout has no ESC byte.
|
|
|
|
### 7.15. html-js-lint
|
|
|
|
> Status: Planned
|
|
|
|
The `html-js-lint` console script is not installed until this
|
|
section is built. The intended tool validates HTML with
|
|
`html5validator` (Java 8+) and JavaScript with `eslint` on PATH.
|
|
|
|
- *7.15.1.* `NOT_IMPLEMENTED` HTML files (default `.html` and
|
|
`.htm`) are checked. Findings include path, line, and column.
|
|
`.gitignore` and read-only zones are skipped.
|
|
- *acceptance*: An unclosed tag is reported. A valid file is not.
|
|
A tree with no HTML prints no HTML findings.
|
|
- *7.15.2.* `NOT_IMPLEMENTED` JavaScript files (default `.js` and
|
|
`.javascript`) are checked with eslint, honoring `.eslintrc`.
|
|
- *acceptance*: A syntax error names path, line, and message.
|
|
A clean file produces no finding.
|
|
- *7.15.3.* `NOT_IMPLEMENTED` Inline `<script>` is linted only when
|
|
`html.lint_inline_js` is true (default false). Findings use the
|
|
HTML path and line.
|
|
- *acceptance*: With the flag off, inline script is ignored.
|
|
- *7.15.4.* `NOT_IMPLEMENTED` CLI is
|
|
`fiona-assistant html-js-lint`, default `chklint .`, shared
|
|
report, exit 0 even with violations, and `--fix` where the
|
|
underlying tool can fix.
|
|
- *acceptance*: `--fix` reports how many files changed.
|
|
- *7.15.5.* `NOT_IMPLEMENTED` Settings, under a dedicated section,
|
|
with defaults when the section is missing:
|
|
|
|
| Key | Default |
|
|
| --- | --- |
|
|
| `html.extensions` | `["html", "htm"]` |
|
|
| `html.ignore_patterns` | `[]` |
|
|
| `html.lint_inline_js` | false |
|
|
| `javascript.extensions` | `["js", "javascript"]` |
|
|
| `javascript.eslint_config` | `.eslintrc` |
|
|
| `javascript.ignore_patterns` | `[]` |
|
|
|
|
- *acceptance*: A custom extension list is the set that is scanned.
|
|
- *7.15.6.* `NOT_IMPLEMENTED` Both `run_sanity.py` and wiki sanity
|
|
run this check.
|
|
- *acceptance*: A violation fails that sanity run.
|
|
|
|
### 7.16. settings setup
|
|
|
|
> Status: Planned
|
|
|
|
`fiona-assistant init` remains the non-interactive wiki bootstrap
|
|
(section 7.1.1): it copies the template and does not ask questions.
|
|
`fiona-assistant setup` is a separate command for reviewing settings
|
|
on an existing wiki. It does not create the wiki tree.
|
|
|
|
- *7.16.1.* `NOT_IMPLEMENTED` `setup` walks every configurable key
|
|
in `fiona-settings.json`, shows the current value, and accepts
|
|
Enter as "keep current".
|
|
- *acceptance*: Enter through every prompt leaves the file
|
|
semantically unchanged.
|
|
- *7.16.2.* `NOT_IMPLEMENTED` Booleans can be toggled. Strings and
|
|
integers accept a typed value. Values are checked before write.
|
|
- *acceptance*: A non-integer for `indent_len` is rejected and the
|
|
file is not written.
|
|
- *7.16.3.* `NOT_IMPLEMENTED` `--yes` or `--default` applies defaults
|
|
with no prompts.
|
|
- *acceptance*: The command exits 0 on a pipe when the flag is set.
|
|
- *7.16.4.* `NOT_IMPLEMENTED` A missing settings file is created
|
|
from the template defaults. A corrupt file is reported and not
|
|
silently replaced.
|
|
- *acceptance*: Exit 0 after creating a missing file. Exit non-zero
|
|
on corrupt JSON, with the parse error on stderr.
|
|
- *7.16.5.* `NOT_IMPLEMENTED` `setup` can be run again later to
|
|
change an existing wiki.
|
|
- *acceptance*: A second run can change `log_level` and leave
|
|
every other key as it was.
|
|
|
|
### 7.17. Logging
|
|
|
|
> Status: Partially Implemented
|
|
|
|
The contract is section 2. Reports stay on stdout. Diagnostics go
|
|
through the logging module to stderr and, when configured, to the
|
|
file sink.
|
|
|
|
- *7.17.1.* Each CLI entry calls the shared setup before doing work.
|
|
- *acceptance*: `LOG_LEVEL=DEBUG` emits debug lines on stderr and
|
|
the human report stays on stdout.
|
|
- *7.17.2.* Dropping `file_log_location` leaves no file handler.
|
|
- *acceptance*: No log file is created for that run.
|
|
- *7.17.3.* `NOT_IMPLEMENTED` The shipped `fiona-settings.json` sets
|
|
`fiona.log_level` to `WARNING`, the same as the default in
|
|
section 5.1. `fiona.file_log_level` stays `INFO`, so `fiona.log`
|
|
keeps the detail. PowerShell shows every native stderr line as a
|
|
`NativeCommandError`, so an `INFO` record looks like a failure
|
|
even when the exit code is 0. When this is built, the shipped
|
|
document in section 5.1 changes to match.
|
|
- *acceptance*: In a fresh wiki, `wiki sanity` on the clean
|
|
template and `wiki ingest` of a local markdown file write nothing
|
|
on stderr, and `fiona.log` has their `INFO` lines.
|
|
|
|
### 7.18. Documentation
|
|
|
|
> Status: Partially Implemented
|
|
|
|
`README.md` links to this spec. `src/README.md`,
|
|
`src/fiona/librarian/README.md`, and `src/fiona/mdoctor/README.md` exist and
|
|
are behind the current CLIs.
|
|
|
|
- *7.18.1.* `NOT_IMPLEMENTED` `librarian/README.md` documents `list`
|
|
and `toc`. `mdoctor/README.md` documents `list`, `chklint`, and
|
|
`fix`.
|
|
- *acceptance*: A new reader can install and run each command
|
|
from those files alone.
|
|
- *7.18.2.* `src/README.md` describes `src/fiona/` (including
|
|
`wiki/`, `python_ruff/`, and `data/llm_wiki/`), repo-root `tests/`,
|
|
and `scripts/` (`run_tests.py`, `run_sanity.py`, `settings_dev.json`).
|
|
- *acceptance*: Those names appear in the structure section.
|
|
- *7.18.3.* Documentation markdown passes `mdoctor chklint`.
|
|
- *acceptance*: `README.md` contains a markdown link to `SPEC.md`
|
|
and the lint report is clean.
|
|
- *7.18.4.* `NOT_IMPLEMENTED` The README Quick Start has a short
|
|
"After ingest" and "On Windows" part. It says that a file in
|
|
`ingest/` moves into `raw/` and a second run on it fails, that
|
|
`LOG_LEVEL=ERROR` hides diagnostics until *7.17.3.* ships, the loop
|
|
`housekeeping`, then fix style violations, then refine and set
|
|
`status: refined`, that `--version` (*2.3.*) or `pip show
|
|
fiona-assistant` gives the installed version, and that commands
|
|
run from the wiki root until section 7.20 is built. The Fiona skill
|
|
carries the same loop.
|
|
- *acceptance*: Each of those five points appears in `README.md`,
|
|
and the after-ingest loop appears in the Fiona skill.
|
|
|
|
### 7.19. Tool Roles
|
|
|
|
> Status: Planned
|
|
|
|
- *7.19.1.* `NOT_IMPLEMENTED` Each `src/` tool has one role:
|
|
ingestion (`ingest/` to `raw/`), processing (`raw/` to
|
|
`wiki/concepts/`), or maintenance (`index.md` / `log.md` and
|
|
the linters).
|
|
- *acceptance*: A table in this spec or the README assigns every
|
|
tool exactly one role.
|
|
- *7.19.2.* `NOT_IMPLEMENTED` Ingestion is `wiki ingest`
|
|
(section 7.1.5). Missing processing steps become scripts under the
|
|
template `utils/` directory, such as `process.py`.
|
|
- *acceptance*: Each new script is named in the role table.
|
|
|
|
### 7.20. Wiki Root Discovery
|
|
|
|
> Status: Planned
|
|
|
|
Today every optional directory defaults to `.`. `toc file *` already
|
|
searches for `fiona-settings.json` as specified in section 7.3.1.
|
|
Graph settings already walk parent directories. The dispatcher does
|
|
not.
|
|
|
|
- *7.20.1.* `NOT_IMPLEMENTED` When the directory argument is omitted,
|
|
commands use the wiki root from Appendix C's discovery rule.
|
|
- *acceptance*: `fiona-assistant mdoctor` from a subdirectory lints
|
|
that root, not the subdirectory.
|
|
- *7.20.2.* `NOT_IMPLEMENTED` An explicit directory always wins.
|
|
- *acceptance*: `mdoctor chklint sub/dir` lints `sub/dir` even when
|
|
a settings file exists higher up.
|
|
- *7.20.3.* `NOT_IMPLEMENTED` With no settings file found, the
|
|
default stays `.`.
|
|
- *acceptance*: Behavior matches today's commands.
|
|
- *7.20.4.* `NOT_IMPLEMENTED` The file is loaded once per process
|
|
and shared. mdoctor already reads it for rules.
|
|
- *acceptance*: Two tools in one invocation see the same object.
|
|
|
|
### 7.21. Bootstrap install modes
|
|
|
|
> Status: Implemented
|
|
|
|
`bootstrap` (bash, sourced) and `bootstrap.ps1` (PowerShell) prepare
|
|
a Python environment and install this repository. With no install
|
|
flag, both create or reuse a virtual environment and run
|
|
`python -m pip install -e ".[dev]"`. `scripts/bootstrap_install.py`
|
|
is the pip-argument source both scripts call.
|
|
|
|
- *7.21.1.* `--local` installs this repository as a normal package.
|
|
The pip arguments are `install` and `.`. They do not include `-e`
|
|
and they do not include the `dev` extra.
|
|
- *acceptance*: `pip_install_args(local=True, no_env=False)` is
|
|
`["install", "."]`.
|
|
- *7.21.2.* `--no-env` does not create or activate a virtual
|
|
environment. The install adds `--user`, so the console scripts
|
|
land in the user scripts directory and run without activating an
|
|
environment.
|
|
- *acceptance*: `pip_install_args(local=False, no_env=True)`
|
|
contains `--user`. PowerShell skips virtual environment creation
|
|
when `$NoEnv` is set. Bash skips it when `NO_ENV` is `true`.
|
|
- *7.21.3.* `--local` and `--no-env` combine. The install is
|
|
`install`, `--user`, and `.`, with no `-e`.
|
|
- *acceptance*: `pip_install_args(local=True, no_env=True)` is
|
|
`["install", "--user", "."]`.
|
|
- *7.21.4.* Neither flag keeps the default: `-e` and `.[dev]`, and
|
|
no `--user`.
|
|
- *acceptance*: `pip_install_args(local=False, no_env=False)` is
|
|
`["install", "-e", ".[dev]"]`.
|
|
- *7.21.5.* Bash adds `--break-system-packages` on every install.
|
|
PowerShell does not.
|
|
- *acceptance*: `break_system_packages=True` inserts that flag
|
|
after `--user` when both are set, and after `install` when
|
|
`--user` is absent.
|
|
- *7.21.6.* A virtual-environment install on PowerShell must be
|
|
dot-sourced so the environment stays active. `--no-env` may be
|
|
executed with `-File`, because there is no environment to keep
|
|
active. Bash is sourced for both modes.
|
|
- *acceptance*: PowerShell reports a dot-source error when
|
|
`--local` is used without `--no-env` and the script is not
|
|
dot-sourced. The message does not call `--local` an unknown
|
|
option.
|
|
|
|
## Appendix A. Non-Goals
|
|
|
|
- *A.1.* No LLM or network API inside Fiona-Assistant. The tools are
|
|
deterministic.
|
|
- *A.2.* No automated edit of `personal/` or `raw/` content.
|
|
`wiki ingest` adds new files under `raw/` and edits none
|
|
(*1.2.3.*).
|
|
- *A.3.* No cloud sync and no remote backup. Version control is local.
|
|
The planned archive in *7.13.* is a local file.
|
|
- *A.4.* Deterministic tools make no cloud API calls. Each command
|
|
uses only local files and finishes with no network access.
|
|
- *acceptance*: With the network unavailable, `wiki sanity` and
|
|
`wiki graph` still complete on a local wiki.
|
|
- *acceptance*: Opening the generated graph HTML is an exception:
|
|
the browser loads D3 from `d3js.org` (*7.1.4.*). The generator
|
|
itself does not call that host.
|
|
- *acceptance*: `wiki ingest <url>` is the other exception. It
|
|
downloads only the URL the user passed (*7.1.5.2.*). With a local
|
|
path it makes no network call.
|
|
|
|
---
|
|
|
|
## Appendix B. Priority Order
|
|
|
|
| Priority | Feature | Section | Status |
|
|
| --- | --- | --- | --- |
|
|
| P1 | wiki update / check-update | 7.1.2 | Implemented |
|
|
| P1 | Git save, snapshot, restore, clean, undo, status | 7.12 | Implemented |
|
|
| P1 | wiki sanity, including dead links | 7.1.3 | Implemented |
|
|
| P1 | housekeeping | 7.2 | Partially Implemented |
|
|
| P1 | wiki init | 7.1.1 | Partially Implemented |
|
|
| P1 | CLI errors without traceback | 1.2.7 | Planned |
|
|
| P1 | wiki graph | 7.1.4 | Partially Implemented |
|
|
| P2 | fmlint | 7.5 | Implemented |
|
|
| P2 | json-lint | 7.11 | Implemented |
|
|
| P2 | mdoctor fix | 7.4 | Partially Implemented |
|
|
| P2 | Logging | 7.17 | Partially Implemented |
|
|
| P2 | Settings schema | 5 | Implemented |
|
|
| P2 | Sanity reporting, coverage, progress | 7.14 | Implemented |
|
|
| P2 | Shellcheck | 7.9 | Planned |
|
|
| P2 | Markdown code-block lint | 7.10 | Planned |
|
|
| P2 | html-js-lint | 7.15 | Planned |
|
|
| P2 | python-ruff fix subcommand | 7.8.2 | Planned |
|
|
| P2 | Wiki root discovery | 7.20 | Planned |
|
|
| P2 | settings setup | 7.16 | Planned |
|
|
| P2 | Bootstrap install modes | 7.21 | Implemented |
|
|
| P2 | wiki ingest | 7.1.5 | Partially Implemented |
|
|
| P2 | Ingest login detection and `--origin` | 7.1.5.2.8, 7.1.5.2.9, 7.1.5.14 | Planned |
|
|
| P3 | `--version` | 2.3 | Planned |
|
|
| P3 | fmlint enforcement of quality signals | 6.2 | Planned |
|
|
| P3 | Backup archive | 7.13 | Planned |
|
|
| P3 | Documentation refresh | 7.18 | Partially Implemented |
|
|
| P2 | Graph skips gitignored files | 7.1.4.17 | Planned |
|
|
| P3 | Node importance filter | 7.1.4.16 | Planned |
|
|
| P2 | Graph directory nodes | 7.1.4.18 | Planned |
|
|
| P2 | Graph live view (`--serve`) | 7.1.4.19 | Planned |
|
|
| P2 | librarian move with link rewrite | 7.3.3 | Planned |
|
|
| P4 | Non-markdown version tags | 7.3.2 | Planned |
|
|
| P4 | Tool-role table | 7.19 | Planned |
|
|
|
|
---
|
|
|
|
## Appendix C. Open Questions
|
|
|
|
- Python and shell version-tag format (section 7.3.2) is undecided.
|
|
Do not build a scanner until this section names the format.
|
|
- Wiki root discovery when several `fiona-settings.json` files exist
|
|
under the start directory: the rule is not chosen. Candidates are
|
|
"the nearest directory wins" and "error when two files sit at the
|
|
same depth". Parent walk is already what graph settings do, and is
|
|
a third candidate for the default directory. Section 7.20 stays
|
|
planned until one rule is written here.
|
|
- json-lint `chklint` exits 1 when violations exist. mdoctor, fmlint,
|
|
and python-ruff exit 0. A future change may align them. Until this
|
|
line changes, implement the split in section 4.
|
|
- `wiki ingest` takes one file or URL. A directory argument, and a
|
|
bare `wiki ingest` that drains `ingest/`, are not specified.
|
|
- More converters for section 7.1.5, each one class under *7.1.5.1.2.*:
|
|
Office files (`mammoth` for docx, `python-pptx`, `openpyxl`), or
|
|
`markitdown` as one optional extra that covers them all but pulls
|
|
in `magika` and `onnxruntime`. OCR for scanned PDFs is not chosen.
|
|
PDF tables need `pdfplumber` or similar; pypdf reads text only.
|
|
- Images and other assets referenced by an ingested source are not
|
|
copied into `raw/assets/`.
|
|
- Third-party converters could register through a package entry
|
|
point group such as `fiona.ingest_converters`. Not decided.
|
|
- The SHA-256 check in *7.1.5.3.3.* scans `raw/` on every run. A
|
|
cached hash index is not specified until a large `raw/` makes the
|
|
scan slow.
|
|
- A directory view (*7.1.4.18.5.*) draws outside nodes that its
|
|
pages point at. Drawing outside nodes that point into the
|
|
directory, as a second setting, is not decided.
|
|
- `--serve` (*7.1.4.19.5.*) reloads the page on every rebuild.
|
|
Patching the drawn graph in place, with no reload, is not
|
|
specified. It needs the D3 joins keyed by node id.
|
|
- `librarian move` (*7.3.3.4.4.*) does not rewrite links in HTML
|
|
tags (`<a href>`, `<img src>`). Whether it should is not
|
|
decided.
|
|
- `git undo` and `git clean` discard work with no prompt. The
|
|
2026-10-02 manual test lost a `git save` commit this way. A TTY
|
|
prompt with `--yes` (*1.2.2.*) would make agents pass `--yes` on
|
|
every call. Not decided.
|
|
- `wiki ingest` sends no credentials (*7.1.5.2.8.1.*). A cookie
|
|
file, a bearer token from an environment variable, or a
|
|
`--header` option would let it read some login-only hosts
|
|
itself. Each one puts secrets near the wiki. Not decided.
|
|
- On exit 4, `--json` prints nothing (*7.1.5.13.*). A JSON error
|
|
object such as `{"error": "auth_required", "url": ...,
|
|
"reason": ...}` would save the agent parsing stderr. Not decided.
|
|
|
|
---
|
|
|
|
## Appendix D. Maintainer Notes
|
|
|
|
These notes are not requirements. They point at the current tree.
|
|
|
|
| Area | Module |
|
|
| --- | --- |
|
|
| CLI dispatch | `src/fiona/__main__.py` |
|
|
| Logging | `src/fiona/logging_config.py` |
|
|
| template path | `src/fiona/paths.py` |
|
|
| librarian list | `src/fiona/librarian/librarian.py` |
|
|
| TOC | `src/fiona/librarian/toc_generator.py` |
|
|
| librarian move (planned) | `src/fiona/librarian/mover.py`, `src/fiona/librarian/link_rewriter.py` |
|
|
| mdoctor | `src/fiona/mdoctor/mdoctor.py` |
|
|
| fmlint, mdlinter, pycustomlint, json-lint | `src/fiona/customlinter/` |
|
|
| ruff wrapper | `src/fiona/python_ruff/python_ruff.py` |
|
|
| wiki init | `src/fiona/wiki/wiki.py` |
|
|
| wiki gitignore | `src/fiona/wiki/ignore.py` |
|
|
| wiki update | `src/fiona/wiki/update_manager.py` |
|
|
| wiki sanity | `src/fiona/wiki/sanity_manager.py` |
|
|
| housekeeping | `src/fiona/wiki/housekeeping.py` |
|
|
| wiki graph | `src/fiona/wiki/graph_generator.py`, `src/fiona/wiki/templates/graph_template.html`, `src/fiona/wiki/templates/graph_physics.js` |
|
|
| wiki graph `--serve` (planned) | `src/fiona/wiki/graph_server.py`, `src/fiona/wiki/templates/graph_live.js` |
|
|
| git | `src/fiona/gittool/git_tool.py` |
|
|
| sanity engine | `src/fiona/sanity_runner/sanity_runner.py` |
|
|
| runners | `src/fiona/run_tests.py`, `src/fiona/run_sanity.py` |
|
|
| launchers | `scripts/run_tests.py`, `scripts/run_sanity.py` |
|
|
| bootstrap | `bootstrap`, `bootstrap.ps1`, `scripts/bootstrap_install.py` |
|
|
|
|
Layout for section 7.1.5, under `src/fiona/wiki/ingest/`:
|
|
|
|
| Module | Classes | Role |
|
|
| --- | --- | --- |
|
|
| `models.py` | `FetchedSource`, `ConversionResult`, `IngestResult` | Dataclasses passed between stages. `IngestResult.to_dict()` is the JSON in section 4.7.6. |
|
|
| `fetchers.py` | `SourceFetcher` (ABC), `LocalFileFetcher`, `UrlFetcher` | `accepts(argument)` and `fetch(argument, dest_dir)`. `UrlFetcher` takes the opener. |
|
|
| `media.py` | `detect_media_type`, `make_slug` | *7.1.5.2.4.* and *7.1.5.2.6.* |
|
|
| `converters/base.py` | `SourceConverter` (ABC), `ConverterRegistry`, `ConversionError` | Class attributes `name`, `media_types`, `extensions`, `raw_category`; `convert(source)`. `ConverterRegistry.find(media_type, extension)`. |
|
|
| `converters/markdown.py`, `text.py`, `html.py`, `pdf.py` | `MarkdownConverter`, `TextConverter`, `HtmlConverter`, `PdfConverter` | One format each. |
|
|
| `archiver.py` | `RawArchiver` | *7.1.5.3.*; `find_by_sha256(digest, size)` scans `raw/` for a reusable copy. |
|
|
| `page_writer.py` | `RawPageWriter` | *7.1.5.5.* |
|
|
| `log_writer.py` | `IngestLog` | *7.1.5.6.* |
|
|
| `pipeline.py` | `IngestPipeline`, `default_pipeline()` | Runs the stages; `list_pending(wiki_root)` is *7.1.5.9.* |
|
|
|
|
Known defects to avoid copying blindly:
|
|
|
|
- `json-lint fix` walks the file list twice. The observable contract
|
|
is one count and one path line per fixed file.
|
|
- The previous spec's graph physics (`mass = 1 + in_degree * 0.5`,
|
|
center `0.1`, `alphaMin` `0.1`, `velocityDecay` `0.6`, and
|
|
"D3 or vis.js") does not match the template. Section 7.1.4 is the
|
|
contract.
|
|
- The previous spec's external-link JSON used `{url, count}` objects.
|
|
The implementation emits unique URL strings plus a total count.
|
|
- pymarkdown rule typos fail silently (section 5.1).
|
|
- `mdoctor fix` does not skip read-only zones (section 4.2).
|