From e9542ccf882c6bc2f8620c3eb2796a4528c08607 Mon Sep 17 00:00:00 2001 From: nvaldez Date: Tue, 6 Oct 2026 11:27:26 +0200 Subject: [PATCH] update fiona skill --- AGENTS.md | 6 +- SPEC.md | 14 +- .../data/llm_wiki/utils/skills/fiona/SKILL.md | 1185 +++++++++-------- .../utils/skills/fiona/llm-wiki-mod.md | 8 +- src/fiona/data/llm_wiki/wiki/schema.md | 8 +- 5 files changed, 676 insertions(+), 545 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 31a0af9..6078db7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,10 +1,10 @@ --- name: Fiona-Assistant Development Agent Rules -version: 1.9.2 +version: 1.9.3 title: Agents type: summary created: 2025-09-23 -updated: 2026-10-01 +updated: 2026-10-06 tags: [agents, rules, guidelines, llm, ai] sources: [] --- @@ -400,7 +400,7 @@ Before step 2 of that lifecycle: **All markdown files in llm-wiki/ MUST have YAML frontmatter** with the required `name` and `version` fields. Every page under `wiki/` also requires the quality signals `confidence`, `llm`, `contested`, and -`contradictions`. How to set them is section 13 of the +`contradictions`. How to set them is section 15 of the [Fiona skill](./src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md). All other fields are optional. fmlint enforces `name` and `version` only. diff --git a/SPEC.md b/SPEC.md index 290c07c..f43e755 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,9 +1,9 @@ --- name: Fiona-Assistant Specification -version: 1.9.0 +version: 1.9.1 title: Project Spec Sheet created: 2026-09-23 -updated: 2026-10-05 +updated: 2026-10-06 type: summary --- @@ -1531,7 +1531,7 @@ zones. `false` otherwise. `contradictions` lists the `name` of each page that disagrees, and is `[]` when none do. fmlint enforces `name` and `version` only. - - *acceptance*: Fiona skill section 13 marks those four fields + - *acceptance*: Fiona skill section 15 marks those four fields required on wiki pages and states the choice rules above. ### 6.3. MDL Codes @@ -3002,7 +3002,7 @@ Candidates for later formats are in Appendix C. signals are set and `llm.model` is not `none`, the page is listed in `wiki/index.md`, and `wiki/log.md` has a line for the refinement. - - *acceptance*: Fiona skill section 16 lists those conditions and + - *acceptance*: Fiona skill section 18 lists those conditions and the steps that meet them, and says when to convert a source by hand. - *7.1.5.11.* The agent checks the frontmatter the @@ -3013,7 +3013,7 @@ Candidates for later formats are in Appendix C. `ingest.fetched`, `ingest.media_type`, or `ingest.tool`. When one of those is wrong, it re-runs the ingest with `--force` or reports the fault to the user. - - *acceptance*: Fiona skill section 16 has a metadata check step + - *acceptance*: Fiona skill section 18 has a metadata check step that names the fields it may edit and the fields it may not. - *7.1.5.12.* Archived sources do not fail `wiki sanity`. Every check skips `raw/`. @@ -3026,7 +3026,7 @@ Candidates for later formats are in Appendix C. `json.loads` and has `conversion: none`. - *7.1.5.14.* `NOT_IMPLEMENTED` On exit 4 the agent gets the source another way, or asks the user for it. The Fiona skill teaches - this in section 16.2. The steps, in order: + this in section 18.3. The steps, in order: - *7.1.5.14.1.* Use a tool the agent already has that can read that host with the user's access, such as an MCP server for Confluence, SharePoint, Jenkins, or a ticket system, or an @@ -3047,7 +3047,7 @@ Candidates for later formats are in Appendix C. source is not the source. Archive the content the tool returned as it is, and say in `ingest.warnings` that it came through that tool. - - *acceptance*: Fiona skill section 4.6 lists exit 4. Section 16.2 + - *acceptance*: Fiona skill section 6.7 lists exit 4. Section 18.3 has the three cases above (a tool with access, ask the user, never ask for credentials) and shows the `--origin` command. diff --git a/src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md b/src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md index c5cdf11..805d0f4 100644 --- a/src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md +++ b/src/fiona/data/llm_wiki/utils/skills/fiona/SKILL.md @@ -3,8 +3,8 @@ name: fiona category: productivity description: Operate an llm-wiki with the fiona-assistant CLI. author: nvaldez -version: 1.15.0 -updated: 2026-10-02 +version: 2.0.0 +updated: 2026-10-06 --- # Fiona @@ -16,172 +16,201 @@ You maintain an llm-wiki. The command for that work is [schema](../../../wiki/schema.md). Markdown rule names are in [markdown rules](./markdown_rules.md). -- [Load llm-wiki-mod](#load-llm-wiki-mod) -- [You own the wiki](#you-own-the-wiki) -- [1. Before a command](#1-before-a-command) -- [2. Install](#2-install) -- [3. Which command](#3-which-command) -- [4. wiki](#4-wiki) -- [5. librarian](#5-librarian) -- [6. mdoctor](#6-mdoctor) -- [7. fmlint](#7-fmlint) -- [8. python-ruff](#8-python-ruff) -- [9. json-lint](#9-json-lint) -- [10. git](#10-git) -- [11. How to read a run](#11-how-to-read-a-run) -- [12. Work loop](#12-work-loop) -- [13. Frontmatter](#13-frontmatter) -- [14. Checks with no own command](#14-checks-with-no-own-command) -- [15. Do not call](#15-do-not-call) -- [16. Ingest cycle](#16-ingest-cycle) -- [17. Housekeeping](#17-housekeeping) +Every paragraph and list item below has its own number. Cite a rule +by that number, for example `section 2.1.3`. -## Load llm-wiki-mod +- [1. Load llm-wiki-mod](#1-load-llm-wiki-mod) +- [2. You own the wiki](#2-you-own-the-wiki) +- [3. Before a command](#3-before-a-command) +- [4. Install](#4-install) +- [5. Which command](#5-which-command) +- [6. wiki](#6-wiki) +- [7. librarian](#7-librarian) +- [8. mdoctor](#8-mdoctor) +- [9. fmlint](#9-fmlint) +- [10. python-ruff](#10-python-ruff) +- [11. json-lint](#11-json-lint) +- [12. git](#12-git) +- [13. How to read a run](#13-how-to-read-a-run) +- [14. Work loop](#14-work-loop) +- [15. Frontmatter](#15-frontmatter) +- [16. Checks with no own command](#16-checks-with-no-own-command) +- [17. Do not call](#17-do-not-call) +- [18. Ingest cycle](#18-ingest-cycle) +- [19. Housekeeping](#19-housekeeping) -[llm-wiki-mod](./llm-wiki-mod.md) is a core part of wiki management: -raw sources, the wiki, the schema, and ingest, query, and lint. -If it is not already in context, read it before any wiki work. -When it is already loaded, do not read it again. +## 1. Load llm-wiki-mod -## You own the wiki +**1.1** [llm-wiki-mod](./llm-wiki-mod.md) is a core part of wiki +management: raw sources, the wiki, the schema, and ingest, query, +and lint. -You are responsible for this wiki: what it says, how it is +**1.2** If it is not already in context, read it before any wiki +work. When it is already loaded, do not read it again. + +## 2. You own the wiki + +**2.1** You are responsible for this wiki: what it says, how it is organized, how pages link, and its history. The `fiona-assistant` tools own nothing. They do the repeated, mechanical work cheaply, so your tokens go to reading sources and making judgments. A tool is a helper you direct, not an authority you defer to. -- A clean tool run is evidence, not a verdict. `wiki sanity` - passing means the style, frontmatter, and links are valid. It - does not mean a page is true, current, complete, or in the right - place. Read the pages. -- Treat tool output as a draft. When a tool gets something wrong, - such as a bad title, a lost table, or the wrong folder, fix the - result within the rules (section 16.3), or re-run it with better - arguments. -- Decisions stay with you: what a page says, what it links to, - which claims conflict, how confident the page is, and when a - source is too complex for a converter (section 16.2). -- When no tool fits, do the work yourself. Do not bend a tool to a - job it was not built for, and do not invent flags (section 15). -- When a tool fails or misbehaves, tell the user the command and - its output. Do not work around it silently. +- **2.1.1** A clean tool run is evidence, not a verdict. + `wiki sanity` passing means the style, frontmatter, and links are + valid. It does not mean a page is true, current, complete, or in + the right place. Read the pages. +- **2.1.2** Treat tool output as a draft. When a tool gets something + wrong, such as a bad title, a lost table, or the wrong folder, fix + the result within the rules (section 18.4), or re-run it with + better arguments. +- **2.1.3** Decisions stay with you: what a page says, what it links + to, which claims conflict, how confident the page is, and when a + source is too complex for a converter (section 18.3). +- **2.1.4** When no tool fits, do the work yourself. Do not bend a + tool to a job it was not built for, and do not invent flags + (section 17). +- **2.1.5** When a tool fails or misbehaves, tell the user the + command and its output. Do not work around it silently. -### Write reusable scripts +### 2.2. Write reusable scripts -When you repeat the same mechanical step and no `fiona-assistant` -command covers it, write a script under +**2.2.1** When you repeat the same mechanical step and no +`fiona-assistant` command covers it, write a script under [utils/scripts](../../scripts/map.md) instead of redoing it by hand each time. Tokens spent once on a script are saved on every later run, and the next session can reuse it. -1. Read the [script map](../../scripts/map.md) first. Extend an - existing script before writing a new one. -2. One job per script. Name the file in `snake_case.py`. -3. Use the standard library first. A third-party import must already - be installed with `fiona-assistant`, or be approved by the user. -4. Start with a module docstring: purpose, inputs, outputs, and one - example call. Parse arguments with `argparse`, so `--help` works. -5. Take the wiki root as an argument, defaulting to `.`. Print - results on stdout, with `--json` when you will parse them. Send - diagnostics through `logging` to stderr. -6. Keep it deterministic. No network access unless that is the - script's job and the user agreed. Never write under `raw/` or - `personal/`. A script that writes files offers `--dry-run`. -7. Add a row to the [script map](../../scripts/map.md). Run - `fiona-assistant python-ruff chklint utils/scripts`. Append a - line to `wiki/log.md`. -8. When a script proves useful in general, suggest to the user that - it become a `fiona-assistant` command. +**2.2.2** How to write one: -### Save after every ingest cycle and housekeeping pass +- **2.2.2.1** Read the [script map](../../scripts/map.md) first. + Extend an existing script before writing a new one. +- **2.2.2.2** One job per script. Name the file in `snake_case.py`. +- **2.2.2.3** Use the standard library first. A third-party import + must already be installed with `fiona-assistant`, or be approved + by the user. +- **2.2.2.4** Start with a module docstring: purpose, inputs, + outputs, and one example call. Parse arguments with `argparse`, + so `--help` works. +- **2.2.2.5** Take the wiki root as an argument, defaulting to `.`. + Print results on stdout, with `--json` when you will parse them. + Send diagnostics through `logging` to stderr. +- **2.2.2.6** Keep it deterministic. No network access unless that + is the script's job and the user agreed. Never write under `raw/` + or `personal/`. A script that writes files offers `--dry-run`. +- **2.2.2.7** Add a row to the [script map](../../scripts/map.md). + Run `fiona-assistant python-ruff chklint utils/scripts`. Append a + line to `wiki/log.md`. +- **2.2.2.8** When a script proves useful in general, suggest to the + user that it become a `fiona-assistant` command. -When an ingest cycle ends (section 16.4, with sanity passing), run: +### 2.3. Save after every ingest cycle and housekeeping pass + +**2.3.1** When an ingest cycle ends (section 18.5, with sanity +passing), run: ```text fiona-assistant git save --description "ingest : " ``` -After a housekeeping pass (lint fixes, index, log, or TOC updates, -page moves, template updates), with sanity passing, read `git diff` -for the files `fiona-assistant git status` lists. +**2.3.2** After a housekeeping pass (lint fixes, index, log, or TOC +updates, page moves, template updates), with sanity passing, read +`git diff` for the files `fiona-assistant git status` lists. -- **Metadata.** On each added or modified page, frontmatter matches - section 13. `name`, `version`, and `updated` are set. A page under - `wiki/` also has `confidence`, `llm`, `contested`, and - `contradictions`. -- **Integrity.** The body still holds the claims, links, tables, and - index or log entries from before this pass. A lint fix, a move, or - a template merge did not drop or garble them. +- **2.3.2.1** **Metadata.** On each added or modified page, + frontmatter matches section 15. `name`, `version`, and `updated` + are set. A page under `wiki/` also has `confidence`, `llm`, + `contested`, and `contradictions`. +- **2.3.2.2** **Integrity.** The body still holds the claims, links, + tables, and index or log entries from before this pass. A lint + fix, a move, or a template merge did not drop or garble them. -When a check fails, fix the page and run sanity again. When you -cannot tell whether a page is intact, ask before saving. Then run: +**2.3.3** When a check fails, fix the page and run sanity again. +When you cannot tell whether a page is intact, ask before saving. +Then run: ```text fiona-assistant git save --description "housekeeping: " ``` -This is a standing instruction. You do not need to ask each time. -It is the step that makes the wiki's history usable: +**2.3.4** This is a standing instruction. You do not need to ask +each time. It is the step that makes the wiki's history usable: -- **One source, one commit.** The raw copy, the page, the index - entry, the log line, and every concept or entity page the source - changed land together. `git log` becomes the record of what - entered the wiki, when, and from where. -- **Reviewable.** The user can run `git show` on that commit and see - exactly how one source changed the wiki, and check your synthesis - against it. -- **Recoverable.** A bad refinement can be undone or restored - without losing other work. That only holds when each cycle is its - own commit. Batched or skipped saves mix sources, so one undo - drops unrelated work. -- **Proof behind the log.** `wiki/log.md` says what happened. The - commit holds the before and after state that proves it. +- **2.3.4.1** **One source, one commit.** The raw copy, the page, + the index entry, the log line, and every concept or entity page + the source changed land together. `git log` becomes the record of + what entered the wiki, when, and from where. +- **2.3.4.2** **Reviewable.** The user can run `git show` on that + commit and see exactly how one source changed the wiki, and check + your synthesis against it. +- **2.3.4.3** **Recoverable.** A bad refinement can be undone or + restored without losing other work. That only holds when each + cycle is its own commit. Batched or skipped saves mix sources, so + one undo drops unrelated work. +- **2.3.4.4** **Proof behind the log.** `wiki/log.md` says what + happened. The commit holds the before and after state that proves + it. -Before either save, run `fiona-assistant git status`. `save` stages -everything in the tree. When it shows changes that are not from -this cycle, ask the user before saving. A housekeeping pass also -reads `git diff` and applies the metadata and integrity checks -above. When sanity fails, fix it first, or ask. A duplicate ingest -writes nothing, so `save` refusing a clean tree is expected. This -standing instruction covers `save` only. `snapshot`, `restore`, -`clean`, and `undo` still need the user's request. +**2.3.5** Before either save, run `fiona-assistant git status`. +`save` stages everything in the tree. When it shows changes that are +not from this cycle, ask the user before saving. -## 1. Before a command +**2.3.6** A housekeeping pass also reads `git diff` and applies the +metadata and integrity checks in section 2.3.2. -Run from the wiki root, the directory that contains +**2.3.7** When sanity fails, fix it first, or ask. + +**2.3.8** A duplicate ingest writes nothing, so `save` refusing a +clean tree is expected. + +**2.3.9** This standing instruction covers `save` only. `snapshot`, +`restore`, `clean`, and `undo` still need the user's request. + +## 3. Before a command + +**3.1** Run from the wiki root, the directory that contains `fiona-settings.json`, or pass that directory as the argument. -Optional directories default to `.`. The CLI does not search -parent directories for the wiki root. `librarian toc file *` -looks for `fiona-settings.json` in the current directory, then -in subdirectories. -`personal/` is read-only, including the directory itself. Never -create, edit, or delete there. `raw/` is the archive: you may add a -new source file (section 16.2), but never edit, move, or delete a -file already there. Never point a fixer at either. +**3.2** Optional directories default to `.`. The CLI does not search +parent directories for the wiki root. -`librarian` and `mdoctor` also exist as their own console scripts. -Call them through `fiona-assistant` so one binary covers every -tool below. +**3.3** `librarian toc file *` looks for `fiona-settings.json` in +the current directory, then in subdirectories. -Human reports go to stdout. Diagnostics go to stderr. Set -`LOG_LEVEL=DEBUG` when a run needs a trace. Do not treat stderr -warnings as the lint report. +**3.4** `personal/` is read-only, including the directory itself. +Never create, edit, or delete there. -## 2. Install +**3.5** `raw/` is the archive: you may add a new source file +(section 18.3), but never edit, move, or delete a file already +there. Never point a fixer at either `personal/` or `raw/`. + +**3.6** `librarian` and `mdoctor` also exist as their own console +scripts. Call them through `fiona-assistant` so one binary covers +every tool below. + +**3.7** Human reports go to stdout. Diagnostics go to stderr. Do not +treat stderr warnings as the lint report. + +**3.8** Set `LOG_LEVEL=DEBUG` when a run needs a trace. + +## 4. Install + +**4.1** Install the package: ```text python -m pip install fiona-assistant ``` -`fiona-assistant` with no command prints help and exits 0. +**4.2** `fiona-assistant` with no command prints help and exits 0. -## 3. Which command +## 5. Which command + +**5.1** Find the question, then run its command. Prefix every row +with `fiona-assistant`. | Question | Command | | --- | --- | -| Maintain the wiki: what needs work? | `housekeeping --json`, then section 17 | +| Maintain the wiki: what needs work? | `housekeeping --json`, then section 19 | | Is this wiki healthy? | `wiki sanity --json --non-interactive` | | Markdown style on files I edited? | `mdoctor chklint ` | | Auto-fix markdown style? | `mdoctor fix ` | @@ -190,53 +219,57 @@ python -m pip install fiona-assistant | Python under the wiki? | `python-ruff chklint ` | | New files or new headings? | `librarian toc` and `librarian toc file` | | What links to what? | `wiki graph` | -| New source, a URL or a file? | `wiki ingest --json`, then section 16 | +| New source, a URL or a file? | `wiki ingest --json`, then section 18 | | Ingested pages not refined yet? | `wiki ingest --pending --json` | | Empty directory, start a wiki? | `wiki init ` | | Template behind the installed package? | `wiki check-update`, then `wiki update --dry-run` | -| A `*-update` file appeared? | Merge it, section 17.1 | +| A `*-update` file appeared? | Merge it, section 19.8 | | Ingest or housekeeping done? | `git save --description TEXT` | -Prefix every row with `fiona-assistant`. +## 6. wiki -## 4. wiki +**6.1** `fiona-assistant wiki` with no subcommand prints help and +exits 1. -`fiona-assistant wiki` with no subcommand prints help and exits 1. - -### 4.1. init +### 6.2. init ```text fiona-assistant wiki init [directory] ``` -Copies the template, substitutes `DATE_PLACEHOLDER` with today's -UTC date, creates a git repo, and commits `Initial llm-wiki commit`. -`.gitignore` starts from the packaged default ignore patterns and -includes `personal/`. An existing `.gitignore` keeps its lines; -missing patterns are appended. Paths matched by those default -patterns are not copied. `personal/` is gitignored. +**6.2.1** Copies the template, substitutes `DATE_PLACEHOLDER` with +today's UTC date, creates a git repo, and commits +`Initial llm-wiki commit`. -Success stdout: +**6.2.2** `.gitignore` starts from the packaged default ignore +patterns and includes `personal/`. An existing `.gitignore` keeps +its lines; missing patterns are appended. + +**6.2.3** Paths matched by those default patterns are not copied. +`personal/` is gitignored. + +**6.2.4** Success stdout: ```text Successfully initialized llm-wiki directory at: ``` -Refuses, and writes nothing, when any of these already exist: -`AGENTS.md`, `fiona-settings.json`, `wiki/`, `ingest/`, `README.md`. +**6.2.5** Refuses, and writes nothing, when any of these already +exist: `AGENTS.md`, `fiona-settings.json`, `wiki/`, `ingest/`, +`README.md`. -### 4.2. check-update +### 6.3. check-update ```text fiona-assistant wiki check-update [directory] [--json] ``` -The directory must contain `fiona-settings.json`. Exit 0 whether -or not updates exist. +**6.3.1** The directory must contain `fiona-settings.json`. Exit 0 +whether or not updates exist. -Human stdout when current: `Wiki is up to date.` +**6.3.2** Human stdout when current: `Wiki is up to date.` -When something differs, the lines that apply: +**6.3.3** When something differs, the lines that apply: ```text Files to update: @@ -248,139 +281,159 @@ Total: file(s) need attention Files skipped (locally modified): ``` -A file that lacks a version on either side is compared by SHA-256. -Text hashes fold `\r\n` and lone `\r` to `\n` first, so a file that -differs from the template only in line endings is up to date. Equal -hashes mean up to date. Different hashes print the first 8 hex -digits of each side instead of versions: +**6.3.4** A file that lacks a version on either side is compared by +SHA-256. Text hashes fold `\r\n` and lone `\r` to `\n` first, so a +file that differs from the template only in line endings is up to +date. + +**6.3.5** Equal hashes mean up to date. Different hashes print the +first 8 hex digits of each side instead of versions: ```text - utils/scripts/example.py (sha256 1a2b3c4d -> 9f8e7d6c) ``` -`--json` prints one object with `files_to_update`, `conflicts`, -`files_to_add`, `directories_to_add`, `files_skipped`, `total`, and -`has_updates`. Each conflict has `path`, `update_path`, +**6.3.6** `--json` prints one object with `files_to_update`, +`conflicts`, `files_to_add`, `directories_to_add`, `files_skipped`, +`total`, and `has_updates`. + +**6.3.7** Each conflict has `path`, `update_path`, `current_version`, and `template_version`. Entries in `files_to_update` and `conflicts` also carry `current_sha256` and `template_sha256`: the full hash, or `null` when versions were compared. -### 4.3. update +### 6.4. update ```text fiona-assistant wiki update [directory] [--dry-run] [-y|--yes] [-f|--force] [--json] ``` -Updates drifted non-markdown core files in place and adds missing -core files. A differing `fiona-settings.json` is a conflict: the -owner's file stays byte-for-byte, and the template is written -beside it as `fiona-settings-update.json`. You merge it during -housekeeping (section 17.1). Never deletes user files. Never writes -inside `personal/` or `raw/`. Never adds or overwrites anything -under `wiki/` (seeded at init, then filled by the LLM). Never -writes a path matched by `.gitignore`. +**6.4.1** Updates drifted non-markdown core files in place and adds +missing core files. -A confirmed update that had something to report writes -`corefiles.json` at the wiki root. It lists each compared -core file's version, template version, owner hash, template hash, -and the UTC date. A later check does not list a file while those -four values still match, so a second update the same day stays -quiet about a file you already updated. `check-update` and -`--dry-run` do not write that file. Only `wiki update` writes -`corefiles.json`. Do not edit it by hand, and do not treat it as -a file to merge. A path in `fiona.wiki.update.ignore_files` is -not an update, a conflict, an addition, or a merge candidate. -The shipped list is `corefiles.json`. +**6.4.2** A differing `fiona-settings.json` is a conflict: the +owner's file stays byte-for-byte, and the template is written beside +it as `fiona-settings-update.json`. You merge it during housekeeping +(section 19.8). -Never overwrites an existing markdown file. When the template has a -newer version of one, such as `AGENTS.md`, `wiki/index.md`, -`wiki/log.md`, or `wiki/schema.md`, the owner's file stays as it is -and the template is written beside it as `-update.md` -(`AGENTS-update.md`, `wiki/schema-update.md`). You merge it during -housekeeping (section 17.1). +**6.4.3** Never deletes user files. Never writes inside `personal/` +or `raw/`. Never adds or overwrites anything under `wiki/` (seeded +at init, then filled by the LLM). Never writes a path matched by +`.gitignore`. -Run `--dry-run` first. It prints the plan, writes nothing, and -exits 0. Without `--yes` or `--force` the command prompts. On a -pipe that prompt cannot be answered and the command exits 1. -Pass `--yes` only after the user agrees to the dry run. +**6.4.4** A confirmed update that had something to report writes +`corefiles.json` at the wiki root. It lists each compared core +file's version, template version, owner hash, template hash, and the +UTC date. -Success stdout: +**6.4.5** A later check does not list a file while those four values +still match, so a second update the same day stays quiet about a +file you already updated. + +**6.4.6** `check-update` and `--dry-run` do not write that file. +Only `wiki update` writes `corefiles.json`. Do not edit it by hand, +and do not treat it as a file to merge. + +**6.4.7** A path in `fiona.wiki.update.ignore_files` is not an +update, a conflict, an addition, or a merge candidate. The shipped +list is `corefiles.json`. + +**6.4.8** Never overwrites an existing markdown file. When the +template has a newer version of one, such as `AGENTS.md`, +`wiki/index.md`, `wiki/log.md`, or `wiki/schema.md`, the owner's +file stays as it is and the template is written beside it as +`-update.md` (`AGENTS-update.md`, `wiki/schema-update.md`). +You merge it during housekeeping (section 19.8). + +**6.4.9** Run `--dry-run` first. It prints the plan, writes nothing, +and exits 0. + +**6.4.10** Without `--yes` or `--force` the command prompts. On a +pipe that prompt cannot be answered and the command exits 1. Pass +`--yes` only after the user agrees to the dry run. + +**6.4.11** Success stdout: ```text Update complete: file(s) updated, added, conflict(s), skipped. ``` -When `` is not zero, the next line is: +**6.4.12** When `` is not zero, the next line is: ```text Merge each -update into its file: run fiona-assistant housekeeping. ``` -### 4.4. sanity +### 6.5. sanity ```text fiona-assistant wiki sanity [directory] [--json] [--non-interactive] ``` -This is the gate for a wiki. Exit 0 only when every check is -clean. Exit 1 otherwise. Requires `fiona-settings.json`. +**6.5.1** This is the gate for a wiki. Exit 0 only when every check +is clean. Exit 1 otherwise. Requires `fiona-settings.json`. -Checks, in order: `markdownlint`, `frontmatter_lint`, +**6.5.2** Checks, in order: `markdownlint`, `frontmatter_lint`, `python-ruff`, `mdlinter`, `jsonlint`. -Use `--json` when you will parse the result. The object has -`target`, `checks`, `total_violations`, and `passed`. Each check +**6.5.3** Use `--json` when you will parse the result. The object +has `target`, `checks`, `total_violations`, and `passed`. Each check has `result` (0 pass, 1 fail), `violation_count`, and `violations`. -`--non-interactive` prints progress dots instead of live bars. It -is also automatic when stdout is not a terminal. +**6.5.4** `--non-interactive` prints progress dots instead of live +bars. It is also automatic when stdout is not a terminal. -A passing human report ends with `SANITY PASSED`. A failing report -ends with `Sanity Failed - checks need attention:`. +**6.5.5** A passing human report ends with `SANITY PASSED`. A +failing report ends with `Sanity Failed - checks need attention:`. -### 4.5. graph +### 6.6. graph ```text fiona-assistant wiki graph [directory] [-o FILE] [--format html|json] [--json] ``` -Defaults: `wiki_graph.html` in the current directory, include -`**/*.md`, wikilinks on, external-link nodes off. A path in -`fiona.wiki.graph.ignore_files` is left out of the graph. The -shipped pattern is `*/table_of_contents.md`, which also matches -`table_of_contents.md` at the wiki root. A table of contents -links to every page and would hide the real links. +**6.6.1** Defaults: `wiki_graph.html` in the current directory, +include `**/*.md`, wikilinks on, external-link nodes off. -Also: `--include-pattern`, `--exclude-pattern`, +**6.6.2** A path in `fiona.wiki.graph.ignore_files` is left out of +the graph. The shipped pattern is `*/table_of_contents.md`, which +also matches `table_of_contents.md` at the wiki root. A table of +contents links to every page and would hide the real links. + +**6.6.3** Also: `--include-pattern`, `--exclude-pattern`, `--wikilinks` / `--no-wikilinks`, `--external-links` / `--no-external-links`. `--json` is an alias for `--format json`. -### 4.6. ingest +### 6.7. ingest ```text fiona-assistant wiki ingest [--wiki DIR] [-r|--raw PATH] [-t|--target PATH] [--raw-only] [-f|--force] [--dry-run] [--json] fiona-assistant wiki ingest --pending [--wiki DIR] [--json] ``` -`` is an `http://` or `https://` URL, or a file path. The -tool is the first phase of an ingest: +**6.7.1** `` is an `http://` or `https://` URL, or a file +path. The tool is the first phase of an ingest: -- A URL is downloaded into `ingest/` as whatever it is: HTML, PDF, - text. No credentials are sent. -- The file is archived under `raw/`. A file in `ingest/` is moved. - Any other file is copied. Nothing already in `raw/` is changed. -- Before archiving, its SHA-256 is compared with every file in - `raw/`. Identical bytes are never archived twice: the existing - copy is reused (`raw_reused: true`). When a page already cites - that copy, nothing is written and `duplicate_page` names it. -- The archive is converted to markdown and written as one page - with `status: raw`, `confidence: zero`, and `llm.model: none`. -- One line is appended to `wiki/log.md`. +- **6.7.1.1** A URL is downloaded into `ingest/` as whatever it is: + HTML, PDF, text. No credentials are sent. +- **6.7.1.2** The file is archived under `raw/`. A file in `ingest/` + is moved. Any other file is copied. Nothing already in `raw/` is + changed. +- **6.7.1.3** Before archiving, its SHA-256 is compared with every + file in `raw/`. Identical bytes are never archived twice: the + existing copy is reused (`raw_reused: true`). When a page already + cites that copy, nothing is written and `duplicate_page` names it. +- **6.7.1.4** The archive is converted to markdown and written as + one page with `status: raw`, `confidence: zero`, and + `llm.model: none`. +- **6.7.1.5** One line is appended to `wiki/log.md`. -It does not refine the page, edit `wiki/index.md`, or touch any -other page. That is your phase, section 16. +**6.7.2** It does not refine the page, edit `wiki/index.md`, or +touch any other page. That is your phase, section 18. + +**6.7.3** Flags: | Flag | Use | | --- | --- | @@ -391,11 +444,14 @@ other page. That is your phase, section 16. | `--dry-run` | Print the plan. Writes nothing in the wiki. A URL is still downloaded, to a temporary directory. | | `--pending` | List pages still at `status: raw`. | -Defaults: raw copies go to `raw/articles/` (HTML, text, markdown), -`raw/papers/` (PDF), or `raw/other/` (no converter). Pages go to -`wiki/sources/.md`. Use `-r` to group related sources, and -`-t` when the user names a place or the source belongs beside -existing pages. +**6.7.4** Defaults: raw copies go to `raw/articles/` (HTML, text, +markdown), `raw/papers/` (PDF), or `raw/other/` (no converter). +Pages go to `wiki/sources/.md`. + +**6.7.5** Use `-r` to group related sources, and `-t` when the user +names a place or the source belongs beside existing pages. + +**6.7.6** Converters: | Converter | Handles | Loses | | --- | --- | --- | @@ -404,55 +460,66 @@ existing pages. | `html` | web pages | Navigation, scripts, and anything drawn by JavaScript. | | `pdf` | PDFs with a text layer | Tables, images, columns, and scanned pages. | -Pass `--json` and read the object: `raw_path`, `page_path`, -`sha256`, `raw_reused`, `duplicate_page`, `converter`, +**6.7.7** Pass `--json` and read the object: `raw_path`, +`page_path`, `sha256`, `raw_reused`, `duplicate_page`, `converter`, `conversion` (`complete`, `partial`, `none`), and `warnings`. The page frontmatter carries the same facts in its `ingest` map. +**6.7.8** Exit codes: + | Exit | Meaning | Next | | --- | --- | --- | -| 0 | Page written, `--raw-only`, or a duplicate | Read `duplicate_page`, `conversion`, and `warnings`, then section 16.1. | -| 1 | Error on stderr, nothing written | Fix the argument, or fetch the source another way (section 16.2). | -| 3 | Raw copy archived, no page | Convert by hand (section 16.2). | +| 0 | Page written, `--raw-only`, or a duplicate | Read `duplicate_page`, `conversion`, and `warnings`, then section 18.2. | +| 1 | Error on stderr, nothing written | Fix the argument, or fetch the source another way (section 18.3). | +| 3 | Raw copy archived, no page | Convert by hand (section 18.3). | -## 5. librarian +## 7. librarian -`fiona-assistant librarian` with no subcommand lists `.`. +**7.1** `fiona-assistant librarian` with no subcommand lists `.`. -### 5.1. list +### 7.2. list ```text fiona-assistant librarian list [directory] ``` +**7.2.1** Stdout: + ```text Found files in '': - ``` -### 5.2. toc +### 7.3. toc ```text fiona-assistant librarian toc [directory] [--recursive-toc] [--non-recursive] [--exclude REGEX] ``` -Writes `table_of_contents.md` in the target folder. Depth comes -from `fiona-settings.json`. `--exclude` is repeatable and takes a -regex. A folder with no markdown prints `no markdown files found` +**7.3.1** Writes `table_of_contents.md` in the target folder. Depth +comes from `fiona-settings.json`. + +**7.3.2** `--exclude` is repeatable and takes a regex. + +**7.3.3** A folder with no markdown prints `no markdown files found` and writes nothing. +**7.3.4** In-file TOCs: + ```text fiona-assistant librarian toc file fiona-assistant librarian toc file * ``` -`toc file ` writes an in-file TOC under the H1. -`toc file *` does that for every markdown file under the wiki -root. It skips `personal/`, `raw/`, and `table_of_contents.md`. -An explicit path inside `personal/` or `raw/` exits 1 and writes -nothing. +**7.3.5** `toc file ` writes an in-file TOC under the H1. -Status lines: +**7.3.6** `toc file *` does that for every markdown file under the +wiki root. It skips `personal/`, `raw/`, and `table_of_contents.md`. + +**7.3.7** An explicit path inside `personal/` or `raw/` exits 1 and +writes nothing. + +**7.3.8** Status lines: ```text wrote toc in @@ -461,13 +528,15 @@ skipped () generated toc for file(s), skipped ``` -The summary line is bulk mode only. A file with no H1 is +**7.3.9** The summary line is bulk mode only. A file with no H1 is `skipped (no H1 heading)` and is not rewritten. -`--sectionized` forces heading numbers for that run. -## 6. mdoctor +**7.3.10** `--sectionized` forces heading numbers for that run. -Markdown style (PyMarkdown). No subcommand means `chklint .`. +## 8. mdoctor + +**8.1** Markdown style (PyMarkdown). No subcommand means +`chklint .`. ```text fiona-assistant mdoctor chklint [directory] @@ -475,36 +544,36 @@ fiona-assistant mdoctor list [directory] fiona-assistant mdoctor fix [directory] ``` -`chklint` prints the shared report with header -`PyMarkdownlint Violations` and exits 0 even when violations -exist. Read `SUMMARY` and each `FILE` block. A clean report says +**8.2** `chklint` prints the shared report with header +`PyMarkdownlint Violations` and exits 0 even when violations exist. +Read `SUMMARY` and each `FILE` block. A clean report says `- No violations found.` -`fix` rewrites files in place. It does not skip `personal/` or -`raw/`. Pass a writable directory, never the wiki root, when those -trees are present. +**8.3** `fix` rewrites files in place. It does not skip `personal/` +or `raw/`. Pass a writable directory, never the wiki root, when +those trees are present. ```text PyMarkdownFix: file(s) fixed - ``` -`list` uses the librarian listing with the prefix +**8.4** `list` uses the librarian listing with the prefix `MDoctor: Found files in '':`. -## 7. fmlint +## 9. fmlint -Frontmatter. No subcommand means `chklint .`. +**9.1** Frontmatter. No subcommand means `chklint .`. ```text fiona-assistant fmlint chklint [directory] ``` -Header: `Frontmatter Lint Violations`. Exits 0 even when -violations exist. Skips `personal/` and `raw/`. A missing -directory exits 1. +**9.2** Header: `Frontmatter Lint Violations`. Exits 0 even when +violations exist. Skips `personal/` and `raw/`. A missing directory +exits 1. -Act on these codes: +**9.3** Act on these codes: | Code | Fix | | --- | --- | @@ -516,21 +585,21 @@ Act on these codes: | FML006 | On `SKILL.md`, `name` must equal the parent folder. | | FML007 | Add `version`. | -## 8. python-ruff +## 10. python-ruff -Python style. No subcommand means `chklint .`. +**10.1** Python style. No subcommand means `chklint .`. ```text fiona-assistant python-ruff chklint [directory] ``` -Header: `Ruff Violations`. Exit 0 when the ruff process itself -succeeds, including when violations are listed. There is no `fix` -subcommand. +**10.2** Header: `Ruff Violations`. Exit 0 when the ruff process +itself succeeds, including when violations are listed. There is no +`fix` subcommand. -## 9. json-lint +## 11. json-lint -JSON syntax and style, including `fiona-settings.json`. +**11.1** JSON syntax and style, including `fiona-settings.json`. No subcommand means `chklint .`. ```text @@ -539,30 +608,22 @@ fiona-assistant json-lint chklint --fix fiona-assistant json-lint fix [directory] ``` -Header: `JSONLint Violations`. Unlike mdoctor, fmlint, and +**11.2** Header: `JSONLint Violations`. Unlike mdoctor, fmlint, and python-ruff, `chklint` exits 1 when the report contains violations and exits 0 when it does not. -`fix` and `chklint --fix` rewrite indentation and trailing commas. -They skip `personal/` and `raw/`. +**11.3** `fix` and `chklint --fix` rewrite indentation and trailing +commas. They skip `personal/` and `raw/`. ```text JSONLintFix: file(s) fixed - ``` -Codes: JSN001 syntax, JSN002 indent, JSN003 trailing comma, +**11.4** Codes: JSN001 syntax, JSN002 indent, JSN003 trailing comma, JSN004 duplicate keys, JSN005 key order, JSN006 line length. -## 10. git - -These commands act on the git repository that contains the current -directory. Run them from the wiki root. `save` runs after each -ingest cycle and housekeeping pass, without asking -([save rule](#save-after-every-ingest-cycle-and-housekeeping-pass)). -`snapshot`, `restore`, `clean`, and `undo` run only on the user's -request. `fiona-assistant git` with no subcommand prints help and -exits 1. +## 12. git ```text fiona-assistant git status @@ -573,10 +634,25 @@ fiona-assistant git clean fiona-assistant git undo ``` -`--description` / `-d` is required for `save` and `snapshot`. -The tool writes the commit message as +**12.1** These commands act on the git repository that contains the +current directory. Run them from the wiki root. + +**12.2** `save` runs after each ingest cycle and housekeeping pass, +without asking +([section 2.3](#23-save-after-every-ingest-cycle-and-housekeeping-pass)). + +**12.3** `snapshot`, `restore`, `clean`, and `undo` run only on the +user's request. + +**12.4** `fiona-assistant git` with no subcommand prints help and +exits 1. + +**12.5** `--description` / `-d` is required for `save` and +`snapshot`. The tool writes the commit message as ` `. Timestamps are UTC. +**12.6** Results: + | Command | Success stdout | Refuses when | | --- | --- | --- | | `save -d TEXT` | `Committed: ` | Tree is already clean | @@ -586,20 +662,24 @@ The tool writes the commit message as | `undo` | `Undo to: ` | Tree is dirty, or only one commit exists | | `status` | `Status: clean` or `Status: dirty`, then `git status` | — | -On failure the tool prints `Error: ` and exits 1. +**12.7** On failure the tool prints `Error: ` and exits 1. -`snapshot` commits, then creates an annotated tag -`_`. `restore` accepts the full tag -or the suffix and adds a new commit. History before that commit -stays. `clean` is `git reset --hard HEAD` and `git clean -fd`. +**12.8** `snapshot` commits, then creates an annotated tag +`_`. + +**12.9** `restore` accepts the full tag or the suffix and adds a new +commit. History before that commit stays. + +**12.10** `clean` is `git reset --hard HEAD` and `git clean -fd`. `undo` is `git reset --hard HEAD~1`. -Do not use `clean`, `undo`, or `restore` to discard work the user -has not agreed to drop. +**12.11** Do not use `clean`, `undo`, or `restore` to discard work +the user has not agreed to drop. -## 11. How to read a run +## 13. How to read a run -Shared report shape for mdoctor, fmlint, python-ruff, and json-lint: +**13.1** Shared report shape for mdoctor, fmlint, python-ruff, and +json-lint: ```text Violations @@ -612,55 +692,60 @@ TARGET: - Total violations: ``` -Trust the report, not the exit code, for mdoctor, fmlint, and -python-ruff. Trust the exit code for json-lint and for +**13.2** Trust the report, not the exit code, for mdoctor, fmlint, +and python-ruff. Trust the exit code for json-lint and for `wiki sanity`. -An invalid directory prints +**13.3** An invalid directory prints `Error: '' is not a valid directory.` on stderr and exits 1, unless a command above states a different message. -## 12. Work loop +## 14. Work loop -After you edit a page: +**14.1** After you edit a page: -1. Bump frontmatter `version` and set `updated` to the edit date. - On a page under `wiki/`, set the quality signals in section 13. - Patch for a clarification, minor for a new section, major for a - breaking change. -2. A new page is listed in `wiki/index.md`. -3. Append one dated line to `wiki/log.md`, and bump that file too. -4. `fiona-assistant fmlint chklint ` -5. `fiona-assistant mdoctor chklint ` -6. If headings changed, `fiona-assistant librarian toc file `. - If files were added or removed, - `fiona-assistant librarian toc `. -7. If you touched JSON, `fiona-assistant json-lint chklint `. -8. If you touched Python, `fiona-assistant python-ruff chklint `. -9. Before you call the wiki healthy, - `fiona-assistant wiki sanity . --json --non-interactive`. -10. When sanity passes, save - ([save rule](#save-after-every-ingest-cycle-and-housekeeping-pass)). +- **14.1.1** Bump frontmatter `version` and set `updated` to the + edit date. On a page under `wiki/`, set the quality signals in + section 15. Patch for a clarification, minor for a new section, + major for a breaking change. +- **14.1.2** A new page is listed in `wiki/index.md`. +- **14.1.3** Append one dated line to `wiki/log.md`, and bump that + file too. +- **14.1.4** `fiona-assistant fmlint chklint ` +- **14.1.5** `fiona-assistant mdoctor chklint ` +- **14.1.6** If headings changed, + `fiona-assistant librarian toc file `. If files were added + or removed, `fiona-assistant librarian toc `. +- **14.1.7** If you touched JSON, + `fiona-assistant json-lint chklint `. +- **14.1.8** If you touched Python, + `fiona-assistant python-ruff chklint `. +- **14.1.9** Before you call the wiki healthy, + `fiona-assistant wiki sanity . --json --non-interactive`. +- **14.1.10** When sanity passes, save + ([section 2.3](#23-save-after-every-ingest-cycle-and-housekeeping-pass)). -Relative markdown links must point at a real file. Wikilinks -`[[page]]` must match a markdown file stem. Both are checked by -`mdlinter` inside `wiki sanity`. Follow +**14.2** Relative markdown links must point at a real file. +Wikilinks `[[page]]` must match a markdown file stem. Both are +checked by `mdlinter` inside `wiki sanity`. Follow [schema](../../../wiki/schema.md) for page names, tags, and link style. -Ingest drops files in `ingest/`. Archive sources under `raw/` and -do not edit them after that. Synthesized pages go under `wiki/` -(`concepts/`, `entities/`, `comparisons/`, `queries/`). The full -ingest is section 16. +**14.3** Ingest drops files in `ingest/`. Archive sources under +`raw/` and do not edit them after that. Synthesized pages go under +`wiki/` (`concepts/`, `entities/`, `comparisons/`, `queries/`). The +full ingest is section 18. -## 13. Frontmatter +## 15. Frontmatter -Required on every markdown file: `name`, `version`. Bump `version` -when the page is edited. Set `updated` when you edit. +**15.1** Required on every markdown file: `name`, `version`. Bump +`version` when the page is edited. Set `updated` when you edit. -Required on every page under `wiki/`: the quality signals -`confidence`, `llm`, `contested`, and `contradictions`. Set them -in the same edit. fmlint enforces `name` and `version` only. +**15.2** Required on every page under `wiki/`: the quality signals +`confidence`, `llm`, `contested`, and `contradictions`. Set them in +the same edit. fmlint enforces `name` and `version` only. + +**15.3** Fields: | Field | Required | Rule | | --- | --- | --- | @@ -672,29 +757,32 @@ in the same edit. fmlint enforces `name` and `version` only. | `type` | no | `entity`, `concept`, `comparison`, `query`, `summary`, `toc` | | `tags` | no | From the taxonomy in schema.md. | | `sources` | no | Paths such as `raw/articles/source-name.md`. | -| `confidence` | wiki pages | `trusted`, `high`, `medium`, `low`, or `zero`. See below. | +| `confidence` | wiki pages | `trusted`, `high`, `medium`, `low`, or `zero`. See section 15.4.2. | | `llm` | wiki pages | `model` is required. Add `reasoning` when the model reports a level. | | `contested` | wiki pages | `true` when a claim is disputed, otherwise `false`. | | `contradictions` | wiki pages | Other page slugs that disagree. `[]` when none. | | `description` | SKILL.md only | Required when the file name is `SKILL.md`. | -| `status` | ingested pages | `raw` from `wiki ingest`. You set `refined` (section 16.4). | -| `ingest`, `source_meta` | ingested pages | Written by `wiki ingest`. Check them (section 16.3). Keep them when you refine. | +| `status` | ingested pages | `raw` from `wiki ingest`. You set `refined` (section 18.5). | +| `ingest`, `source_meta` | ingested pages | Written by `wiki ingest`. Check them (section 18.4). Keep them when you refine. | -### Quality signals +### 15.4. Quality signals -On every page under `wiki/`, fill these before you finish the edit. -Refresh them when you change the page. `llm.model` is the model -that made this edit. +**15.4.1** On every page under `wiki/`, fill these before you finish +the edit. Refresh them when you change the page. `llm.model` is the +model that made this edit. -`confidence` is how sure the page is: +**15.4.2** `confidence` is how sure the page is: -- `trusted`: the user stated it, or it is copied from a cited source, with no inference. -- `high`: more than one source agrees, or one source and no known contradiction. -- `medium`: one source, or a synthesis you are reasonably sure of. -- `low`: inference, thin evidence, or an open gap. -- `zero`: unknown, or not checked. +- **15.4.2.1** `trusted`: the user stated it, or it is copied from a + cited source, with no inference. +- **15.4.2.2** `high`: more than one source agrees, or one source + and no known contradiction. +- **15.4.2.3** `medium`: one source, or a synthesis you are + reasonably sure of. +- **15.4.2.4** `low`: inference, thin evidence, or an open gap. +- **15.4.2.5** `zero`: unknown, or not checked. -`llm` records who wrote the page: +**15.4.3** `llm` records who wrote the page: ```yaml llm: @@ -702,23 +790,26 @@ llm: - reasoning: high ``` -`model` is required. Use `none` when no model wrote the page. -Use `Auto` when it is unknown due to the harness using an automatic routing -and can't determine. -Add `reasoning` when the model reports a level. +- **15.4.3.1** `model` is required. Use `none` when no model wrote + the page. +- **15.4.3.2** Use `Auto` when the harness routes to a model + automatically and the model cannot be determined. +- **15.4.3.3** Add `reasoning` when the model reports a level. -`contested` is `true` when any claim on the page is disputed by a -source or by another page. Otherwise `false`. +**15.4.4** `contested` is `true` when any claim on the page is +disputed by a source or by another page. Otherwise `false`. -`contradictions` lists the `name` of each page that disagrees. -Use `[]` when none do. +**15.4.5** `contradictions` lists the `name` of each page that +disagrees. Use `[]` when none do. -`SKILL.md` `name` must match `^[a-z0-9-]+$` and the parent folder. +**15.4.6** `SKILL.md` `name` must match `^[a-z0-9-]+$` and the +parent folder. -## 14. Checks with no own command +## 16. Checks with no own command -`wiki sanity` also runs `mdlinter` and the same frontmatter, ruff, -and JSON checks. There is no `fiona-assistant mdlinter` command. +**16.1** `wiki sanity` also runs `mdlinter` and the same +frontmatter, ruff, and JSON checks. There is no +`fiona-assistant mdlinter` command. | Code | Meaning | | --- | --- | @@ -726,89 +817,95 @@ and JSON checks. There is no `fiona-assistant mdlinter` command. | MDL002 | A `[[wikilink]]` matches no page stem. | | MDL003 | A table row continues onto the next line. | -`pycustomlint` (PYC001 inline import, PYC002 import after code, -PYC003 late declaration) -runs in the product sanity gate, not as a `fiona-assistant` -subcommand. +**16.2** `pycustomlint` (PYC001 inline import, PYC002 import after +code, PYC003 late declaration) runs in the product sanity gate, not +as a `fiona-assistant` subcommand. -## 15. Do not call +## 17. Do not call -These are not implemented. Do not invent flags for them. +**17.1** These are not implemented. Do not invent flags for them. -- `fiona-assistant html-js-lint` -- `fiona-assistant setup` -- `fiona-assistant pycustomlint` -- `fiona-assistant python-ruff fix` +- **17.1.1** `fiona-assistant html-js-lint` +- **17.1.2** `fiona-assistant setup` +- **17.1.3** `fiona-assistant pycustomlint` +- **17.1.4** `fiona-assistant python-ruff fix` -## 16. Ingest cycle +## 18. Ingest cycle -An ingest has two phases. `wiki ingest` (section 4.6) converts the -source into a page at `status: raw`. You refine that page to -`status: refined`. The ingest is done only when both phases are -done. The pattern is section 3.1 of +**18.1** An ingest has two phases. `wiki ingest` (section 6.7) +converts the source into a page at `status: raw`. You refine that +page to `status: refined`. The ingest is done only when both phases +are done. The pattern is section 3.1 of [llm-wiki-mod](./llm-wiki-mod.md). -### 16.1. Convert +### 18.2. Convert -1. When the raw or page location is not obvious, run - `fiona-assistant wiki ingest --dry-run --json` and - check `raw_path` and `page_path`. -2. Run `fiona-assistant wiki ingest --json`, with `-r` and - `-t` when needed. -3. Read the result. When `duplicate_page` is set, the same bytes - are already archived and ingested. Nothing was written. Refine - that page if it is still `status: raw`, else stop and tell the - user which page holds the source. Use `--force` only when the - user wants it converted again. -4. On exit 0 with `conversion: complete` and no warning that lost - content, check the metadata (section 16.3), then refine - (section 16.4). Otherwise decide with section 16.2. +- **18.2.1** When the raw or page location is not obvious, run + `fiona-assistant wiki ingest --dry-run --json` and check + `raw_path` and `page_path`. +- **18.2.2** Run `fiona-assistant wiki ingest --json`, with + `-r` and `-t` when needed. +- **18.2.3** Read the result. When `duplicate_page` is set, the same + bytes are already archived and ingested. Nothing was written. + - **18.2.3.1** Refine that page if it is still `status: raw`, else + stop and tell the user which page holds the source. + - **18.2.3.2** Use `--force` only when the user wants it converted + again. +- **18.2.4** On exit 0 with `conversion: complete` and no warning + that lost content, check the metadata (section 18.4), then refine + (section 18.5). Otherwise decide with section 18.3. -### 16.2. Convert by hand +### 18.3. Convert by hand -The tool is the default, not a rule. You decide when the source is -too complex for it and read the source yourself. Convert by hand -when: +**18.3.1** The tool is the default, not a rule. You decide when the +source is too complex for it and read the source yourself. Convert +by hand when: -- The command exited 3: no converter, or conversion failed. -- `conversion` is `partial` and the warnings show lost text: no text - layer (a scanned PDF), no text content (a page drawn by JavaScript - or a login page), or pages with no text. -- Layout carries the meaning: tables, multi-column papers, forms, - slides, spreadsheets, diagrams, or a bundle of logs. -- The download failed on authentication (401, 403, or a login page - in the body). Fetch the source with a tool that has access, save - it into `ingest/`, and ingest that file. +- **18.3.1.1** The command exited 3: no converter, or conversion + failed. +- **18.3.1.2** `conversion` is `partial` and the warnings show lost + text: no text layer (a scanned PDF), no text content (a page drawn + by JavaScript or a login page), or pages with no text. +- **18.3.1.3** Layout carries the meaning: tables, multi-column + papers, forms, slides, spreadsheets, diagrams, or a bundle of + logs. +- **18.3.1.4** The download failed on authentication (401, 403, or a + login page in the body). Fetch the source with a tool that has + access, save it into `ingest/`, and ingest that file. -How: +**18.3.2** How: -1. Make sure the raw copy exists. When it does not, run - `fiona-assistant wiki ingest --raw-only --json`. Without - the tool, first compute the file's SHA-256 (`sha256sum `, - or `Get-FileHash -Algorithm SHA256 ` in PowerShell) and - compare it with the files under `raw/` of the same size and with - `ingest.sha256` in `wiki/` pages. Reuse a match. Do not archive - the same bytes twice. With no match, copy the file (move it when - it is in `ingest/`) to the default folder in section 4.6, under - a lowercase hyphenated name that is not taken. -2. Read the raw file with your own tools. Never edit it. -3. Write the page where the tool would have put it, with the - complete frontmatter in section 16.5. Set - `ingest.converter: agent`, and fill `sha256`, `size_bytes`, - `media_type`, and `fetched` from the raw file. -4. You may write the refined page in one pass. Then follow - section 16.4 from step 3. +- **18.3.2.1** Make sure the raw copy exists. When it does not, run + `fiona-assistant wiki ingest --raw-only --json`. + - **18.3.2.1.1** Without the tool, first compute the file's + SHA-256 (`sha256sum `, or + `Get-FileHash -Algorithm SHA256 ` in PowerShell). + - **18.3.2.1.2** Compare it with the files under `raw/` of the + same size and with `ingest.sha256` in `wiki/` pages. Reuse a + match. Do not archive the same bytes twice. + - **18.3.2.1.3** With no match, copy the file (move it when it is + in `ingest/`) to the default folder in section 6.7.4, under a + lowercase hyphenated name that is not taken. +- **18.3.2.2** Read the raw file with your own tools. Never edit it. +- **18.3.2.3** Write the page where the tool would have put it, with + the complete frontmatter in section 18.6. Set + `ingest.converter: agent`, and fill `sha256`, `size_bytes`, + `media_type`, and `fetched` from the raw file. +- **18.3.2.4** You may write the refined page in one pass. Then + follow section 18.5.3 onward. -When the tool archived the wrong file or to the wrong place, tell -the user. Do not move or delete anything under `raw/`. +**18.3.3** When the tool archived the wrong file or to the wrong +place, tell the user. Do not move or delete anything under `raw/`. -### 16.3. Check the metadata +### 18.4. Check the metadata -The tool fills every key in section 16.5, but it reads the source -mechanically. Check the frontmatter against the source before you -refine, and fix what is wrong. You may edit `title`, `type`, `tags`, -`source_meta`, `ingest.conversion`, and `ingest.warnings`. You may -not hand-edit `ingest.sha256`, `ingest.size_bytes`, `ingest.fetched`, +**18.4.1** The tool fills every key in section 18.6, but it reads +the source mechanically. Check the frontmatter against the source +before you refine, and fix what is wrong. + +**18.4.2** You may edit `title`, `type`, `tags`, `source_meta`, +`ingest.conversion`, and `ingest.warnings`. You may not hand-edit +`ingest.sha256`, `ingest.size_bytes`, `ingest.fetched`, `ingest.media_type`, or `ingest.tool`. | Key | Check | May edit | @@ -823,58 +920,71 @@ not hand-edit `ingest.sha256`, `ingest.size_bytes`, `ingest.fetched`, | `ingest.origin`, `origin_kind`, `final_url` | Match the URL or path the user gave. | only when wrong | | `ingest.sha256`, `size_bytes`, `media_type`, `fetched`, `tool` | Facts computed by the tool. | no | -Verify `ingest.sha256` against the raw file with `sha256sum` or -`Get-FileHash`. When a computed fact is wrong, re-run the ingest -with `--force` or tell the user. Do not hand-edit it. Bump the page -`version` patch and `updated` when you edit the metadata. +**18.4.3** Verify `ingest.sha256` against the raw file with +`sha256sum` or `Get-FileHash`. -### 16.4. Refine +**18.4.4** When a computed fact is wrong, re-run the ingest with +`--force` or tell the user. Do not hand-edit it. -1. `fiona-assistant wiki ingest --pending --json` lists the pages - still at `status: raw`. Without the tool, search `wiki/` for - `status: raw`. Check each page's metadata first (section 16.3). - A markdown source keeps its body. The tool writes the page - frontmatter when the source did not already have it. Source - frontmatter is in `source_meta`. A text source (`.txt`, `.text`, - `.log`) is one fenced `text` block plus that frontmatter, still - at `status: raw`. Rewrite that fence. The tool does not refine it. -2. Read the page and the source it cites. Open the raw file when - `ingest.conversion` is `partial` or a claim looks garbled. -3. Rewrite the page as a source summary: - - The first paragraph says what the source is and why it matters. - - Keep facts, numbers, decisions, dates, and names. Keep a short - code or log excerpt when it backs a claim. - - Drop boilerplate: menus, cookie notices, repeated headers and - footers, page numbers. - - Stay under the 200-line split rule in the schema. Detail stays - in the raw file, which `sources` cites. -4. Add at least two outbound links, as the schema asks. Link the - concept and entity pages the source touches. -5. Update or create those concept and entity pages, using the page - thresholds in the schema. When the source disagrees with a page, - set `contested` and `contradictions` on both. -6. When the page belongs elsewhere, move it and fix every inbound - link. -7. Set the frontmatter: `status: refined`, `version: 1.0.0`, - `updated`, `type`, `tags` from the taxonomy, and the quality - signals from section 13 with your model in `llm`. Keep - `sources`, `source_meta`, and every `ingest` key. -8. Add the page to `wiki/index.md`. Append - `- YYYY-MM-DD - refined from ` to - `wiki/log.md`. -9. Run the work loop in section 12, ending with `wiki sanity`. +**18.4.5** Bump the page `version` patch and `updated` when you edit +the metadata. -A page is refined when `status` is `refined`, `version` is `1.0.0` -or later, the quality signals are set and `llm.model` is not `none`, -the page is listed in `wiki/index.md`, and `wiki/log.md` has a line -for the refinement. The ingest is done when every page from the run -meets that and `wiki sanity` passes. Then save it -([save rule](#save-after-every-ingest-cycle-and-housekeeping-pass)). +### 18.5. Refine -### 16.5. Raw page frontmatter +- **18.5.1** `fiona-assistant wiki ingest --pending --json` lists the + pages still at `status: raw`. + - **18.5.1.1** Without the tool, search `wiki/` for + `status: raw`. + - **18.5.1.2** Check each page's metadata first (section 18.4). + - **18.5.1.3** A markdown source keeps its body. The tool writes + the page frontmatter when the source did not already have it. + Source frontmatter is in `source_meta`. + - **18.5.1.4** A text source (`.txt`, `.text`, `.log`) is one + fenced `text` block plus that frontmatter, still at + `status: raw`. Rewrite that fence. The tool does not refine it. +- **18.5.2** Read the page and the source it cites. Open the raw + file when `ingest.conversion` is `partial` or a claim looks + garbled. +- **18.5.3** Rewrite the page as a source summary: + - **18.5.3.1** The first paragraph says what the source is and why + it matters. + - **18.5.3.2** Keep facts, numbers, decisions, dates, and names. + Keep a short code or log excerpt when it backs a claim. + - **18.5.3.3** Drop boilerplate: menus, cookie notices, repeated + headers and footers, page numbers. + - **18.5.3.4** Stay under the 200-line split rule in the schema. + Detail stays in the raw file, which `sources` cites. +- **18.5.4** Add at least two outbound links, as the schema asks. + Link the concept and entity pages the source touches. +- **18.5.5** Update or create those concept and entity pages, using + the page thresholds in the schema. When the source disagrees with + a page, set `contested` and `contradictions` on both. +- **18.5.6** When the page belongs elsewhere, move it and fix every + inbound link. +- **18.5.7** Set the frontmatter: `status: refined`, + `version: 1.0.0`, `updated`, `type`, `tags` from the taxonomy, and + the quality signals from section 15 with your model in `llm`. Keep + `sources`, `source_meta`, and every `ingest` key. +- **18.5.8** Add the page to `wiki/index.md`. Append + `- YYYY-MM-DD - refined from ` to + `wiki/log.md`. +- **18.5.9** Run the work loop in section 14, ending with + `wiki sanity`. -`wiki ingest` writes every key below, even when a value is empty -(`[]`, `{}`, or `null`). Write the same when you convert by hand. +**18.5.10** A page is refined when `status` is `refined`, `version` +is `1.0.0` or later, the quality signals are set and `llm.model` is +not `none`, the page is listed in `wiki/index.md`, and `wiki/log.md` +has a line for the refinement. + +**18.5.11** The ingest is done when every page from the run meets +section 18.5.10 and `wiki sanity` passes. Then save it +([section 2.3](#23-save-after-every-ingest-cycle-and-housekeeping-pass)). + +### 18.6. Raw page frontmatter + +**18.6.1** `wiki ingest` writes every key below, even when a value +is empty (`[]`, `{}`, or `null`). Write the same when you convert by +hand. ```yaml --- @@ -909,87 +1019,108 @@ contradictions: [] --- ``` -When you write it by hand, set `llm.model` to your model and -`confidence` from section 13. +**18.6.2** When you write it by hand, set `llm.model` to your model +and `confidence` from section 15.4.2. -## 17. Housekeeping +## 19. Housekeeping ```text fiona-assistant housekeeping [directory] [--json] ``` -One call runs the maintenance routine and tells you what needs work. -Run it when the user asks for maintenance, after `wiki update`, and -before you call the wiki healthy. It writes only `wiki_graph.html`. -It never runs `wiki update`, merges, or fixes anything. That is -your part. +**19.1** One call runs the maintenance routine and tells you what +needs work. + +**19.2** Run it when the user asks for maintenance, after +`wiki update`, and before you call the wiki healthy. + +**19.3** It writes only `wiki_graph.html`. It never runs +`wiki update`, merges, or fixes anything. That is your part. + +**19.4** Steps: | Step | Status when not clean | You do | | --- | --- | --- | | `check-update` | `WARN`: the template has updates, conflicts, or additions | Ask the user, then `wiki update --dry-run` and `wiki update --yes`. | -| `merge` | `FAIL`: a file whose stem ends with `-update` exists | Section 17.1. | -| `ingest` | `WARN`: a page under `wiki/` is at `status: raw` | Refine it, section 16.4. | -| `sanity` | `FAIL`: a `wiki sanity` violation | Fix it, section 12. | +| `merge` | `FAIL`: a file whose stem ends with `-update` exists | Section 19.8. | +| `ingest` | `WARN`: a page under `wiki/` is at `status: raw` | Refine it, section 18.5. | +| `sanity` | `FAIL`: a `wiki sanity` violation | Fix it, section 14. | | `graph` | `WARN`: a page under `wiki/` has no link in or out | Link it from a related page or from `wiki/index.md`. | -Every step runs, even after one fails. Exit 0 when no step is -`FAIL`; `WARN` steps do not fail the run. Pass `--json` and read -`steps[].status`, `details`, and `next`: `next` is the action to -take. The human report ends with `HOUSEKEEPING PASSED`, or +**19.5** Every step runs, even after one fails. Exit 0 when no step +is `FAIL`; `WARN` steps do not fail the run. + +**19.6** Pass `--json` and read `steps[].status`, `details`, and +`next`: `next` is the action to take. The human report ends with +`HOUSEKEEPING PASSED`, or `HOUSEKEEPING FAILED - step(s) need attention: `. -Work through the `FAIL` steps, then run housekeeping again until it -passes. Fix a `WARN` when the user's task covers it; otherwise -report it. Then save the pass -([save rule](#save-after-every-ingest-cycle-and-housekeeping-pass)). +**19.7** Work through the `FAIL` steps, then run housekeeping again +until it passes. Fix a `WARN` when the user's task covers it; +otherwise report it. Then save the pass +([section 2.3](#23-save-after-every-ingest-cycle-and-housekeeping-pass)). -### 17.1. Merge an update file +### 19.8. Merge an update file -`wiki update` yields to the owner. When the packaged template has a -newer version of a markdown file the owner may have edited, such as -`AGENTS.md`, `wiki/index.md`, `wiki/log.md`, or `wiki/schema.md`, -the owner's file stays as it is and the template is written beside -it as `-update.md`. `AGENTS-update.md` belongs to -`AGENTS.md`; `wiki/schema-update.md` belongs to `wiki/schema.md`. -A differing `fiona-settings.json` is the same kind of conflict: -the owner's file stays, and the template is -`fiona-settings-update.json`. The update file is the new template -text, as `wiki init` would write it. You decide how the two become -one file: +**19.8.1** `wiki update` yields to the owner. When the packaged +template has a newer version of a markdown file the owner may have +edited, such as `AGENTS.md`, `wiki/index.md`, `wiki/log.md`, or +`wiki/schema.md`, the owner's file stays as it is and the template +is written beside it as `-update.md`. -1. Read both files. Compare them, for example with - `git diff --no-index -update.md`. -2. Keep every owner edit: their wording, tags, sections, index - entries, and log lines. The owner wins where the two disagree on - content. -3. Take what the template adds or corrects: new sections, new - rules, new frontmatter fields, and fixed command names, flags, - or paths. A tool contract (a command, a flag, a required field) - comes from the template. -4. For `wiki/index.md` and `wiki/log.md`, keep the owner's entries. - Take only new structure from the template. Never replace the - owner's entries with the template's. -5. Set `version` to at least the update file's `version`, and - `updated` to today. Keep the owner's `created`. A lower version - brings the conflict back on the next `wiki update`. -6. Delete the update file. -7. Append `- YYYY-MM-DD - merged -update.md into ` to - `wiki/log.md`, and bump that file. -8. Run `fiona-assistant housekeeping` again. `merge` passes when no - update file is left. +**19.8.2** `AGENTS-update.md` belongs to `AGENTS.md`; +`wiki/schema-update.md` belongs to `wiki/schema.md`. -For `fiona-settings-update.json`, merge by the settings rule. Start -from the template object. For each user key, if both sides are -objects, merge recursively; otherwise the user value replaces the -template value. Keep new template keys the user does not set. Write -the result as JSON with indent 2 and a trailing newline, then delete -`fiona-settings-update.json`. Settings have no `version` field. -Line endings alone are not a difference. A later `wiki update` -lists the file again only when the text still differs and -`corefiles.json` does not already record that same version -and hash pair. Log the merge the same way as step 7. +**19.8.3** A differing `fiona-settings.json` is the same kind of +conflict: the owner's file stays, and the template is +`fiona-settings-update.json`. -When you cannot tell whether an owner edit or a template change -should win, ask the user. Do not delete an update file you have not -merged unless the user says to drop it. When the target file is -missing, ask the user whether to restore it from the update file. +**19.8.4** The update file is the new template text, as `wiki init` +would write it. You decide how the two become one file: + +- **19.8.4.1** Read both files. Compare them, for example with + `git diff --no-index -update.md`. +- **19.8.4.2** Keep every owner edit: their wording, tags, sections, + index entries, and log lines. The owner wins where the two + disagree on content. +- **19.8.4.3** Take what the template adds or corrects: new + sections, new rules, new frontmatter fields, and fixed command + names, flags, or paths. A tool contract (a command, a flag, a + required field) comes from the template. +- **19.8.4.4** For `wiki/index.md` and `wiki/log.md`, keep the + owner's entries. Take only new structure from the template. Never + replace the owner's entries with the template's. +- **19.8.4.5** Set `version` to at least the update file's + `version`, and `updated` to today. Keep the owner's `created`. A + lower version brings the conflict back on the next `wiki update`. +- **19.8.4.6** Delete the update file. +- **19.8.4.7** Append + `- YYYY-MM-DD - merged -update.md into ` to + `wiki/log.md`, and bump that file. +- **19.8.4.8** Run `fiona-assistant housekeeping` again. `merge` + passes when no update file is left. + +**19.8.5** For `fiona-settings-update.json`, merge by the settings +rule: + +- **19.8.5.1** Start from the template object. +- **19.8.5.2** For each user key, if both sides are objects, merge + recursively; otherwise the user value replaces the template value. +- **19.8.5.3** Keep new template keys the user does not set. +- **19.8.5.4** Write the result as JSON with indent 2 and a trailing + newline, then delete `fiona-settings-update.json`. +- **19.8.5.5** Log the merge the same way as section 19.8.4.7. + +**19.8.6** Settings have no `version` field. Line endings alone are +not a difference. A later `wiki update` lists the file again only +when the text still differs and `corefiles.json` does not already +record that same version and hash pair. + +**19.8.7** When you cannot tell whether an owner edit or a template +change should win, ask the user. + +**19.8.8** Do not delete an update file you have not merged unless +the user says to drop it. + +**19.8.9** When the target file is missing, ask the user whether to +restore it from the update file. diff --git a/src/fiona/data/llm_wiki/utils/skills/fiona/llm-wiki-mod.md b/src/fiona/data/llm_wiki/utils/skills/fiona/llm-wiki-mod.md index e4dce85..51da04f 100644 --- a/src/fiona/data/llm_wiki/utils/skills/fiona/llm-wiki-mod.md +++ b/src/fiona/data/llm_wiki/utils/skills/fiona/llm-wiki-mod.md @@ -1,8 +1,8 @@ --- name: llm-wiki-mod title: LLM Wiki Pattern -version: 1.4.0 -updated: 2026-09-30 +version: 1.4.1 +updated: 2026-10-06 type: concept tags: [wiki, schema] --- @@ -80,7 +80,7 @@ edited again. An ingest has two phases. A tool fetches the source, archives it under `raw/`, and converts it into a page marked `status: raw`. The agent then refines that page, updates the pages it touches, -and marks it `status: refined`. The steps are section 16 of the +and marks it `status: refined`. The steps are section 18 of the [Fiona skill](./SKILL.md). ### 3.2. Query @@ -122,7 +122,7 @@ one cleanup can be reviewed or undone on its own. File names, frontmatter, tags, links, and when to create or split a page are in [schema](../../../wiki/schema.md). How to fill the -frontmatter and the quality signals is section 13 of the +frontmatter and the quality signals is section 15 of the [Fiona skill](./SKILL.md). This file does not repeat them. ## 6. Why this works diff --git a/src/fiona/data/llm_wiki/wiki/schema.md b/src/fiona/data/llm_wiki/wiki/schema.md index ffead12..ee6bc58 100644 --- a/src/fiona/data/llm_wiki/wiki/schema.md +++ b/src/fiona/data/llm_wiki/wiki/schema.md @@ -1,9 +1,9 @@ --- name: llm-wiki-schema title: LLM Wiki Schema -version: 1.6.0 +version: 1.6.1 created: 2025-09-18 -updated: 2026-09-30 +updated: 2026-10-06 type: summary tags: [wiki, schema] confidence: trusted @@ -38,7 +38,7 @@ and extend the tag taxonomy to match it. `name` and `version` are required on every md file. Every page under `wiki/` also requires the quality signals `confidence`, `llm`, -`contested`, and `contradictions`. How to set them is section 13 of +`contested`, and `contradictions`. How to set them is section 15 of the [Fiona skill](../utils/skills/fiona/SKILL.md). All other fields are optional. This is the single canonical frontmatter schema. fmlint enforces `name` and `version` only. @@ -60,7 +60,7 @@ llm: - reasoning: high # when the model reports a level contested: true contradictions: [other-page-slug] -# written by wiki ingest (Fiona skill section 16): +# written by wiki ingest (Fiona skill section 18): status: raw | refined source_meta: {author: Name} # metadata read from the source ingest: # provenance; every key always present