Files
2026-09-29 10:10:20 +02:00

95 KiB

name, version, title, created, updated, type
name version title created updated type
Fiona-Assistant Specification 0.39.0 Project Spec Sheet 2026-09-23 2026-09-29 summary

Fiona-Assistant Specification

Spec sheet for the Fiona-Assistant toolkit. Derived from the current codebase state. Each feature section opens with a Status line followed by requirements and acceptance criteria. Status values: Implemented, Partially Implemented, Planned, Won't Do.

The spec is organized in three parts: llm-wiki specifications (the content system the tools operate on), Fiona tool specifications (one numbered section per src/ tool — add new tools as a new numbered section there), and project infrastructure (cross-cutting concerns).


1. Product Overview

Fiona-Assistant is a CLI toolkit for managing, organizing, and maintaining LLM wiki content with minimal AI token dependence.

1.1. Document Structure Convention

Status: Implemented

Covers: This document's structure and formatting rules.

All sections follow a consistent numbering and formatting convention.

  • 1.1.1. Main sections use ## N. Section Title (e.g., ## 1. Product Overview)

  • 1.1.2. Subsections use ### N.M. Subsection Title (e.g., ### 3.1. llm-wiki Lifecycle Management)

  • 1.1.3. Sub-subsections use #### N.M.K. Sub-subsection Title (e.g., #### 3.1.1. \wiki init``)

    • acceptance: All headings follow the N.M.K numbering pattern
  • 1.1.4. Each section/subsection starts with description text

  • 1.1.5. Requirements use format: - *N.M.K.* <requirement>

    • acceptance: All requirements are numbered according to their section hierarchy
  • 1.1.6. Unimplemented requirements use NOT_IMPLEMENTED tag (e.g., - *N.M.K.* \NOT_IMPLEMENTED``)

    • acceptance: All unimplemented items are clearly tagged
  • 1.1.7. Acceptance criteria use format: - *acceptance*: <criteria>

    • acceptance: All acceptance criteria are explicitly stated

Example:

### 3.1.1. Feature Name

> Status: Implemented
>
> Covers: `module/file.py`

Description of what this feature does.

- *3.1.1.1.* Requirement description
  - *acceptance*: Specific acceptance criteria for this requirement

Note: TOC is auto-generated by fiona-assistant librarian toc file. Do not manually edit the TOC between <!-- toc-start: headings 2..6 --> and <!-- toc-end -->.

  • 1.2. Core philosophy: use deterministic scripts for deterministic tasks, reserve LLM tokens for creative and analytical work.
  • 1.3. Constraints that apply to every feature:
    • 1.3.1. All tool output must be AI-friendly: parseable, consistently structured (see output format examples in README.md).
    • 1.3.2. Interaction model — interactive by default for humans, flags for AI: when a command needs confirmation or input, prompt interactively; the same command must also run non-interactively through explicit flags (e.g. --force, --yes, --non-interactive) so AI agents get deterministic behavior without a TTY.
    • 1.3.3. All code changes follow TDD (tests first, python src/run_tests.py green).
    • 1.3.4. python src/run_sanity.py must pass before any change is finalized.
    • 1.3.5. Python code passes Ruff; markdown passes PyMarkdown lint.
    • 1.3.6. Directory permissions are enforced: personal/ and raw/ are read-only for agents; wiki/concepts/ is read-write.

2. Current Capabilities

All commands below are implemented and reachable through the single fiona-assistant console script (section 4.11). Every optional [directory] argument currently defaults to the current directory; the planned wiki root discovery 4.11.1 will replace that default with the directory containing fiona-settings.json.

2.1. fiona-assistant

  • No args: print the command help and exit 0.

2.2. fiona-assistant librarian [directory]

  • No subcommand: default to list . — list all files recursively.
  • list [directory]: List all files in the directory tree.
  • toc [directory]: Generate a table_of_contents.md per folder: every md file with its frontmatter version and its headers as anchored links.
  • toc file <path_to_file>: Generate an in-file TOC below the H1 4.10.2; --sectionized forces section numbering on for the run.
  • toc file *: Generate in-file TOCs for every md file in the current directory / llm-wiki root 4.10.2; --sectionized applies as above.

2.3. fiona-assistant mdoctor [subcommand] [directory]

  • No subcommand: default to chklint . — lint the current directory.
  • list [directory]: List all files in the directory tree.
  • chklint [directory]: Scan markdown with pymarkdown and print an AI-friendly violation report (file, line:column, rule).
  • fix [directory]: Apply pymarkdown auto-fixes in place; print the number of fixed files followed by each path.

2.4. fiona-assistant python-ruff [subcommand] [directory]

  • No subcommand: default to chklint . — lint the current directory.
  • chklint [directory]: Lint Python sources with ruff and print an AI-friendly violation report (file, line:column, rule code).

2.5. fiona-assistant fmlint [subcommand] [directory]

  • No subcommand: default to chklint ..
  • chklint [directory]: Validate markdown frontmatter (4.2). The report lists violations and the command exits 0.

2.6. fiona-assistant json-lint [subcommand] [directory]

  • No subcommand: default to chklint ..
  • chklint [directory]: Validate JSON syntax and style (4.13).
  • fix [directory]: Auto-fix indentation and trailing commas in place.

2.7. fiona-assistant git [subcommand]

Local git operations on an llm-wiki (3.2).

  • No subcommand: print the git help and exit 1.
  • save --description TEXT: Commit current edits.
  • snapshot <tag_name> --description TEXT: Save, then create a tag.
  • restore <tag_name>: New commit restoring the tagged tree.
  • clean: Discard uncommitted edits.
  • undo: Move back one commit. Fails when the working tree is dirty.
  • status: Print a short git status.

2.8. fiona-assistant wiki [subcommand] [directory]

  • No subcommand: print the wiki help and exit 1.
  • init [directory]: Initialize the directory as an llm-wiki by copying the packaged template; refuses with FileExistsError if an llm-wiki structure already exists.
  • check-update [directory]: Check for updates to llm-wiki core files against the packaged template version. Supports --json flag for AI-friendly output format.
  • update [directory] [-f|--force] [-y|--yes] [--dry-run]: Update core files in the wiki to the current template version. Note: the fiona-settings.json file receives special handling to preserve user customizations when updating.
  • graph [directory] [--output FILE] [--format html|json] [--include-pattern GLOB] [--exclude-pattern GLOB] [--wikilinks/--no-wikilinks] [--json]: Generate an interactive Obsidian-style graph visualization of markdown file connections as HTML or JSON. Supports --json flag for AI-friendly output format.

Notes:

  • Invalid directories exit 1 with an error message on stderr.
  • chklint commands print violations in their report but exit 0; the violation count lives in the output, not the exit code.
  • The standalone console scripts librarian and mdoctor behave like fiona-assistant librarian list and the fiona-assistant mdoctor subcommands respectively.
  • librarian toc accepts --recursive-toc, --non-recursive, and repeatable --exclude REGEX flags (see 4.10). Depth limits come from the markdown.toc_generator section of the packaged fiona-settings.json (max_depth_from_file, max_depth_from_dir; absent or 0 = unlimited, see 4.10).

3. llm-wiki Specifications

3.1. llm-wiki Lifecycle Management

Status: Implemented

Covers: wiki init (implemented, 3.1.1), wiki update (implemented, 3.1.2), wiki sanity (implemented, 3.1.3), wiki graph (implemented, 3.1.4).

3.1.1. wiki init

Status: Implemented

Covers: src/wiki/wiki.py and the fiona-assistant wiki init subcommand (see 4.11).

Initialize the current or given directory as an llm-wiki directory by copying the packaged template (structure + core files) from src/llm-wiki/, replacing DATE_PLACEHOLDER with the current UTC date.

  • 3.1.1.1. If target directory already contains an llm-wiki structure, refuse rather than silently overwriting: the command raises FileExistsError naming the conflicting indicator files (AGENTS.md, fiona-settings.json, wiki/, ingest/, README.md).
    • acceptance: Running wiki init on an empty directory produces a valid llm-wiki tree that passes mdoctor chklint (and wiki sanity once 3.1.3 ships).
  • 3.1.1.2. Copied files carry frontmatter with version and created/updated dates.
    • acceptance: Re-running on an initialized directory fails with FileExistsError and does not clobber user content.
  • 3.1.1.3. The template copy is a 1:1 copy of the template directory, Python files included; the only exclusions are files and directories matching the glob patterns in fiona.llm-wiki.ignore_files in the template's fiona-settings.json (a matching directory skips its whole subtree; user decision, 2026-09-27). An absent settings file or key copies everything; a non-list ignore_files value fails with ValueError.
    • acceptance: With a pattern in fiona.llm-wiki.ignore_files, wiki init skips the matching files and copies every other template file.

3.1.2. wiki update

Status: Implemented

Update an existing llm-wiki directory to the current template version.

Covers: src/wiki/update_manager.py and the fiona-assistant wiki update and fiona-assistant wiki check-update subcommands.

  • 3.1.2.1. Compare version frontmatter of each core md file/script against the packaged template; update only files whose template version is newer.
    • acceptance: On a pristine wiki init output, wiki check-update reports "already up to date" and wiki update is a no-op.
  • 3.1.2.2. For files without version tagging (e.g., JSON files), compare SHA256 hashes to detect differences.
    • acceptance: On a wiki with outdated core files, wiki check-update identifies all files that need updates and wiki update updates only those files.
  • 3.1.2.3. Update directory structure if the template structure changed (add missing directories, never delete user content).
    • acceptance: Settings files are updated with user customizations preserved.
  • 3.1.2.4. Never overwrite files with local modifications (version bumped or content diverged) silently: confirm interactively when a human runs the command, and provide an explicit flag (e.g. --force or --yes) so AI agents can run it non-interactively (see Product Overview interaction model); report skipped files instead.
    • acceptance: Files in read-only zones are never modified.
  • 3.1.2.5. Respect read-only zones (personal/, raw/).
    • acceptance: Running with --force or --yes skips interactive confirmation.
  • 3.1.2.6. Special handling for fiona-settings.json: when updating, merge user customizations from the existing file with the new template version, giving priority to user values for user-configurable fields. This ensures user settings are preserved across updates.
    • acceptance: Running with --dry-run shows what would be updated without making changes.
  • 3.1.2.7. wiki check-update command: check and report what files need updating without modifying anything. Supports --json flag for AI-friendly output format.
  • 3.1.2.8. wiki update command: perform the actual updates. Supports --dry-run flag to preview changes without applying them, and --json flag for AI-friendly output format.

3.1.3. wiki sanity

Status: Implemented

Covers: src/wiki/sanity_manager.py and the fiona-assistant wiki sanity subcommand (see 4.11).

Run all validation checks over an llm-wiki directory in one command.

  • 3.1.3.1. Runs markdown lint (mdoctor), frontmatter lint (fmlint, 4.2), Python lint (python-ruff), the custom markdown linter (mdlinter, 4.3.3), JSON lint (jsonlint, 4.13), and any other registered linters (shellcheck once 4.6 ships).

    • acceptance: Exits 0 on a clean wiki, non-zero on any violation.
  • 3.1.3.2. Checks for dead links (relative and wiki-internal links resolving to existing files).

    • acceptance: A wiki containing a link to a deleted file is reported as a dead link.
  • 3.1.3.3. Aggregated, AI-friendly summary report: per-check pass/fail, violation counts, overall exit code.

  • 3.1.3.4. Checks run concurrently (one thread each) through the shared SanityRunner engine 4.9.5 ; per-check durations are wall-clock times of parallel checks, so they can overlap and need not sum to the total 4.9.4 . The Results: table keeps the registry order regardless of completion order.

  • 3.1.3.5. The dead-link check lives in the custom markdown linter mdlinter (4.3.3 , codes MDL001/MDL002); relative markdown links and [[wikilinks]] are checked, fenced code blocks, inline code, external links, and pure anchors are ignored.

  • 3.1.3.6. Link sources in read-only zones (personal/, raw/) are skipped: files there cannot be fixed. Links pointing into these zones are still resolved.

  • 3.1.3.7. Markdown lint uses the wiki's own fiona-settings.json.

  • 3.1.3.8. Human-readable runs draw the same live per-check progress as run_sanity 4.9.6 , reusing its ProgressDisplay: one bar line per check on a terminal, Running Wiki Sanity .... progress dots on an interval when piped or with --non-interactive. JSON runs skip the display so their output stays parseable.

  • 3.1.3.9. Checks live in SANITY_CHECKS, a (name, runner) registry in src/wiki/sanity_manager.py; add new checks (e.g. shellcheck once 4.6 ships) by implementing a runner and appending it. Current checks: markdownlint (mdoctor), frontmatter_lint (fmlint, 4.2 — required name/version frontmatter per project agent rule 10.1), python-ruff, mdlinter, jsonlint (4.13).

  • 3.1.3.10. The report mirrors run_sanity 4.9.3/ 4.9.4: failing checks' linter reports first (clean runs print no report), a Results: table with per-check PASS/FAIL - <count> violations (MM:SS) lines, then a SANITY PASSED (MM:SS) banner or a Sanity Failed - N checks need attention: verdict with per-check detail lines.

  • 3.1.3.11. --json outputs the aggregated results (per-check result, counts, durations, and raw violations) in JSON for AI agents.

3.1.4. wiki graph

Status: Implemented

Covers: src/wiki/graph_generator.py, src/wiki/templates/graph_template.html, and the fiona-assistant wiki graph subcommand.

Phases 1-4 are in that module and template: link parser, CLI, color and physics, and the BFS layout.

Generate an interactive Obsidian-style graph visualization showing connections between markdown files based on internal links. The graph helps users visualize knowledge relationships and navigate their wiki structure.

  • 3.1.4.1. Parse markdown files to extract internal links.

    • 3.1.4.1.1. Support standard markdown links: [text](path/to/file.md).
    • 3.1.4.1.2. Support relative paths: ./file.md, ../dir/file.md.
    • 3.1.4.1.3. Support absolute paths within wiki root.
    • 3.1.4.1.4. Support wikilinks format: [[file]], [[dir/file]] with configurable enable/disable.
    • 3.1.4.1.5. Resolve link paths to absolute paths relative to wiki root.
    • 3.1.4.1.6. Identify and handle external links (URLs starting with http://, https://, ftp://, etc.).
    • 3.1.4.1.7. Track external links count per file for reporting.
    • 3.1.4.1.8. --external-links / --no-external-links: Enable/disable visibility of external link nodes in the graph (default: disabled).
      • acceptance: All valid internal link formats are extracted and resolved to target files. External links are identified and can be optionally visualized.
  • 3.1.4.2. Build a directed graph data structure.

    • 3.1.4.2.1. Nodes represent markdown files.
    • 3.1.4.2.2. Edges represent links from source to target files.
    • 3.1.4.2.3. Track edge weight (number of links between same files).
    • 3.1.4.2.4. Identify orphan files (no incoming or outgoing links).
      • acceptance: Graph accurately represents all internal links between files.
  • 3.1.4.3. Generate interactive HTML visualization.

    • 3.1.4.3.1. Use D3.js or vis.js for rendering (no external Python dependencies; JS libraries loaded from CDN with local fallback).
    • 3.1.4.3.2. Include all linked JS/CSS inline or from CDN with fallback.
    • 3.1.4.3.3. Support force-directed layout (default).
    • 3.1.4.3.4. Support hierarchical layout option.
    • 3.1.4.3.5. Node hover shows: filename, number of incoming/outgoing links.
    • 3.1.4.3.6. Edge hover shows: link count between files.
    • 3.1.4.3.7. On node hover, highlight the node in light blue with slightly larger size, and highlight all connected edges in blue (if they are gray).
    • 3.1.4.3.8. Single click on node: toggles persistent highlight (light blue, slightly larger), displays node's frontmatter card in upper left corner.
    • 3.1.4.3.9. Clicking on another node switches the highlight to that node.
    • 3.1.4.3.10. Clicking on empty space or any non-node area removes the highlight.
    • 3.1.4.3.11. Double click on node opens the markdown file in default editor or browser.
    • 3.1.4.3.12. Color coding: nodes use a spectrum from gray (no links) to orange to red (high connectivity) based on in-degree, relative to the maximum in-degree in the graph.
    • 3.1.4.3.13. Node sizing: circle radius scales proportionally to the node's in-degree (number of incoming links) relative to the maximum in-degree in the graph, with minimum radius 5 and maximum radius 25.
    • 3.1.4.3.14. Edge coloring: use a spectrum from gray (single link) to yellow to red (many links) based on link count, rather than increasing line thickness.
    • 3.1.4.3.15. Gravity force: mass-based physics system with continuous fluid movement.
      • Node mass: mass = 1 + in_degree * 0.5 (bigger nodes = heavier)
      • Central gravity: proportional to mass (0.02 * mass * alpha)
        • heavier nodes pulled more strongly to center
      • Repulsion: forceManyBody().strength(d => -20 * d.mass) - heavier nodes repel more strongly
      • All nodes continuously pulled toward center, even during dragging
      • Nodes continuously repel each other for fluid movement
      • Bigger nodes tend toward center; smaller nodes give way
      • Center force: 0.1, alphaMin: 0.1, velocityDecay: 0.6 for continuous motion
      • acceptance: Generated HTML renders correctly in modern browsers with interactive features.
  • 3.1.4.4. CLI Integration.

    • 3.1.4.4.1. Add graph subcommand to fiona-assistant wiki.
    • 3.1.4.4.2. Default directory is current directory or wiki root (via settings discovery).
    • 3.1.4.4.3. --output FILE or -o FILE: Output file path (default: wiki_graph.html in current directory).
    • 3.1.4.4.4. --format html|json: Output format (default: html).
    • 3.1.4.4.5. --include-pattern GLOB: Include only files matching pattern (default: **/*.md).
    • 3.1.4.4.6. --exclude-pattern GLOB: Exclude files matching pattern.
    • 3.1.4.4.7. --wikilinks / --no-wikilinks: Enable/disable wikilink parsing (default: enabled).
    • 3.1.4.4.8. --json: Alias for --format json.
    • 3.1.4.4.9. --external-links / --no-external-links: Enable/disable visibility of external link nodes in the graph (default: disabled).
      • acceptance: Command integrates with existing CLI and follows interaction model (interactive confirmation for overwrites).
  • 3.1.4.5. JSON Output Format.

    • 3.1.4.5.1. Output structure:

      {
        "metadata": {
          "wiki_root": "/path/to/wiki",
          "file_count": 42,
          "link_count": 128,
          "generated_at": "2026-09-28T12:00:00Z"
        },
        "nodes": [
          {
            "id": "wiki/concepts/file1.md",
            "name": "file1.md",
            "in_degree": 5,
            "out_degree": 3
          }
        ],
        "edges": [
          {
            "source": "wiki/concepts/file1.md",
            "target": "wiki/concepts/file2.md",
            "weight": 2
          }
        ],
        "orphans": ["wiki/orphan/file.md"]
      }
      
      • acceptance: JSON output is valid and contains all graph data.
    • 3.1.4.5.2. Include external links information in metadata:

      {
        "external_links_count": 10,
        "external_links": [
          {"url": "https://example.com", "count": 3},
          {"url": "https://another.com", "count": 7}
        ]
      }
      
  • 3.1.4.6. HTML Output Features.

    • 3.1.4.6.1. Self-contained HTML file with embedded visualization.
    • 3.1.4.6.2. Responsive design (adapts to viewport size).
    • 3.1.4.6.3. Search/filter nodes by name with degree-based visibility:
      • Matching nodes highlighted in light blue and enlarged (1.5x radius)
      • All neighbor nodes (any degree) shown in gray
      • Non-reachable nodes hidden
    • 3.1.4.6.4. Toggle button (not checkbox) to show/hide orphan nodes.
      • acceptance: HTML file opens in browser and displays interactive graph.
  • 3.1.4.7. Error Handling.

    • 3.1.4.7.1. Gracefully handle broken links (target file doesn't exist).
    • 3.1.4.7.2. Report files with no valid links.
    • 3.1.4.7.3. Validate output path is writable.
      • acceptance: Errors are reported clearly without crashing.
  • 3.1.4.8. Performance.

    • 3.1.4.8.1. Handle wikis with 1000+ files efficiently.
    • 3.1.4.8.2. Use lazy loading for large graphs in HTML.
      • acceptance: Graph generation completes in < 10 seconds for 1000 files on modern hardware.
  • 3.1.4.9. Settings Integration.

    • 3.1.4.9.1. Respect fiona-settings.json configuration section:

      {
        "fiona": {
          "wiki": {
            "graph": {
              "wikilinks_enabled": true,
              "default_layout": "force-directed",
              "max_files": 5000,
              "node_colors": {
                "low": "#808080",
                "mid": "#ffa500",
                "high": "#ff0000"
              },
              "node_radius": {
                "min": 5,
                "max": 25
              },
              "hover": {
                "color": "#add8e6",
                "edge_color": "#0000ff",
                "size_multiplier": 1.3
              }
            }
          }
        }
      }
      
      • acceptance: Settings override defaults when present.
      • acceptance: node_colors configures the color spectrum (low, mid, high).
      • acceptance: node_radius configures the radius range (min, max).
      • acceptance: hover configures hover behavior (color, edge_color, size_multiplier).
      • acceptance: Hover size is capped at max_radius to prevent nodes from growing too large.
      • acceptance: On hover exit, nodes restore to their original size (not minimum size).
  • 3.1.4.10. Module Structure

    • 3.1.4.10.1. MarkdownLinkParser: Parses markdown files to extract internal links. Handles standard markdown links and optional wikilinks. Resolves relative paths to absolute paths within the wiki root.
    • 3.1.4.10.2. WikiGraph: Builds and maintains a directed graph data structure. Stores nodes (markdown files), edges (links between files with weights), and identifies orphans (files with no connections).
    • 3.1.4.10.3. HTMLGraphRenderer: Generates the interactive HTML visualization using D3.js or vis.js. Renders nodes as circles with size proportional to in-degree, edges with thickness proportional to link count, and includes interactive features.
    • 3.1.4.10.4. wiki_graph_main: CLI entry point that orchestrates parsing, graph building, and rendering based on command-line arguments.
  • 3.1.4.11. Polish Features (Phase 4).

    • 3.1.4.11.1. BFS tree layout: Nodes are arranged in a BFS traversal order with most connected node as root, creating a hierarchical structure.
    • 3.1.4.11.2. Animated node appearance: Nodes appear sequentially following BFS order with fade-in animation, most connected nodes first.
    • 3.1.4.11.3. 360 degree circular edge separation: Child nodes are positioned in perfect circles around their parent at equal angular intervals (360/N degrees).
    • 3.1.4.11.4. Physics-based clustering: Force-directed physics simulation with configurable parameters for link strength, charge repulsion, circular layout, collision detection, and motion damping.
    • 3.1.4.11.5. Continuous correction: Circular layout force continuously enforces 360 degree division, maintaining perfect positioning even during interactions.
    • 3.1.4.11.6. Parent-child following: Children nodes follow their parent when dragged, maintaining cluster integrity through the link force.
    • 3.1.4.11.7. Editable physics constants: All physics parameters are exposed as configurable constants at the top of the template for easy tuning.
      • acceptance: Graph nodes are arranged in visually appealing clusters with children evenly distributed around parents, and the graph remains interactive and fluid.

3.2. Git Version Control for llm-wiki

Status: Implemented

Local version control, snapshot, and backup of wiki content via a GitPython wrapper module (src/gittool/git_tool.py).

  • 3.2.0.1. Operates on the llm-wiki directory (auto-initialize a repo on first use via fiona-assistant wiki init).
    • acceptance: save on a dirty tree produces a commit with the specified message format.
  • 3.2.0.2. Never touches personal/ (exclude it explicitly via .gitignore).
    • acceptance: undo on a dirty tree fails with a clear error message mentioning clean.
  • 3.2.0.3. All operations produce concise, parseable output.
    • acceptance: snapshot then restore <tag_name> returns file contents to the tagged (creates a new commit) state.
  • 3.2.0.4. clean leaves git status empty.

Operations:

Command Behavior
save Commit current edits with message yyyy-mm-dd HH:MM:SS <description>
snapshot save, then create an annotated git tag yyyy-mm-dd_HH-MM-SS_<tag_name>
restore Create a new commit restoring to <tag_name> state; does NOT modify history
clean Discard current uncommitted edits; git status must be clean afterwards
undo Move back one commit. Error if the working tree is dirty; run clean first
status Print a short git status. Does not change the repository

3.2.1. Backup Script

Status: Planned

A user-facing backup mechanism so content can be recovered if something goes wrong.

  • 3.2.1.1. NOT_IMPLEMENTED Bundled script (llm-wiki utils/ or CLI subcommand) that creates a restorable backup of the wiki directory.
    • acceptance: A backup taken before a destructive operation can restore the wiki to the pre-operation state.
  • 3.2.1.2. NOT_IMPLEMENTED Backups are timestamped and stored outside the wiki tree.
  • 3.2.1.3. NOT_IMPLEMENTED Documented restore procedure.
  • 3.2.1.4. NOT_IMPLEMENTED archive in zstd compression, with filename YYYYMMDD_hhmm_description.zstd
    • ask the user for directory or ~/Downloads as default

4. Fiona Tools Specifications

Each tool under src/ gets its own numbered section; insert new tools as the next free section number.

4.1. mdoctor (Markdown Lint)

Status: Implemented

Covers: src/mdoctor/. Wraps the pymarkdown linter over markdown sources with an AI-friendly aggregated violation report in the shared linter format (file, line:column, rule; see 4.9.3). Also provides mdoctor list [directory], which lists all files (not only md) in a directory via the librarian (see 4.8).

4.1.1. MDoctor Auto-Fix (mdoctor fix)

Status: Implemented

Invoke the PyMarkdown auto-fix from the CLI so lint violations with fix-capable rules can be repaired deterministically.

  • 4.1.1.1. fiona-assistant mdoctor fix [directory] and mdoctor fix [directory] run the pymarkdown auto-fix recursively over the target directory (default: current directory).
    • acceptance: Running mdoctor fix on a directory whose md file has an auto-fixable violation (e.g. MD047, missing trailing newline) repairs the file and lists its path.
  • 4.1.1.2. Only rule violations whose plugins support auto-fix are repaired; files are modified in place.
    • acceptance: A directory with nothing to fix reports 0 file(s) fixed.
  • 4.1.1.3. Output is AI-friendly and parseable: the number of fixed files followed by the path of each fixed file.
    • acceptance: An invalid directory exits 1 with an error message.
  • 4.1.1.4. Invalid directories and pymarkdown API failures produce a clear error on stderr with a non-zero exit code, not a stack trace.
  • 4.1.1.5. Limitation: the command does not exclude read-only zones (personal/, raw/) by itself; do not point it at a tree containing them until zone exclusion is added (future hardening).

4.1.2. Lint Configuration (fiona-settings.json)

Status: Implemented

pymarkdown rule configuration is data-driven via fiona-settings.json (shipped with the llm-wiki template and copied into user wikis by wiki init).

  • 4.1.2.1. The markdown.pymarkdown section supports:
    • extensions: list of pymarkdown extension identifiers to enable (e.g. front-matter, so YAML frontmatter is not parsed as setext headings)
    • respect_gitignore: bool; when true, scans and fixes skip files excluded by .gitignore (run_sanity.py relies on this to skip build artifacts)
    • <rule_id>: per-rule property maps; the JSON value type selects the API call (bool → set_boolean_property, int → set_integer_property, str → set_string_property)
    • acceptance: Changing a rule in fiona-settings.json (e.g. md013 line_length) changes scan results on the next run, with no code changes.
  • 4.1.2.2. Known limitation — tables are invisible to sanity: pymarkdown only parses pipe tables when the markdown-tables extension is enabled, and enabling it is blocked. pymarkdownlnt 0.9.40 (latest at the time of writing) crashes with BadTokenizationError on list lines that contain pipes (e.g. generated table_of_contents.md entries for headers like ## [date] create | subject), which breaks run_sanity.py on this repo. Until an upstream fix lands, no table rule (MD055/MD056/MD058/MD060) runs in sanity. The Node.js markdownlint used by IDE extensions parses tables natively, which is why table style errors (MD060) appear in the IDE but not in run_sanity.py. Enable the extension once pymarkdown is fixed.
    • acceptance: Scanning a file whose only issue is a line between the default and configured md013.line_length reports no violation.
  • 4.1.2.3. MD060 (table-column-style) is explicitly disabled for when that day comes: pymarkdown's implementation (new in pymarkdownlnt 0.9.40, plugin v0.5.0) enforces different delimiter-row expectations than the Node.js markdownlint used by IDE extensions (it flags well-formed compact tables whose delimiter width differs from the cell width). Revisit once the implementations converge.
    • acceptance: A table with a structural defect (e.g. missing blank line before it) is reported by run_sanity.py — blocked until the pymarkdown markdown-tables crash is fixed and the extension can be enabled (see the known limitation above).
  • 4.1.2.4. Project table style is "compact": one space around cell content in header, body, and delimiter rows (e.g. | --- | ------- |), which satisfies the Node.js markdownlint MD060 compact style.
  • 4.1.2.5. MDoctor loads the settings file dynamically at construction (default: the packaged llm-wiki fiona-settings.json, overridable via settings_file=), so run_sanity.py and mdoctor fix apply the configured rules without code changes.
  • 4.1.2.6. A missing settings file falls back to plain pymarkdown defaults.

Supported keys today (all under markdown.pymarkdown unless marked otherwise):

Key Status Effect
extensions Implemented Extension ids enabled at scan/fix time (today:
front-matter)
respect_gitignore Implemented Skip files excluded by .gitignore
md025.front_matter_title Implemented no-match: frontmatter title
ignored by MD025
md013 Implemented Line length 100; code blocks and tables exempt
md060.enabled Implemented False; table style rule diverges from Node.js
python.ruff Reserved Unused placeholder for future ruff config
4.5
markdown.toc_generator.max_depth_from_file Implemented Header level cap
per file in TOCs
markdown.toc_generator.max_depth_from_dir Implemented Subfolder
recursion cap for TOCs

How to extend the file (no code changes required):

  1. Look up the pymarkdown rule id and its property names in the pymarkdown rule documentation (e.g. md013 owns line_length).

  2. Add a property map under markdown.pymarkdown, keyed by the rule id — the existing md013 entry shows the shape:

    "md013": {
        "line_length": 100,
        "code_blocks": false,
        "tables": false
    }
    
  3. The JSON value type selects the pymarkdown API call, applied as plugins.<rule_id>.<property>: bool → set_boolean_property, int → set_integer_property, str → set_string_property.

  4. To toggle an entire rule, set "enabled": false inside its map (see md060 above).

  5. To enable a parser extension, append its identifier to extensions (front-matter today; for markdown-tables read the known limitation above first).

  6. Verify with python src/run_sanity.py; the new configuration applies at the next scan.

  7. Caution: typos fail silently. Invalid rule ids and property names are accepted without any error and simply have no effect — pymarkdown validates neither at MDoctor construction nor during the scan. Confirm a new setting by observing its effect on scan results (e.g. a file that violates the rule just configured).

Related but separate: the sanity coverage gate lives in src/settings_dev.json (pytests.required-coverage, see 4.9.2), not in fiona-settings.json. The template copy shipped with the package is src/llm-wiki/fiona-settings.json; wiki init copies it into user wikis 3.1.1 and it marks the wiki root 4.11.1.


4.2. Frontmatter Linter (fmlint)

Status: Implemented

Covers: src/customlinter/fmlint.py. A custom linter specifically for validating markdown file frontmatter against project requirements. This ensures all md files have the required name and version fields for source control and version tracking (see project agent rule 10.1).

4.2.1. Frontmatter Validation

Status: Implemented

Validate that all markdown files in an llm-wiki directory have proper frontmatter with required fields.

Frontmatter schema (canonical; reconciles the formerly competing schemas in project agent rule 10.1 and wiki/schema.md — this schema wins):

Field Required Purpose
name Yes Document name (lowercase, hyphens for wiki pages)
version Yes Source control and change detection (semver); bump when
editing
title No Human-readable page title
created No Creation date (YYYY-MM-DD)
updated No Last-update date (YYYY-MM-DD); bump when editing
type No Page type: entity, concept, comparison, query,
summary, toc
tags No Taxonomy tags (see wiki/schema.md)
sources No Source file references (e.g. raw/articles/source-name.md)
confidence / contested / contradictions No Quality signals
  • 4.2.1.1. Scan all .md files in a directory tree for frontmatter presence.
    • acceptance: A md file missing name field is reported with file path and missing field.
  • 4.2.1.2. Validate that the required fields name and version are present (the minimum for source control per project agent rule 10.1). All other fields, including title, are optional.
    • acceptance: A md file missing version field is reported with file path and missing field.
  • 4.2.1.3. Support configurable required fields via fiona-settings.json for different file types (e.g., type: toc files may have different requirements .
    • acceptance: A md file with name and version but no title passes validation (title is optional).
  • 4.2.1.4. Report violations in AI-friendly format: file path, missing fields and line number (if known). Violation messages use the format <code> - <message>. For frontmatter violations, line numbers are omitted as frontmatter is always at the start of the file (line 1), so 1:1 prefix is redundant.
    • acceptance: A md file with no frontmatter at all is reported as missing all required fields.
  • 4.2.1.5. Handle files without frontmatter (report as missing all required fields).
    • acceptance: A md file with malformed YAML frontmatter is reported with parse error details.
  • 4.2.1.6. Handle files with malformed YAML frontmatter (report as parse error).
    • acceptance: Running on a directory with all valid frontmatter reports no violations.
  • 4.2.1.7. Respect .gitignore and read-only zones (personal/, raw/).
    • acceptance: The linter integrates with wiki sanity 3.1.3 and run_sanity.py 4.9.

4.2.2. CLI Integration

Status: Implemented

Provide CLI access to the frontmatter linter via fiona-assistant.

  • 4.2.2.1. fiona-assistant fmlint chklint [directory] scans for frontmatter violations.

    • acceptance: Running fiona-assistant fmlint on a directory with invalid frontmatter produces a structured violation report.
  • 4.2.2.2. fiona-assistant fmlint with no subcommand defaults to chklint.

    • acceptance: Running on a clean directory reports no violations.
  • 4.2.2.3. Output format matches other linters: AI-friendly violation report with file paths, line numbers, and rule codes.

  • 4.2.2.4. Exit code 0 even with violations (consistent with other linters).

4.2.3. Settings Integration

Status: Implemented

Configure frontmatter linter behavior via fiona-settings.json.

  • 4.2.3.1. Load required fields from markdown.frontmatter.required_fields in fiona-settings.json (default: ['name', 'version']).
    • acceptance: Changing required fields in fiona-settings.json changes validation behavior on the next run.
  • 4.2.3.2. Support per-type requirements: markdown.frontmatter type_requirements maps file types to their specific required fields.
    • acceptance: Disabling the linter via settings skips frontmatter validation.
  • 4.2.3.3. Allow disabling the linter via markdown.frontmatter.enabled (default: true).
    • acceptance: Rules listed in ignore_rules are not reported as violations.
  • 4.2.3.4. Support ignoring specific violation rules via markdown frontmatter.ignore_rules (e.g., ['FML002'] to ignore missing frontmatter).
    • acceptance: Files matching patterns in ignore_file are skipped during linting.
  • 4.2.3.5. Support ignoring specific file patterns via markdown.frontmatter ignore_file (e.g., ['*/AGENTS.md', '*/README.md'] to skip certain files).

4.2.4. SKILL.md Validation

Status: Implemented

Special validation for files named SKILL.md to ensure skill name conventions and required fields.

  • 4.2.4.1. For any file named SKILL.md (case-sensitive), validate the name field in frontmatter.
    • acceptance: A SKILL.md file with name: Python TDD is reported with FML005 violation.
  • 4.2.4.2. Skill name must only contain lowercase letters, numbers, and hyphens (regex: ^[a-z0-9-]+$).
    • acceptance: A SKILL.md file in folder python-tdd/ with name: python-tdd passes validation.
  • 4.2.4.3. Skill name must match the parent folder name (e.g., SKILL.md in folder python-tdd/ must have name: python-tdd).
    • acceptance: A SKILL.md file in folder python-tdd/ with name: other-name is reported with FML006 violation.
  • 4.2.4.4. SKILL.md files must have a description field in frontmatter.
    • acceptance: A SKILL.md file with valid lowercase name matching folder passes validation.
  • 4.2.4.5. Report violations with rule code FML005 for invalid characters, FML006 for folder name mismatch, and FML004 for missing description.
    • acceptance: A SKILL.md file without a description field is reported with FML004 violation.

4.3. Custom Linter Framework

Status: Implemented

Covers: src/customlinter/. A unified framework for custom linters providing shared functionality for file scanning, pattern matching, settings management, and violation reporting. fmlint 4.2 , pycustomlint 4.4, and mdlinter 4.3.3 extend this framework.

4.3.1. Base Linter Class

Status: Implemented

Provide a common base class for all custom linters with shared functionality.

  • 4.3.1.1. Base class BaseLinter in src/customlinter/base.py provides common methods for:
    • File collection (recursive directory scanning)
    • .gitignore and read-only zone handling
    • Settings loading from fiona-settings.json
    • Violation collection and formatting
    • Compressed violations reporting via compress_violations() and format_compressed_violations(), with the report header derived from the linter_name class attribute
    • Common CLI interface with chklint subcommand
    • acceptance: Both FrontmatterLinter and PythonCustomLinter extend BaseLinter.
  • 4.3.1.2. Violation class LintViolation standardizes violation representation with:
    • filepath, line, column, code, message, rule_id, severity
    • to_dict() method for JSON serialization
    • String representation for human-readable output
    • acceptance: Common functionality (file collection, settings, formatting) is not duplicated.
  • 4.3.1.3. Exception classes: LinterError (base), CustomLintError, FrontmatterLintError
    • acceptance: Violation objects from both linters have consistent structure.

4.3.2. Modular Rule System

Status: Implemented

Design the framework to support adding new lint rules as separate modules.

  • 4.3.2.1. Each rule type (e.g., inline imports, frontmatter validation) is implemented as a separate visitor or validator class.
    • acceptance: New rule types can be added by creating new visitor classes.
  • 4.3.2.2. Rules can be added without modifying the base framework.
    • acceptance: Existing linters continue to work when new rules are added.
  • 4.3.2.3. Violation codes follow a consistent naming scheme (e.g., PYC001, FML001).
    • acceptance: Violation codes are unique and documented.
  • 4.3.2.4. Each linter maintains its own violation code dictionary with descriptions.

4.3.3. Markdown Linter (mdlinter)

Status: Implemented

Covers: src/customlinter/mdlinter.py. A custom markdown linter built on BaseLinter; its checks run inside wiki sanity 3.1.3 .

  • 4.3.3.1. MarkdownLinter runs a registry of per-file checks (MarkdownLinter.CHECKS, ordered); new checks are added by implementing a _check_* method and appending its name.
    • acceptance: A wiki containing a link to a deleted file is reported as MDL001 with file, line, and target.
  • 4.3.3.2. The first check is the dead-link check (moved from wiki.sanity_manager.find_dead_links):
    • MDL001: a relative markdown link that does not resolve to an existing file
    • MDL002: a wiki-internal [[wikilink]] that does not resolve to any md page in the wiki (Obsidian-style, matched against md file stems, case-insensitive)
    • acceptance: find_dead_links(wiki_root) returns LintViolation objects and flags only dead links, regardless of other checks.
  • 4.3.3.3. Fenced code blocks, inline code, external links, and pure anchors are ignored. Link sources in read-only zones (personal/, raw/) are skipped; links pointing into those zones are still resolved.
    • acceptance: The report carries the same sections in the same order as the other custom linters.
  • 4.3.3.4. lint_file lints a single file, resolving wikilinks against the file's parent tree; lint_directory lints the whole wiki.
  • 4.3.3.5. Reports reuse the shared format 4.9.3 via compress_violations() / format_compressed_violations() with the Markdown Lint header.

4.4. Python Custom Linter (pycustomlint)

Status: Implemented

Covers: src/customlinter/pycustomlint.py. A custom linter for enforcing Python code rules, including inline import detection as per python-tdd skill requirements. Extends the Custom Linter Framework 4.3.

  • 4.4.1. Detect inline imports (import statements inside functions or blocks instead of at module level).
    • acceptance: A Python file with an inline import (e.g., import os inside a function) is reported with PYC001 violation.
  • 4.4.2. Report violations with code PYC001 for inline imports.
    • acceptance: A Python file with all imports at module level reports no violations.
  • 4.4.3. Respect .gitignore and read-only zones (personal/, raw/).
    • acceptance: The linter integrates with the Custom Linter Framework for shared functionality.
  • 4.4.4. Support JSON output format via --json flag.
    • acceptance: The linter runs as the pycustomlint check in the sanity runner 4.9.
  • 4.4.5. Provide CLI access via pycustomlint chklint [directory].

4.5. python-ruff (Python Lint)

Status: Implemented

Covers: src/python_ruff/. Wraps the external ruff linter over Python sources with an AI-friendly aggregated violation report in the shared linter format (see 4.9.3).

  • 4.5.1. Violations report file, line:column, and rule code in an AI-friendly aggregated report using the shared linter format 4.9.3.
    • acceptance: A file with a lint violation (e.g. an unused import) is reported with file, position, and rule code in DEFINITIONS, FILE, and SUMMARY sections.
  • 4.5.2. Integrated into the sanity runner 4.9.
  • 4.5.3. The library supports auto-fix: PythonRuff.lint(target, fix=True) passes --fix through to the ruff binary and modifies files in place. Limitation: the capability is not yet exposed as a CLI subcommand (future: python-ruff fix, mirroring 4.1.1).

4.6. Shellcheck Library

Status: Planned

Lint bash/shell scripts in the wiki and project.

  • 4.6.1. NOT_IMPLEMENTED Wrap shellcheck following the same pattern as python_ruff (external linter, AI-friendly aggregated violation report).
    • acceptance: Running against a directory with a shell script containing a known shellcheck violation reports file, line, and rule code.
  • 4.6.2. NOT_IMPLEMENTED Graceful error when shellcheck is not installed on the system.
    • acceptance: Missing binary produces a clear install-hint message, not a stack trace.
  • 4.6.3. NOT_IMPLEMENTED Integrated into wiki sanity 3.1.3.

4.7. MD Code Block Linting

Status: Planned

Lint fenced code blocks inside markdown files for quality.

  • 4.7.1. NOT_IMPLEMENTED Extract fenced code blocks from md files and lint them by language: python blocks via Ruff, shell blocks via shellcheck, others skipped.
    • acceptance: A md file whose python code block has an unused import is reported.
  • 4.7.2. NOT_IMPLEMENTED Report violations with the md file path and the position inside the code block (block index and line within block, plus md line offset).
    • acceptance: Non-code md content produces no false positives.
  • 4.7.3. NOT_IMPLEMENTED Integrated into wiki sanity 3.1.3.

4.8. librarian (Content Management)

Status: Implemented

Covers: src/librarian/ (librarian.py, md_frontmatter). Wiki content operations: file listing, frontmatter parsing, validation, and version tracking.

4.8.1. Non-MD File Support

Status: Planned

Extend frontmatter-style tracking beyond md files.

  • NOT_IMPLEMENTED Python files: TBD — likely version header/docstring-based tracking.
  • NOT_IMPLEMENTED Bash scripts: TBD — likely comment-based version tags.

Acceptance criteria: TBD once the format is decided. Blocked on a design decision; do not implement until the tagging format is specified.


4.9. Test and Sanity Runners

Status: Implemented

Covers: src/run_tests.py (TestRunner) and src/run_sanity.py (parallel pytests with a coverage gate, markdownlint, frontmatter, ruff, pycustomlint, and jsonlint checks; see 4.9.1 to 4.9.6).

4.9.1. Failure Reporting

Status: Implemented

The sanity runner names every failing test and its assert explanation without depending on pytest's terminal formatting.

  • 4.9.1.1. The pytest report in run_sanity.py lists per-file pass/fail counts with each failing test nested under its file entry as Class : test_name, followed by the assert explanation (or the raw exception message when no assert line exists).
    • acceptance: A failing test appears under its file entry with name and assert even when pytest's short summary is suppressed (e.g. PYTEST_ADDOPTS=--no-summary).
  • 4.9.1.2. The Sanity Failed verdict lists only the failing test names; the assert details live in the report body, not the verdict.
    • acceptance: The full suite passes in environments with a suppressed short summary or forced colors (PY_COLORS=1).
  • 4.9.1.3. run_tests.py runs pytest with a temporary JUnit XML report (junit_family=xunit1) and parses failure ids, locations, and messages from it, so details survive terminal formatting quirks (missing short summary, colors, plugin output).
    • acceptance: LOG_LEVEL=DEBUG shows the pytest command, return code, parsed failure details, and the raw pytest output on parse mismatches.
  • 4.9.1.4. Regex-parsed pytest output stays the preferred source for displayed messages; junit entries fill missing fields, replace entries without a real message, and never duplicate entries across path forms (matched by file basename and test name).
  • 4.9.1.5. The pytest command pins --color=no; a missing short summary logs a WARNING naming the likely causes (PYTEST_ADDOPTS, --no-summary, plugin output).
  • 4.9.1.6. Logging is configured from the LOG_LEVEL environment variable (default WARNING); module loggers carry the level explicitly because pymarkdown resets the root logger to WARNING mid-run.

4.9.2. Coverage Gate

Status: Implemented

The sanity runner enforces a minimum test coverage threshold loaded from src/settings_dev.json.

  • 4.9.2.1. run_sanity.py loads pytests.required-coverage from src/settings_dev.json (currently 96%) and fails the pytests check when the total coverage is below it, even if every test passes.
    • acceptance: A run with all tests passing but coverage below the requirement fails sanity.
  • 4.9.2.2. The summary prints a separate coverage: line (PASS/FAIL - <percent>% (required <percent>%)); under Sanity Failed, a coverage shortfall is listed as its own entry.
    • acceptance: Raising required-coverage in settings_dev.json changes the verdict with no code changes.

4.9.3. Linter Report Format

Status: Implemented

Every linter check in the sanity runner (markdownlint, frontmatter_lint, pycustomlint, mdlinter, jsonlint) prints the same AI-friendly report structure so the reports are diffable across linters.

  • 4.9.3.1. The report starts with a <Linter Name> Violations header followed by TARGET: <scanned path>.
    • acceptance: The markdownlint, frontmatter_lint, and pycustomlint reports all carry the same sections in the same order.
  • 4.9.3.2. A - DEFINITIONS: section lists each violated rule as <code>: <rule_name>, <description>.
  • 4.9.3.3. A - FILE: entry per violating file nests one line per violation as <code> <message>.
  • 4.9.3.4. A - SUMMARY: section closes the report with Total files with violations: <count> and Total violations: <count>.
  • 4.9.3.5. Clean runs print no report; the Results: table reports the violation count instead.
    • acceptance: A new linter check reuses the shared format (BaseLinter.format_violations_report, 4.3 ) instead of a bespoke one.

4.9.4. Run Timing

Status: Implemented

The sanity runner reports how long each check and the whole run take.

  • 4.9.4.1. Every check summary line in the Results: table is suffixed with the check's elapsed time as (MM:SS), zero-padded, with minutes allowed to exceed 59 (e.g. ruff: PASS - 0 violations (00:07)).
    • acceptance: A green run prints every summary line with an (MM:SS) suffix and ends with a SANITY PASSED (MM:SS) banner.
  • 4.9.4.2. Durations are wall-clock times of parallel checks, so per-check times can overlap and need not sum to the total.
    • acceptance: A check result without a measured duration still renders (the suffix is optional).
  • 4.9.4.3. The coverage: line stays untimed; it reports on the pytests run.
  • 4.9.4.4. The pass banner title reads SANITY PASSED (MM:SS) with the total wall time of the whole run; the Sanity Failed verdict carries no total time.

4.9.5. Shared Sanity Runner Engine

Status: Implemented

Covers: src/sanity_runner/. The SanityRunner class shared by the repository sanity runner 4.9 and wiki sanity 3.1.3.

  • 4.9.5.1. Concurrent execution: one thread per registered check, per-check wall-clock durations 4.9.4; results keep the registry order regardless of completion order.
    • acceptance: run_sanity and wiki sanity produce their reports through SanityRunner; the engine and format code are not duplicated.
  • 4.9.5.2. Shared output format: one-line check summaries (PASS/FAIL - <detail> (MM:SS)), the aligned Results: rows with an optional extra-row hook (e.g. the coverage line under pytests), the SANITY PASSED (MM:SS) banner (names in registry or sorted order), and the Sanity Failed - N checks need attention: footer with an optional per-check details hook.
    • acceptance: The check reports are byte-identical to the pre-refactor runners; run_sanity's and wiki sanity's in-run progress output follows 4.9.6.
  • 4.9.5.3. Per-check specifics plug in as callables: summary formatters (e.g. pytests "N tests failed", coverage "X% (required Y%)").
  • 4.9.5.4. Progress reporting plugs in as optional run_all callbacks: a progress(name, result) callable invoked as each check completes and a heartbeat(remaining_names) callable invoked roughly every 0.5s while checks are still running. The check lifecycle (start and completion with duration) is logged at INFO through the logging module. run_sanity and wiki sanity 3.1.3 pass their progress display as these callbacks (see 4.9.6).
  • 4.9.5.5. Runner-specific presentation (which reports print and with which headers) stays with each runner.

4.9.6. Progress Display

Status: Implemented

Covers: src/run_sanity.py (ProgressDisplay, format_progress_bar, load_progress_settings, parse_args) and the --non-interactive CLI argument of both run_sanity and wiki sanity 3.1.3, which reuses the same display.

Live per-check progress while the sanity checks run, replacing the [DONE]/[WAITING] heartbeat.

  • 4.9.6.1. Interactive mode (default on a terminal) draws one bar line per check in registry order and redraws the block in place (ANSI cursor-up plus per-line clears) on every engine poll, e.g. pytests [###------------] 22% 6.5s.
    • acceptance: On a terminal each check gets its own live bar line.
  • 4.9.6.2. Bars fill against the check's elapsed time scaled by the expected-seconds setting (default 30); the fraction is capped at 100%.
  • 4.9.6.3. A finished check shows a full bar with its measured duration: markdownlint [###############] 100% done (00:07).
  • 4.9.6.4. The final block stays visible above the reports (nothing is erased), so the terminal keeps a record of the run's progress.
  • 4.9.6.5. --non-interactive — also automatic when stdout is not a terminal — prints the label once (Running Sanity for run_sanity, Running Wiki Sanity for wiki sanity), then one progress dot every interval seconds (default 3) while checks run, e.g. Running Sanity ....; the line ends with a newline before the reports. Runs shorter than the interval print no progress output.
    • acceptance: on a pipe or with --non-interactive the output contains no ANSI escape codes and one progress dot appears per interval seconds after the label.
  • 4.9.6.6. Settings (settings_dev.json, progress section): interval (non-interactive print cadence) and expected-seconds (bar scale); each falls back to its default when the section or key is absent.
  • 4.9.6.7. The shared SanityRunner engine 4.9.5 is unchanged; the display plugs in as the engine's progress and heartbeat callbacks.
  • 4.9.6.8. wiki sanity 3.1.3 plugs the same display into its checks: the bars carry the SANITY_CHECKS registry names and scale against the same settings; --json runs skip the display so the JSON output stays parseable.

4.10. toc-generator (Table of Contents)

Status: Implemented

Covers: src/librarian/toc_generator.py. Generates a table_of_contents.md per folder, listing every md file with its frontmatter version and its headers as anchored links.

  • 4.10.1.1. Each covered folder gets a table_of_contents.md with YAML frontmatter (title, created, updated, type: toc).
    • acceptance: A folder with two md files gets a TOC listing both files with their versions and headers.
  • 4.10.1.2. Entries list folders, file links, and the file's frontmatter version (shown as v<version>; v0.0.0 when missing or unreadable).
    • acceptance: A header inside a fenced code block does not appear in the TOC.
  • 4.10.1.3. The target must be a directory: a file or missing path exits 1 with an error on stderr pointing at librarian toc file <path> for single files. A folder without md files is reported with no markdown files found on stdout (logged at INFO instead in --recursive-toc mode); nothing is written in either case.
    • acceptance: An md file matching an exclude pattern is not listed.
  • 4.10.1.4. Headers inside each file are nested as links with GitHub-style anchors; fenced code blocks are not scanned for headers.
    • acceptance: With max_depth_from_file: 3, a file's #### header does not appear in the TOC while its ### header does; with the key absent or 0, every header level is listed.
  • 4.10.1.5. The TOC file itself, hidden folders, and exclude-pattern matches are skipped; --recursive-toc writes a TOC in every subfolder.
    • acceptance: With max_depth_from_dir: 1, md files two subfolder levels deep are not listed; with the key absent or 0, the whole tree is listed.
  • 4.10.1.6. --non-recursive limits a TOC to the md files directly in the folder (default: recursive file collection within the folder).
  • 4.10.1.7. --exclude REGEX (repeatable) excludes matching files and folders from the TOC and from recursion.
  • 4.10.1.8. Files are read BOM-tolerantly (utf-8-sig) so a leading BOM does not hide the first header.
  • 4.10.1.9. Depth limits are data-driven from the markdown.toc_generator section of fiona-settings.json (loaded from the packaged llm-wiki template, see 4.1.2):
    • max_depth_from_file caps the header level listed per file (e.g. 3 lists h1 through h3; deeper headers are omitted)
    • max_depth_from_dir caps how deep a TOC recurses into subfolders (e.g. 1 lists files in the folder and its direct subfolders)
    • An absent key or 0 means unlimited (today's behavior); any other value must be a non-negative integer, otherwise the command exits 1 with a clear error naming the offending key
    • --non-recursive still forces direct files only regardless of max_depth_from_dir

4.10.2. Per-File TOC Generation (librarian toc file)

Status: Implemented

Covers: src/librarian/toc_generator.py (FileTocConfig, generate_file_toc, generate_file_tocs, find_wiki_root, file_toc_main) and the librarian toc file CLI subcommands in src/fiona/__main__.py. In contrast to the per-folder table_of_contents.md files 4.10 , this writes a TOC inside each md file, directly below its H1 heading.

  • 4.10.2.1. fiona-assistant librarian toc file <path_to_file> generates the TOC for a single file; the standalone form is toc_generator file <path_to_file>.
    • acceptance: A markdown file (*.md) with #, ##, and ### headings gets a TOC below the H1 listing only the ## and ### headings as anchored links.
  • 4.10.2.2. fiona-assistant librarian toc file * generates TOCs for all md files recursively: the root is the llm-wiki root discovered via the nearest fiona-settings.json (current directory first, then its subdirectories — hidden folders and build artifacts (build/, dist/, __pycache__/, *.egg-info/) are skipped so a stale build copy never shadows the real wiki; falls back to the current directory, see 4.11.1).
    • acceptance: Re-running the command on a file that already has a TOC replaces the old TOC instead of appending a second one.
  • 4.10.2.3. All headings except the H1 # are included; the TOC is inserted directly below the H1 (after any frontmatter).
    • acceptance: With sectionize: true, headings are renumbered in the file and the TOC links to them still resolve.
  • 4.10.2.4. The TOC block is delimited by <!-- toc-start: ... --> and <!-- toc-end --> markers. The start marker records the generation params — sectionized (present only when section numbering was on) and the heading range, e.g. <!-- toc-start: sectionized, headings 2..6 --> — and on regeneration the marker params have priority: a sectionized TOC keeps its numbering and the recorded heading range sticks, so settings changes do not rewrite an existing TOC's identity. Resolution order: existing marker params first, then command-line arguments (--sectionized), then fiona-settings.json (incl. glob overrides) as the default for fields no higher source specifies. Bare legacy <!-- toc-start --> markers carry no params and fall through to arguments/settings, and are upgraded on regeneration. An invalid marker range is ignored with a warning. Re-running the command replaces the marked block in place, so an existing TOC is regenerated, never duplicated. The entry list is blank-separated from the markers so generated files pass MD032 (blanks-around-lists). A file whose TOC is already current is reported as unchanged and not rewritten.
    • acceptance: With max_heading: 3, #### headings do not appear in the TOC; with min_heading: 3, ## headings do not appear.
  • 4.10.2.5. --sectionized on toc file <path> / toc file * forces section numbering on for the run, overriding the sectionize setting; the regenerated marker then records sectionized, so the numbering persists for later runs.
    • acceptance: A file matching an ignore_files pattern is not modified by toc file *.
  • 4.10.2.6. Headings inside fenced code blocks are not listed; frontmatter content (e.g. # YAML comment lines) is not mistaken for the H1.
    • acceptance: Running toc file on a file inside personal/ or raw/ reports an error and does not write.
  • 4.10.2.7. Optional section numbering: when sectionize is true, TOC entries carry section numbers (e.g. 1.1.) and the same numbers are written into the actual heading lines (existing numbers are stripped before renumbering), so the TOC links keep resolving.
  • 4.10.2.8. Parameters are configurable via the file section under markdown.toc_generator in fiona-settings.json:
    • min_heading: minimum heading level to include (default: 2; values below 2 are rejected since the H1 is never listed)
    • max_heading: maximum heading level to include (default: 6; must not be below min_heading)
    • sectionize: bool; add section numbers to TOC entries and to the headings themselves (default: false)
    • ignore_files: list of glob patterns of files to skip
    • Any key whose value is a mapping is a per-file override keyed by an fnmatch glob matched against the file's path relative to the wiki root (e.g. "*/rules.md": {"max_heading": 2}).
    • Invalid keys or values exit 1 with a clear error naming the offending key (unlike pymarkdown rule typos, see 4.1.2).
  • 4.10.2.9. toc file * skips files inside read-only zones (personal/, raw/), hidden files and folders, generated table_of_contents.md files, and ignore_files matches; running toc file on an explicit path inside a read-only zone (or on a missing or non-md path) exits 1 with a clear error instead of writing.
  • 4.10.2.10. Output is AI-friendly: one status line per file (wrote toc in . ., toc already up to date in ..., skipped ... (<reason>)), plus a generated toc for N file(s), skipped M summary in bulk mode. A file without an H1 is skipped with skipped ... (no H1 heading) and left unmodified.

4.10.3. MD Frontmatter Tagging

Status: Partially Implemented

Tag md files with metadata used for version control and quality signals; the version tag feeds the TOC entries above.

Tag Status Purpose
version Implemented Source control, wiki update change detection, TOC display
date Planned Creation/last-update timestamps maintained by tooling
confidence Planned Quality signal (e.g. high/medium/low) on concept notes
contested Planned Boolean flag marking notes with disputed claims
  • 4.10.3.1. Frontmatter is validated by the md_frontmatter module (see 4.8).
    • acceptance: A concept note missing confidence is flagged by wiki sanity 3.1.3.
  • 4.10.3.2. Tags follow the canonical frontmatter schema (see 4.2.1): name and version are required, title and all other fields are optional.
  • 4.10.3.3. wiki sanity 3.1.3 warns on files missing required tags.

4.11. fiona (CLI Entry Point)

Status: Implemented

Covers: src/fiona/__main__.py. The fiona-assistant console script dispatches to the tool modules so every tool is reachable through one command. Each tool also keeps its standalone CLI; librarian and mdoctor are installed as console scripts, while python-ruff is available only through fiona-assistant. fmlint, json-lint, and the git subcommands are also only through fiona-assistant. Dispatch modules fiona, python_ruff, customlinter, wiki, and gittool ship in the wheel.

Command tree:

Command Behavior
fiona-assistant librarian [directory] List files via librarian
4.8
fiona-assistant librarian list [directory] Same as above
fiona-assistant librarian toc [directory] Generate a TOC
4.10
fiona-assistant librarian toc file <path_to_file> Generate an in-file TOC
4.10.2
fiona-assistant librarian toc file * Generate in-file TOCs recursively
4.10.2
fiona-assistant mdoctor [args] Delegate to the mdoctor CLI; defaults to
chklint .
fiona-assistant fmlint [args] Frontmatter lint; defaults to chklint .
4.2
fiona-assistant python-ruff [args] Delegate to the python-ruff CLI;
defaults to chklint .
fiona-assistant json-lint [args] JSON lint; defaults to chklint .
4.13
fiona-assistant git save/snapshot/restore/clean/undo/status Wiki git
operations 3.2
fiona-assistant wiki init [directory] Initialize an llm-wiki
3.1.1
fiona-assistant wiki check-update [directory] Check for core-file updates
3.1.2
fiona-assistant wiki update [directory] Update core files
3.1.2
fiona-assistant wiki sanity [directory] Run all sanity checks
3.1.3
fiona-assistant wiki graph [directory] Write the link graph
3.1.4
  • 4.11.1. With no subcommand, print help and exit 0.
    • acceptance: Each subcommand behaves identically to the corresponding standalone CLI (covered by src/tests/test_fiona_main.py).
  • 4.11.2. Delegation rewrites sys.argv and calls the target tool's argparse CLI, so behavior matches the standalone tools.

4.11.1. Wiki Root Discovery (fiona-settings.json)

Status: Planned

fiona-settings.json marks the root of an llm-wiki. When it is present, commands should target the wiki root instead of the current directory.

Current behavior: every command's optional [directory] argument defaults to the current directory; the dispatcher does not consult fiona-settings.json when picking a target.

  • 4.11.1.1. NOT_IMPLEMENTED When fiona-assistant is invoked, locate the nearest fiona-settings.json: first in the current directory, then in its subdirectories.
    • acceptance: Running fiona-assistant mdoctor from inside a wiki subdirectory lints the wiki root, not the subdirectory.
  • 4.11.1.2. NOT_IMPLEMENTED The directory containing that file is the llm-wiki root and becomes the default directory for every command that accepts an optional [directory] (librarian list/toc, mdoctor list/chklint/fix, python-ruff chklint, wiki init).
    • acceptance: Running fiona-assistant librarian inside a wiki lists the wiki root's files.
  • 4.11.1.3. NOT_IMPLEMENTED An explicit [directory] argument always overrides the discovered root.
  • 4.11.1.4. NOT_IMPLEMENTED When no fiona-settings.json is found, commands keep the current directory as the default (today's behavior).
  • 4.11.1.5. NOT_IMPLEMENTED The settings file is loaded once per invocation and shared by all tools; mdoctor already consumes it for lint rules (see 4.1.2).
    • acceptance: With no fiona-settings.json in or below the current directory, behavior is unchanged from today.

4.12. html-js-lint (HTML and JavaScript Linter)

Status: Planned

pyproject.toml still lists package html_js_lint and a html-js-lint console script. That directory does not exist. Remove both entries until this section is implemented.

Covers: src/html_js_lint/. An HTML and JavaScript linter that validates HTML syntax and structure, and JavaScript code quality. Uses existing Python libraries where available: html5validator for HTML validation (W3C standard, requires Java 8+ runtime), and eslint via subprocess for JavaScript linting (requires Node.js and ESLint installation).

4.12.1. HTML Validation

Status: Planned

Validate HTML files for syntax correctness and standards compliance.

  • 4.12.1.1. NOT_IMPLEMENTED Use html5validator library for W3C-compliant HTML5 validation.
    • acceptance: An HTML file with syntax errors (e.g., unclosed tags, invalid nesting) is reported with file path, line number, and error description.
  • 4.12.1.2. NOT_IMPLEMENTED Check that all HTML files in the target directory tree are valid.
    • acceptance: An HTML file with valid syntax reports no errors.
  • 4.12.1.3. NOT_IMPLEMENTED Report HTML validation errors with file path, line number, and column position.
    • acceptance: Running on a directory with no HTML files produces no errors.
  • 4.12.1.4. NOT_IMPLEMENTED Handle files with .html and .htm extensions by default; support configurable extensions via settings.
  • 4.12.1.5. NOT_IMPLEMENTED Respect .gitignore and read-only zones (personal/, raw/).
  • 4.12.1.6. NOT_IMPLEMENTED Detect and validate inline JavaScript in <script> tags.
    • acceptance: HTML files with inline JavaScript are parsed and the JS code is validated for syntax errors.

4.12.2. JavaScript Linting

Status: Planned

Lint JavaScript files for syntax errors, style violations, and code quality issues.

  • 4.12.2.1. NOT_IMPLEMENTED Use eslint via subprocess call for JavaScript linting.
    • acceptance: A JavaScript file with a syntax error is reported with file path, line number, and error description.
  • 4.12.2.2. NOT_IMPLEMENTED Support configurable ESLint rules via .eslintrc configuration.
    • acceptance: Custom rules in .eslintrc are respected when linting.
  • 4.12.2.3. NOT_IMPLEMENTED Check that all JavaScript files in the target directory tree are valid.
    • acceptance: A JavaScript file with valid syntax but no style violations reports no errors.
  • 4.12.2.4. NOT_IMPLEMENTED Handle files with .js and .javascript extensions by default; support configurable extensions via settings.
  • 4.12.2.5. NOT_IMPLEMENTED Respect .gitignore and read-only zones (personal/, raw/).
  • 4.12.2.6. NOT_IMPLEMENTED Optionally extract and lint inline JavaScript from HTML <script> tags using ESLint.
    • acceptance: Inline JavaScript in HTML files is extracted and linted when enabled, with violations reported with the HTML file path and line number.

4.12.3. CLI Integration

Status: Planned

Provide CLI access to the HTML and JavaScript linter via fiona-assistant.

  • 4.12.3.1. NOT_IMPLEMENTED fiona-assistant html-js-lint chklint [directory] scans for HTML and JavaScript violations.
    • acceptance: Running fiona-assistant html-js-lint chklint on a directory with invalid HTML or JS produces a structured violation report.
  • 4.12.3.2. NOT_IMPLEMENTED fiona-assistant html-js-lint with no subcommand defaults to chklint ..
    • acceptance: Running on a clean directory reports no violations.
  • 4.12.3.3. NOT_IMPLEMENTED Output format matches other linters: AI-friendly violation report with file paths, line numbers, column numbers, and rule codes.
    • acceptance: Output includes both HTML and JS violations in consistent format.
  • 4.12.3.4. NOT_IMPLEMENTED Exit code 0 even with violations (consistent with other linters).
  • 4.12.3.5. NOT_IMPLEMENTED Support --fix flag to auto-fix certain violations where supported by the underlying linters.
    • acceptance: Running with --fix on a file with fixable violations corrects the file and reports the number of fixed files.

4.12.4. Configuration

Status: Planned

Load linter configuration from fiona-settings.json.

  • 4.12.4.1. NOT_IMPLEMENTED HTML validation configuration in fiona-settings.json:
    • html.extensions: list of file extensions to check (default: ["html", "htm"])
    • html.ignore_patterns: list of glob patterns to ignore
    • html.lint_inline_js: bool, whether to lint inline JavaScript in <script> tags (default: false)
    • acceptance: Custom HTML extensions are respected during scanning.
  • 4.12.4.2. NOT_IMPLEMENTED JavaScript linting configuration in fiona-settings.json:
    • javascript.extensions: list of file extensions to check (default: ["js", "javascript"])
    • javascript.eslint_config: path to ESLint configuration file (default: .eslintrc)
    • javascript.ignore_patterns: list of glob patterns to ignore
    • acceptance: Custom JS extensions are respected during scanning.
  • 4.12.4.3. NOT_IMPLEMENTED A missing html_js_lint section in settings uses sensible defaults.

4.12.5. Sanity Integration

Status: Planned

Integrate the HTML and JavaScript linter into the sanity checker.

  • 4.12.5.1. NOT_IMPLEMENTED run_sanity.py runs html-js-lint chklint on the codebase.
    • acceptance: An HTML or JS file with violations causes run_sanity.py to report HTML/JS lint failures.
  • 4.12.5.2. NOT_IMPLEMENTED wiki sanity 3.1.3 runs html-js-lint chklint on the wiki directory.
    • acceptance: wiki sanity 3.1.3 reports HTML/JS violations alongside other linter violations.
  • 4.12.5.3. NOT_IMPLEMENTED HTML/JS lint violations are reported in the aggregated sanity output.

4.13. json-lint (JSON Linter)

Status: Implemented

Covers: src/customlinter/jsonlint.py. A JSON linter and checker that validates JSON syntax and applies configurable style rules. Uses existing Python libraries (json module for parsing, potentially json5 or similar for extended syntax support).

4.13.1. JSON Syntax Validation

Status: Implemented

Validate that JSON files have correct syntax and are parseable.

  • 4.13.1.1. Use Python's standard json module for syntax validation.
    • acceptance: A JSON file with a syntax error (e.g., missing comma, unquoted key) is reported with file path and error location.
  • 4.13.1.2. Check that all JSON files in the target directory tree are valid JSON.
    • acceptance: A JSON file with valid syntax but no style violations reports no errors.
  • 4.13.1.3. Report syntax errors with file path, line number, and column position.
    • acceptance: Running on a directory with no JSON files produces no errors.
  • 4.13.1.4. Handle files with .json extension by default; support configurable extensions via settings.
  • 4.13.1.5. Respect .gitignore and read-only zones (personal/, raw/).

4.13.2. JSON Style Rules

Status: Implemented

Apply configurable style rules to JSON files.

  • 4.13.2.1. Check indentation: tabs should be converted to spaces, with configurable indent length (default: 2 spaces per the project standard).
    • acceptance: A JSON file with tab indentation when indent: "space" and indent_len: 2 is configured reports a style violation.
  • 4.13.2.2. Check trailing commas (optional based on settings).
  • 4.13.2.3. Check key ordering (optional based on settings).
    • acceptance: A JSON file with 4-space indentation when indent_len: 2 is configured reports a style violation.
  • 4.13.2.4. Check for duplicate keys in objects.
    • acceptance: A JSON file with duplicate keys is reported as a violation.
  • 4.13.2.5. Load style rules from the json section of fiona-settings.json:
    • indent: "space" or "tab" (default: "space")
    • indent_len: number of spaces for indentation (default: 2)
    • trailing_commas: bool, whether to allow trailing commas (default: false)
    • require_key_order: bool, whether object keys must be sorted (default: false)
    • Additional rules can be added without code changes via configuration.
  • 4.13.2.6. A missing json section in settings uses sensible defaults.

4.13.3. CLI Integration

Status: Implemented

Provide CLI access to the JSON linter via fiona-assistant.

  • 4.13.3.1. fiona-assistant json-lint chklint [directory] scans for JSON violations.
    • acceptance: Running fiona-assistant json-lint on a directory with invalid JSON produces a structured violation report.
  • 4.13.3.2. fiona-assistant json-lint with no subcommand defaults to chklint ..
    • acceptance: Running on a clean directory reports no violations.
  • 4.13.3.3. Output format matches other linters: AI-friendly violation report with file paths, line numbers, column numbers, and rule codes.
  • 4.13.3.4. Exit code 0 even with violations (consistent with other linters).
  • 4.13.3.5. Support --fix flag to auto-fix certain violations (indentation, trailing commas) in place.
    • acceptance: Running with --fix on a file with fixable violations corrects the file and reports the number of fixed files.

4.13.4. Sanity Integration

Status: Implemented

Integrate the JSON linter into the sanity checker.

  • 4.13.4.1. run_sanity.py runs json-lint chklint on the codebase.
    • acceptance: A JSON file with violations causes run_sanity.py to report JSON lint failures.
  • 4.13.4.2. wiki sanity 3.1.3 runs json-lint chklint on the wiki directory.
    • acceptance: wiki sanity 3.1.3 reports JSON violations alongside other linter violations.
  • 4.13.4.3. JSON lint violations are reported in the aggregated sanity output.

4.14. settings_manager (Settings Management)

Status: Planned

Covers: src/fiona/settings_manager.py. Provides interactive and non-interactive settings management for fiona-assistant.

4.14.1. Interactive Setup (fiona-assistant setup)

Status: Planned

Run an interactive setup utility to configure fiona-assistant settings.

  • 4.14.1.1. NOT_IMPLEMENTED Present all configurable settings from fiona-settings.json to the user
    • acceptance: Running fiona-assistant setup with no arguments starts the interactive flow
  • 4.14.1.2. NOT_IMPLEMENTED Allow user to enable/disable boolean settings by checking or unchecking
    • acceptance: User can navigate through all settings with Enter to accept defaults
  • 4.14.1.3. NOT_IMPLEMENTED Allow user to add custom values for string/integer settings where applicable
    • acceptance: User can toggle boolean settings (check/uncheck)
  • 4.14.1.4. NOT_IMPLEMENTED Provide a complete path where user can press Enter through all prompts to accept all defaults without modification
    • acceptance: User can input custom values for string/integer settings
  • 4.14.1.5. NOT_IMPLEMENTED Display current values for each setting before prompting
  • 4.14.1.6. NOT_IMPLEMENTED Support non-interactive mode via flags (e.g., --yes, --default) for AI agents to run deterministically
    • acceptance: All inputs are validated before saving to settings file
  • 4.14.1.7. NOT_IMPLEMENTED Validate all inputs before saving
    • acceptance: Non-interactive mode applies all defaults without prompts
  • 4.14.1.8. NOT_IMPLEMENTED Write validated settings to fiona-settings.json in the wiki root
    • acceptance: Exit code 0 on successful completion, non-zero on validation error or failure
  • 4.14.1.9. NOT_IMPLEMENTED Handle missing or corrupted settings files gracefully
    • acceptance: Running fiona-assistant setup in a directory without fiona-settings.json creates one with defaults

4.14.2. Contrast with fiona-assistant init

Status: Planned

Clarify the difference between init and setup commands.

  • 4.14.2.1. NOT_IMPLEMENTED fiona-assistant init [directory]: Initializes an llm-wiki directory with default settings applied automatically without user interaction (see 3.1.1)
    • acceptance: Documentation clearly distinguishes the two commands and their use cases
  • 4.14.2.2. NOT_IMPLEMENTED fiona-assistant setup: Runs interactive review of all possible settings, allowing user to customize each setting before applying
    • acceptance: init remains non-interactive and applies all defaults automatically
  • 4.14.2.3. NOT_IMPLEMENTED init is designed for quick bootstrap with sensible defaults
    • acceptance: setup provides full interactive review of all configurable settings
  • 4.14.2.4. NOT_IMPLEMENTED setup is designed for users who want to review and customize settings before initialization or update existing settings
    • acceptance: setup can be run on an existing wiki to modify settings
  • 4.14.2.5. NOT_IMPLEMENTED init creates the wiki structure; setup configures settings in an existing wiki
  • 4.14.2.6. NOT_IMPLEMENTED Both commands respect the wiki root discovery mechanism 4.11.1

5. Project Infrastructure

5.1. Logging Library

Status: Partially Implemented

Replace print-based output with a logging library across the codebase.

Progress note: run_sanity.py and run_tests.py log diagnostics via Replace print-based output with a logging library across the codebase using the standard logging module, with levels controlled by LOG_LEVEL (see 4.9.1 ); CLI-facing reports intentionally remain stdout prints. Every CLI entry point (fiona, librarian, librarian toc, librarian toc file, mdoctor, fmlint, python-ruff, wiki check-update, wiki update) configures logging through the shared fiona.logging_config.setup_logging() module, which resolves the level from (highest precedence first) the LOG_LEVEL environment variable, fiona.log_level in fiona-settings.json (e.g. INFO), and WARNING as the fallback. Log records go to stderr; CLI-facing reports stay stdout prints. Tool package loggers (fiona, librarian, mdoctor, python_ruff, wiki) carry the level explicitly because pymarkdown resets the root logger mid-run (same pattern as 4.7). Physical-file logging: fiona.file_log_location in fiona-settings.json attaches a managed file handler (timestamped lines, %(asctime)s %(levelname)s %(name)s: %(message)s) writing every record at or above fiona.file_log_level (defaults to fiona.log_level when absent or invalid). Relative locations resolve against the current working directory; dropping file_log_location disables file logging; an unopenable location logs a warning and keeps console logging.

  • 5.1.1. The standard logging module is the diagnostics channel, configured by fiona.logging_config.setup_logging(). No new hard dependency.
    • acceptance: CLI entry points call setup_logging().
  • 5.1.2. NOT_IMPLEMENTED Error and warning diagnostics go to stderr through the logging module. CLI-facing reports stay stdout prints.
    • acceptance: Production modules do not print errors. Reports may still print.
  • 5.1.3. Existing AI-friendly report format is preserved for consumers.
    • acceptance: Tool output consumed by tests and scripts is unchanged.

5.2. Documentation (medium priority)

Status: Partially Implemented

Progress note: src/README.md, src/librarian/README.md, and src/mdoctor/README.md exist with a project description, pip install -e . setup instructions, and basic usage examples; all three are stale relative to the current CLIs.

  • 5.2.1. NOT_IMPLEMENTED Document the actual subcommands in librarian/README.md (list, toc) and mdoctor/README.md (list, chklint, fix) — both READMEs predate those subcommands.
    • acceptance: A new contributor can install, run, and use each tool using only READMEs.
  • 5.2.2. NOT_IMPLEMENTED Refresh the src/README.md project structure (omits wiki/, python_ruff/, run_tests.py, run_sanity.py, settings_dev.json).
    • acceptance: All markdown documentation passes mdoctor chklint.
  • 5.2.3. The root README.md links this spec under related documentation.
    • acceptance: README.md contains a markdown link to SPEC.md.

5.3. Project Clarification (low priority)

Status: Planned

Define the role of each src/ tool in the wiki workflow.

  • 5.3.1. NOT_IMPLEMENTED Document the purpose of every tool under src/.
  • 5.3.2. NOT_IMPLEMENTED Classify tools by role:
    • Ingestion: moving files from ingest/ to raw/.
    • Processing: converting raw/ to wiki/concepts/.
    • Maintenance: updating index.md / log.md.
    • acceptance: The classification table exists in this spec or the README and every src/ tool has exactly one assigned role.
  • 5.3.3. NOT_IMPLEMENTED Where gaps exist, add utility scripts to llm-wiki utils/ (e.g. ingest.py, process.py).

6. Non-Goals

  • No LLM/API integration: Fiona-Assistant is deterministic tooling only.
  • No modification of personal/ or raw/ content by any automated tool.
  • No cloud sync or remote backup; version control is local-first.

7. Priority Order

Priority Feature Section Status
P1 wiki update / wiki check-update 3.1.2 Implemented
P1 Git wrapper (save/snapshot/restore/clean/undo/status) 3.2 Implemented
P1 wiki sanity incl. dead-link check 3.1.3 Implemented
P1 wiki graph 3.1.4 Implemented
P2 Frontmatter linter (fmlint) 4.2 Implemented
P2 JSON linter (json-lint) 4.13 Implemented
P2 MDoctor auto-fix (mdoctor fix) 4.1.1 Implemented
P2 Lint configuration (fiona-settings.json) 4.1.2 Implemented
P2 Sanity runner failure reporting 4.9.1 Implemented
P2 Shellcheck library 4.6 Planned
P2 MD code block linting 4.7 Planned
P2 HTML/JS linter (html-js-lint) 4.12 Planned
P2 Logging library 5.1 Partial
P2 Wiki root discovery via fiona-settings.json 4.11.1 Planned
P2 Settings manager (fiona-assistant setup) 4.14 Planned
P3 Frontmatter tags (date, confidence, contested) 4.10.3 Planned
P3 Backup script 3.2.1 Planned
P3 Documentation 5.2 Partial
P4 Non-MD file tracking (py/bash) 4.8.1 Planned
P4 Project clarification 5.3 Planned

8. Open Questions

  • Tagging format for python files and bash scripts 4.8.1 is undecided.
  • Git wrapper location: implemented at src/gittool/git_tool.py.
  • Wiki root discovery 4.11.1 : which fiona-settings.json wins when several subdirectories contain one, and should ancestor directories be searched as a fallback before falling back to the current directory?
    • the one nearest wins, or throw an error if the subdir is same level