Files
fiona-assistant/todo.md
T

38 KiB

Fiona Project TODO List

Ordered by priority. Each item references its section in SPEC.md (the source of truth for requirements and acceptance criteria).

Done

  • Include the llm-wiki template as part of the PyPI package
  • MD frontmatter parser for version control of core files (SPEC 6.2)
    • version tag support
  • fiona-assistant wiki init — copy packaged template into a target directory (basic flow; hardening tracked under P1)
  • Windows PowerShell bootstrap (bootstrap.ps1): repo-local venv under .venvs\<name>, same flags as the bash bootstrap

P0 — Packaging and Structure Fixes

From the 2026-09-23 structure audit, plus the 2026-09-29 layout audit below. These are repo-hygiene items that block a correct install or make the docs contradict the code.

  • Fix PyPI packaging so a non-editable install works (wheel build confirmed python_ruff is missing entirely)
    • Add python_ruff to the packages.find include list in pyproject.toml
    • Add src/python_ruff/__init__.py
    • Add src/fiona/__init__.py
    • Re-verify: build a wheel, install it fresh, confirm the fiona-assistant console script runs and the llm-wiki template data ships
  • Resolve the duplicated raw/ zone in the llm-wiki template
    • Pick one canonical location (llm-wiki/raw/ at the wiki root, per user decision) and remove src/llm-wiki/wiki/raw/
    • Align root AGENTS.md, src/llm-wiki/AGENTS.md, and wiki/schema.md on the same location
  • Reconcile the two competing frontmatter schemas
    • Decide between project agent rule 10.1 (name/version/title) and wiki/schema.md (title/sources/tags); one schema wins: name/version required, title optional (user decision, 2026-09-25; documented in SPEC 6.2)
    • Add missing type values (e.g. toc) to the allowed types list (project agent rule 10.1, wiki/schema.md, SPEC 6.2)
    • Stamp frontmatter on the template md files that lack it; only personal/README.md and raw/README.md were missing frontmatter (all other template files already carry name/version; title is optional, so wiki/index.md, wiki/log.md, and the other READMEs already conform)
  • Update the root AGENTS.md structure diagram to match reality: llm-wiki lives at src/llm-wiki/ (packaged template), and src/ also contains wiki/, run_tests.py, run_sanity.py, settings_dev.json, utils.sh, install_packages.sh
  • Raise test coverage from 87% to the required 90% (settings_dev.json) so run_sanity.py can pass
  • Document the src/llm-wiki/__init__.py setuptools hack (hyphenated, non-importable data-only "package") in pyproject.toml comments
  • Remove src/llm-wiki/utils/scripts/example.py — won't do: wiki init no longer skips .py files (it now makes a 1:1 copy of the template, so example.py ships to user wikis; user decision, 2026-09-27; SPEC 7.1, version 0.30.0)
  • Deduplicate conftest: src/conftest.py vs src/tests/conftest.py
    • Consolidated into src/tests/conftest.py (sys.path bootstrap moved there, outer file removed, 2026-09-27); verified python src/run_tests.py (619/619), python -m pytest without PYTHONPATH, and python src/run_sanity.py all pass — the 6055ac6 revert was a misdiagnosis: the then-failing sanity was the terminal-width issue below, not the conftest change
  • run_sanity.py run fails, need fixing
    • run sanity should show what exactly to fix
      • pytests should show which test are failing,
      • markdownlint should show which files are in violation , what line
      • same thing for ruff
  • Remove the phantom html_js_lint package and html-js-lint console script from pyproject.toml until SPEC 7.17 is implemented. The directory src/html_js_lint/ does not exist. Detail is item 2 in the 2026-09-29 audit below

2026-09-29 structure audit

pyproject.toml declares a PEP 621 project and the code lives under src/, but the tree is nine sibling import packages plus loose scripts. A src layout is meant to install one import package. Target shape when this cluster is done:

src/fiona/
├── __main__.py
├── librarian/
├── mdoctor/
├── customlinter/
├── python_ruff/
├── gittool/
├── sanity_runner/
├── wiki/
└── data/llm_wiki/     # templates, no fake package
tests/                 # repo root, not installed

Keep a single required entry point: fiona-assistant = fiona.__main__:main. Optional thin scripts can stay as extra entry points that call into those submodules. Do this as one change set: import paths, tests, entry points, and docs move together. Half-moving a package leaves both layouts on sys.path.

  • Collapse the nine top-level packages into fiona subpackages. Imports, pyproject.toml, tests, and docs moved with the directories.
    • Current install names, from packages.find include and from src/fiona_assistant.egg-info/top_level.txt: customlinter, fiona, gittool, librarian, llm-wiki, mdoctor, python_ruff, sanity_runner, wiki
    • fiona/__main__.py imports them as top-level modules (from librarian.librarian import main, from wiki.wiki import wiki_init, and the same for customlinter, gittool, mdoctor, python_ruff)
    • Same-name module pattern to preserve while moving: librarian/librarian.py, mdoctor/mdoctor.py, python_ruff/python_ruff.py, sanity_runner/sanity_runner.py. gittool is the odd one: the module is git_tool.py
    • wiki is a generic top-level name and will collide with any other installed project that also ships a wiki package
    • After the move, packages.find include should list fiona only. An allow-list is how python_ruff was shipped late and how html_js_lint stayed declared after the directory vanished
  • Remove the phantom html_js_lint declaration (same checkbox as the item above this section)
    • [project.scripts] maps html-js-lint to html_js_lint.html_js_lint:main
    • packages.find include lists html_js_lint
    • src/fiona_assistant.egg-info/entry_points.txt still has that script from the last editable install
    • src/html_js_lint/ does not exist. Invoking html-js-lint fails at import. SPEC 7.17 is the future feature; the entry point should not ship until that directory exists
    • html5validator is in runtime dependencies and is not imported anywhere under src/. Drop it with the phantom script, or keep it only when SPEC 7.17 lands
  • Delete src/__init__.py
    • The file's only content is the comment # Fiona tools package
    • An __init__.py in src/ makes src itself a package whenever the repo root is on sys.path. The src layout exists so packages are importable only after install
    • That is why bytecode for run_sanity.py and run_tests.py lands in src/__pycache__/ (run_sanity.cpython-313.pyc, run_tests.cpython-313.pyc, __init__.cpython-313.pyc)
  • Stop tests from editing sys.path
    • src/tests/conftest.py does sys.path.insert(0, os.path.dirname(os.path.dirname(...))), which puts src/ on the path, then imports librarian, mdoctor, python_ruff, and wiki as top-level names
    • src/tests/__init__.py exists, so tests is a package sitting inside the source root. Standard layout is a repo-root tests/ directory that is not installed
    • [tool.pytest.ini_options] sets testpaths = ["src/tests"] and has no pythonpath. Coverage source = ["src"] treats the mixed script-and-package directory as the measured tree
    • Replacement: pip install -e . in the dev env, or pythonpath = ["src"] once there is a single fiona package. The path edit hides "package not installed" failures
    • The 2026-09-27 conftest consolidation (outer src/conftest.py removed) stays valid. This item replaces the remaining sys.path bootstrap, it does not undo that merge
  • Decide where the loose src/ files live
    • src/run_sanity.py and src/run_tests.py are modules, not packages. They are absent from packages.find, so a wheel does not install them. They are how this repo runs its own sanity and tests (python src/run_sanity.py)
    • src/install_packages.sh, src/utils.sh, and src/settings_dev.json sit in the same directory. Dev config and shell helpers are not package source
    • Either give run_sanity / run_tests real entry points under fiona, or move them to a repo-root scripts/ (or tools/) directory that is not on the package path
  • Replace the hyphenated llm-wiki package with package data
    • import llm-wiki is a SyntaxError. src/llm-wiki/__init__.py exists only so setuptools detects the directory. The pyproject.toml comment already documents that hack
    • [tool.setuptools.package-data] maps "llm-wiki" = ["*", "**/*"]
    • src/llm-wiki/__init__.py sets __version__ = "0.1.0" while [project] version is 0.36.0. Two versions for one distribution
    • Move the template tree to something importable, for example src/fiona/data/llm_wiki/, and load files with importlib.resources. Update wiki init / wiki update path helpers (get_llm_wiki_source_path) in the same change
    • Root README.md links ./llm-wiki/AGENTS.md. The file is src/llm-wiki/AGENTS.md
  • Get setuptools metadata out of src/
    • src/fiona_assistant.egg-info/ holds PKG-INFO, entry_points.txt, requires.txt, SOURCES.txt, top_level.txt. That is install metadata. It belongs at the repo root (or nowhere, if it is only a local editable-install artifact)
    • .gitignore already ignores *.egg-info/, so this directory should not be committed, but where = ["src"] can still see it during package discovery
    • The egg name fiona_assistant is the setuptools normalization of project name fiona-assistant. Leave the normalization alone; only the directory location is wrong
  • Keep a single coverage config, and point it at the code that exists
    • Both .coveragerc and [tool.coverage] in pyproject.toml exist. Coverage reads .coveragerc first, so the pyproject block is unused
    • .coveragerc still omits src/mdoctor/fmlint.py and src/python_ruff/pycustomlint.py. Those modules live under src/customlinter/ (fmlint.py, pycustomlint.py)
    • Both files share the same exclude_lines list. After deleting .coveragerc, confirm pytest --cov still honors the pyproject source, omit, and exclude_lines
    • coverage.xml at the repo root is a generated report. It is not matched by .gitignore (*.log and .coverage are; coverage.xml is not)
  • Raise the setuptools build pin so [project] is valid
    • [build-system] requires is setuptools>=42
    • The [project] table (PEP 621) needs setuptools 61 or newer. An older setuptools that satisfies >=42 can fail the build
    • Set requires = ["setuptools>=61"] (or a current floor) and rebuild the wheel
  • Sort runtime dependencies from repo-only tools
    • ruff is imported by src/python_ruff/python_ruff.py, so it stays in dependencies
    • pylint is in dependencies with no version pin, and nothing under src/ imports it. Move it to the dev extra or drop it
    • There is no [tool.ruff] table and no ruff.toml. If this repo is linted with Ruff, the rule set belongs in pyproject.toml
    • requires-python is >=3.8. Local bytecode in src/__pycache__/ is CPython 3.13. Classifiers only say Programming Language :: Python :: 3. The bootstrap examples use 3.11. Record the versions this repo actually runs, and confirm syntax such as str | None is either covered by from __future__ import annotations or by a higher requires-python
  • Make the structure docs match the tree (also listed under P3)
    • src/README.md "Project Structure" lists only fiona/, librarian/, mdoctor/, tests/, and README.md. Missing from that diagram: customlinter/, python_ruff/, gittool/, sanity_runner/, wiki/, llm-wiki/, run_sanity.py, run_tests.py, settings_dev.json, utils.sh, install_packages.sh
    • MDoctor's description in that file is still [Description to be added], and Contributing is still [Contribution guidelines to be added]
    • Refresh this diagram again after the package collapse, so the docs describe src/fiona/ rather than today's sibling packages

Code defects — 2026-09-29 audit

Source review after the package directories moved under src/fiona/. src/ on disk is fiona/ plus README.md. Tests live in tests/. The template lives in src/fiona/data/llm_wiki/. The package-move items below are done. The command defects after them are still open.

Finish the package move

  • Point every import at fiona.<package>
    • src/fiona/__main__.py still imports top-level librarian, mdoctor, customlinter, python_ruff, gittool, and wiki. Those packages are src/fiona/<name>/.
    • tests/test_fiona_main.py still does from librarian import toc_generator.
    • TOOL_LOGGER_NAMES in src/fiona/logging_config.py still lists librarian, mdoctor, python_ruff, and wiki. Loggers are now fiona.* via getLogger(__name__). The parent name fiona already covers them; drop the stale names.
  • Point template and settings paths at fiona/data/llm_wiki
    • get_llm_wiki_source_path in src/fiona/wiki/wiki.py and src/fiona/wiki/update_manager.py looks for a sibling llm-wiki (src/fiona/llm-wiki). The template is src/fiona/data/llm_wiki/. wiki init and wiki update miss it. Load it with importlib.resources (same end state as the P0 llm-wiki package-data item).
    • The default settings path in logging_config.py, customlinter/base.py, mdoctor/mdoctor.py, and librarian/toc_generator.py is parent.parent / "llm-wiki" / "fiona-settings.json", which is src/llm-wiki/.... The file is src/fiona/data/llm_wiki/fiona-settings.json.
    • src/fiona/run_sanity.py still lints ./src/llm-wiki for frontmatter. Point it at src/fiona/data/llm_wiki.
  • Match packaging and the test runner to the new tree
    • pyproject.toml packages.find include and [project.scripts] still name librarian, mdoctor, python_ruff, customlinter, sanity_runner, llm-wiki, wiki, gittool, and html_js_lint. librarian and mdoctor console scripts import modules that are no longer top-level.
    • include = ["fiona"] does not match subpackages. Use fiona* so fiona.wiki and the rest are installed.
    • [tool.pytest.ini_options] testpaths is still src/tests. The suite is tests/.
    • tests/conftest.py walks two directories up and inserts the repo root. That math used to reach src/ when the file lived at src/tests/conftest.py. Insert src/ so import fiona works without an editable install.
    • src/__init__.py is already gone. Leave it gone.
    • run_tests.py and run_sanity.py already sit in src/fiona/. Give them entry points or document python -m fiona.run_tests / python -m fiona.run_sanity in place of python src/run_tests.py (SPEC 4.13).

Fixes in commands the spec already calls Implemented

  • json-lint fix walks the file list twice (SPEC appendix, SPEC 4.6). The second fix_file pass finds the files already clean, so the path lines never print. Walk once: one count and one path line per fixed file.
  • json-lint chklint --fix is parsed and ignored (src/fiona/customlinter/jsonlint.py). SPEC 4.6 says chklint --fix rewrites the same way fix does.
  • json-lint fix treats any path part named personal or raw as a read-only zone, including a folder outside the wiki. Skip with the wiki-relative check in BaseLinter (src/fiona/customlinter/base.py), the same rule as chklint.
  • git failures print Error: on stdout and catch Exception (src/fiona/gittool/git_tool.py). SPEC 4.7 and section 2: the message goes to stderr. Catch the errors GitTool raises. This is the concrete case of the P2 logging item.
  • wiki graph prints Error: Directory not found on stdout (src/fiona/wiki/graph_generator.py). The other graph errors use stderr. SPEC section 4: an invalid directory is stderr and exit 1.
  • wiki graph replaces --wikilinks / --no-wikilinks and --external-links with fiona.wiki.graph settings after the CLI has set them (load_graph_settings in wiki_graph_main). An explicit flag wins. Settings fill the defaults only (SPEC 4.12).
  • wiki graph drops files above max_files with a log line only. Print the kept count and the limit on stderr.
  • mdoctor fix rewrites personal/ and raw/ (SPEC 4.3, appendix). Constraint 1.2.3 says those zones are read-only for every automated tool. Skip them in fix_file_programmatically, then change SPEC 4.3 to say the command skips those zones.
  • graph_generator.py logs with f-strings. Use lazy % formatting, the same style as logging_config.py.
  • fiona-assistant python-ruff has an empty help string and a #TODO in src/fiona/__main__.py. Use a one-line help like the other linters.
  • fiona.__main__ rewrites sys.argv and calls each tool main(), which parses the arguments a second time. Call the library functions directly so each command is parsed once.

P1 — Wiki Lifecycle and Version Control

  • wiki init hardening (SPEC 7.1)
    • Warn or refuse when the target is already an llm-wiki directory
    • Do not clobber user content on re-run (idempotent)
  • fiona-assistant wiki update (SPEC 7.2)
    • Create src/wiki/update_manager.py with stub functions
    • Add CLI commands: wiki check-update and wiki update
    • Add comprehensive test suite (48 tests)
    • Update core md files / scripts to current template versions (compare version frontmatter)
    • Update directory structure when the template changed
    • Skip locally-modified files and report them
  • fiona-assistant wiki sanity (SPEC 7.3)
    • Run all linters: mdlint, python-ruff, shellcheck (once P2 lands)
    • Dead-link check (relative and wiki-internal links)
  • Git version control for llm-wiki (SPEC 7.5)
    • GitPython library wrapper
    • Basic operations:
      • save — commit current edits with yyyy-mm-dd HH:MM:SS <description>
      • snapshot — save plus a git tag yyyy-mm-dd_HH-MM-SS_<tag_name>
      • restore — create new commit restoring to <tag_name> state
      • clean — discard current edits (staging area); git status must be clean afterwards
      • undo — go back one commit; error if the working tree is dirty and suggest clean first

P2 — Quality Tooling

  • Restore the run_sanity progress heartbeat ([WAITING] lines while checks run) on top of the shared SanityRunner engine (SPEC 7.16.1); dropped when the engine was generalized
    • SanityRunner.run_all takes optional progress and heartbeat callbacks; check lifecycle logged at INFO
    • run_sanity passes stdout callbacks (DONE markers and WAITING lines, assigned via a carriage-return constant); wiki sanity output unchanged
  • Frontmatter linter (fmlint) (SPEC 7.8)
    • Implement src/customlinter/fmlint.py
    • Write TDD tests for fmlint
    • Add CLI integration: fiona-assistant fmlint chklint [directory]
    • Integrate into run_sanity.py and wiki sanity
    • Add settings support via fiona-settings.json
    • SKILL.md name, folder, and description checks (FML004-FML006)
  • Generalize compress and format violations for all linters (SPEC 6.6, 6.1)
    • BaseLinter provides compress_violations() and format_compressed_violations(); report header comes from the linter_name class attribute
    • fmlint delegates format_violations and lint_directory compression to the shared base methods (output unchanged)
    • Migrate mdoctor (compress_violations/format_violations, same compressed shape as fmlint) onto the shared base format
    • Migrate python_ruff (different compressed shape: dict[rel_path, list[dict]]) onto the shared base format
    • Keep all linter reports diffable: same sections, same order (update SPEC 7.7/7.11 and tests together)
  • JSON linter (json-lint) (SPEC 7.18)
    • Implement src/customlinter/jsonlint.py
    • Write TDD tests for json-lint
    • JSON syntax validation using the standard json module
    • JSON style rules (indentation, trailing commas, key ordering)
    • CLI: fiona-assistant json-lint chklint and fix
    • Settings via the json section of fiona-settings.json
    • Integrated into run_sanity.py and wiki sanity
  • Shellcheck library (SPEC 7.12)
    • add shellcheck to toml
    • Wrap shellcheck like python_ruff (AI-friendly violation report)
    • Clear install-hint error when shellcheck is missing
  • Lint fenced code blocks inside MD files (SPEC 7.13)
    • Python blocks via Ruff, shell blocks via shellcheck
  • Logging library instead of print for diagnostics (SPEC 7.20)
    • Shared fiona.logging_config.setup_logging() on CLI entry points
    • Error and warning diagnostics go to stderr through logging. CLI reports stay stdout prints. git_tool, graph_generator, and several CLIs still print errors
  • Enable every Ruff rule (select = ["ALL"] in pyproject.toml)
    • The table under [tool.ruff] currently leaves Ruff on its built-in default. That skips style, docstrings, line length, pyupgrade, and other families
    • Turn on ALL, then clear the findings on src/ or add an explicit ignore only where a rule fights the spec (CLI print on stdout is one)
    • Keep test_lint_detects_unused_import green. It must find unused-import F401 even when another rule is reported first
  • Clear existing lint violations (SPEC 1 constraint: Ruff and PyMarkdown must pass)
    • Ruff: 0 violations — removed unused imports and unnecessary pass statements, sorted imports, made dates tz-aware (UTC), added explicit check=False to subprocess.run, added from __future__ import annotations to wiki.py
    • Markdownlint: 0 violations — fixed blanks around headings/lists, line length, fence languages; mdoctor now enables the pymarkdown front-matter extension (files with frontmatter no longer raise phantom MD003/MD022/MD041) and ignores the frontmatter title for MD025
  • Settings manager, fiona-assistant setup (SPEC 7.19)
  • When running sanity, make sure that markdownlint tests also include outer dir of the src/ because there are markdown files there
  • in run_sanity.py, show the failing test function and assert ex.
  • add pyspellchecker in spec for spelling checks in markdown
  • add in spec, we add a new function in toc_generator,
    • generate toc for files.
      • All headings except H1 # will be included
      • the toc will be generated below the H1,
      • make a logic to detect of a toc exists then regenerate to update.
      • make a logic to add section numbers, (also add section numbers to the actual headings to preserve link is working)
      • in fiona-settings.json, add specific section to modify parameters such as, max heading to include e.g min:2, max:6, sectionize: true/false, ignore_files:[], etc
      • fiona-assistant librarian toc file <path_to_file> will generate the toc for specific file
      • fiona-assistant librarian toc file * will generate the toc for all files recursively in the current directory/llm-wiki dir if detected
  - test_md_frontmatter.py       43/44 passed, 1 failed
     - TestParseFrontmatter : test_parse_valid_frontmatter
     - assert 'Test Document' == 'Test Documen'
  - test_mdoctor.py              33/33 passed
  • prevent the pytest short summary missing from output
WARNING run_tests: pytest short summary missing from output; failure details come from the junit report (check PYTEST_ADDOPTS, --no-summary, or plugin output)
  • Fix terminal-width-dependent failure of the failure-report integration tests (TestFailedTestReportIntegration in src/tests/test_run_sanity.py)

    • Symptom: test_real_error_shows_function_and_message (and its assert-based sibling) fail on python src/run_sanity.py in an interactive terminal with the report literally showing RuntimeErro..., but pass when CI is set in the environment
    • Root cause: pytest trims the short-summary line ERROR <nodeid> - <message> to the terminal width (_format_trimmed in _pytest/terminal.py; width from shutil.get_terminal_size, i.e. COLUMNS or the tty) unless CI/BUILD_NUMBER is set or -vv is used. The temp-test nodeids (~117 chars, ../../../../tmp/pytest-of-user/...) push the line past a ~140-column terminal, so the message is cut to RuntimeErro.... run_tests._parse_failed_tests_output captures that trimmed text as a real (non-generic) message and _merge_failure_details keeps it over the junit report's full RuntimeError: boom, so the truncated text reaches the report and the exact-match assertion fails
    • Reproduce: env -u CI COLUMNS=140 python -m pytest -q fails; CI=true python -m pytest -q passes. Bisect context: last commit where sanity passes in a plain terminal is 43cb7f3 ("clean up pytests"); e750a3d/cb7c3d8 introduced the env-sensitive integration tests (057ac8b additionally has a genuinely broken test, fixed later)
    • Recommended fix (TDD): in run_tests.py, treat short-summary messages that end in ... as unreliable — same as the generic fallback — so _merge_failure_details lets the junit entry (which always carries the full message) replace them. Write the failing test first: unit-test _merge_failure_details with a ...-trimmed regex entry vs a junit entry, assert junit wins
    • Alternative one-liner: set CI=1 in the sanitized subprocess environment in _sanitize_pytest_env (pytest skips summary trimming when CI is set); keep junit parsing as the safety net
  • Make scripts/run_sanity.py fast on Windows (2026-09-29 investigation). A clean run takes about 2 to 3.5 minutes on Windows and at most 20 seconds on Linux

    • Symptom, measured with LOG_LEVEL=INFO and --non-interactive. The machine had other load during the run, so the absolute times are high. The ratios hold
      • pytests took 03:26 and markdownlint took 00:26. ruff, frontmatter_lint, pycustomlint, and jsonlint each took 00:00
      • SanityRunner.run_all runs the checks in one thread each. The total is the slowest check, which is pytests
      • python -m pytest tests alone, without coverage, took 165 s for 962 tests. The sanity run adds --cov, which makes it slower
    • Root cause: each git process is slow to start on Windows, and the run starts hundreds of them
      • Measured cost per subprocess.run on this Windows box: git --version about 198 ms, python -c pass about 150 ms, cmd /c exit about 51 ms. On Linux a git call takes about 1 to 3 ms
      • C:\Program Files\Git\cmd\git.exe is a wrapper that starts mingw64\bin\git.exe, so each call creates two processes. Endpoint security that scans each new process likely adds to the cost
    • Source 1: pymarkdown respect_gitignore starts git twice for every directory it walks
      • fiona-settings.json sets respect_gitignore: true. MDoctor.scan_file_programmatically passes it to PyMarkdownApi.scan_path(..., recurse_if_directory=True)
      • In application_file_scanner (a pymarkdownlnt dependency), each directory in the os.walk loop calls GitProcessor.get_check_ignores. That runs git rev-parse --show-toplevel and then git check-ignore (application_file_scanner.py, about line 850; git_processor.py)
      • One more batched git check-ignore runs over the final file list (__remove_any_gitignored_paths)
      • Repo-level run_markdownlint scans .. That walks the repo root, including .git, .ruff_cache, and the other directories that are not ignored. It took 18 to 26 s alone, which is longer than the whole Linux run
      • MDoctor._ignored_by_gitignore reads .gitignore without git, but it only filters the violations afterwards. It does not stop pymarkdown from walking and calling git
    • Source 2: the test suite runs real sanity scans and real git
      • Slowest single item: the session fixtures in tests/conftest.py. pristine_wiki calls wiki_init (1.7 s). pristine_check_results calls run_wiki_checks. The two together take 24 s of setup on test_sanity_manager.py::TestSanityChecksRegistry::test_registry_drives_results
      • The initialized wiki is a git repo with 96 files and 75 directories. Timed per check, run_wiki_checks spends 29.2 s in markdownlint: about 150 git calls at about 190 ms each. frontmatter_lint takes 0.1 s, python-ruff 0.3 s, mdlinter 0.1 s, and jsonlint 0.0 s
      • Other tests repeat full wiki sanity scans at 1.3 to 2.6 s each (test_sanity_manager.py, test_fiona_main.py test_main_wiki_sanity*, test_mdoctor.py::TestMDoctorGitignore::test_scan_skips_gitignored_files)
      • test_gittool.py: the git_repo fixture uses GitPython, which starts real git processes. Setup takes 1.3 to 2 s per test, and calls take up to 3.4 s (test_restore_by_full_tag, test_restore_by_partial_tag, test_undo, test_clean)
      • wiki_init tests take 2 to 3.3 s each (test_wiki.py, test_fiona_main.py::test_main_wiki_init). Each copies the template and runs a git init and commit
      • TestFailedTestReportIntegration in test_run_sanity.py starts a nested pytest subprocess, about 2.2 s per test
      • The other ~900 tests average about 60 ms each. Temp directory and file creation is slow on Windows, more so with antivirus scanning
    • Recommended fixes, in order of impact (TDD, SPEC first when behavior changes)
      • Stop pymarkdown from calling git. Pass respect_gitignore=False to scan_path and fix_path. Build exclude_patterns from the .gitignore directory names that MDoctor._gitignore_dir_names already parses, plus .git. Or collect the file list with the librarian and scan each file. This removes nearly all git calls from the markdownlint check and from the wiki-sanity tests. Done in MDoctor._pymarkdown_path_options. Patterns are omitted when the scan root is outside the working directory, because the matcher rejects those paths. The post-scan directory check still drops ignored files. SPEC 7.7.3
      • Make the pristine and minimal wiki fixtures cheaper. Use a wiki that is not a git repo, or turn off respect_gitignore in the fixture's settings. The scan change above removes the git calls from pristine_check_results, which was the 29 s cost. The fixtures themselves are unchanged. wiki_init is still about 1.7 s once per session
      • Share one session-scoped git repo across the read-only test_gittool.py tests. Keep per-test repos only for tests that change state. Fixture is read_only_repo
      • Run pytest in parallel with pytest-xdist (-n auto) in TestRunner.run_pytest. Confirm coverage still combines correctly (pytest-cov supports xdist). Tried on 2026-09-30: coverage still merged at 96.0%, and pytests went from 01:56 to 01:40. Worker progress lines ([gwN] [100%] FAILED path) do not match the short-summary parser, so TestFailedTestReportIntegration reported Tests: 0/1 passed and dropped the assert text. The flag was removed. Parallel workers are not a win on this box
      • Optional, outside the repo: exclude the repo directory and git.exe from antivirus scanning, if IT policy allows it
    • Acceptance: record Windows and Linux timings for python scripts/run_sanity.py --non-interactive before and after the change. Windows should finish well under 1 minute. Linux was not re-measured (this machine is Windows). The todo baseline is at most 20 s on Linux. Windows before, this machine: about 02:50 in the session that wrote the git fix, and the 2026-09-29 note above (pytests 03:26, markdownlint 00:26). Windows after the git fix, shared read-only repo, and os.pathsep (2026-09-30, LOG_LEVEL=INFO): pytests 01:54, markdownlint 00:10, SANITY PASSED 01:54 (968 tests, coverage 96.0%). An earlier pass the same day was 01:56. That is under the old 2 to 3.5 minutes and still above 1 minute. The remaining time is pytest itself: temp files, the tests that still start git, and nested pytest
    • Side finding, not a speed issue: run_tests.py joins PYTHONPATH with ":" (f"{src_dir}:{current_pythonpath}"). On Windows the separator is ;. Use os.pathsep. Done. SPEC 4.13
  • in toc generator settings, per-file glob overrides in the file section (SPEC 7.15.2)

  • instead of current directory as default, find a fiona-settings.json in the current directory / subdirectory, that will be the llm-wiki root and default directory for commands (SPEC 7.23)

  • Enable the pymarkdown markdown-tables extension in fiona-settings.json so table rules (MD055/MD056/MD058) run in run_sanity.py (SPEC 5.1 known limitation)

    • Blocked: pymarkdownlnt 0.9.40 crashes with BadTokenizationError on list lines containing pipes (e.g. generated table_of_contents.md entries); upgrade pymarkdownlnt once fixed and re-run sanity

P3 — Metadata, Backup, and Documentation

  • Backup script for llm-wiki so users can recover if something goes wrong (SPEC 7.6)
  • Frontmatter tags on MD files (SPEC 6.2)
    • version
    • date tag
    • Quality signal: confidence
    • contested flag (true if claims are disputed)
  • Project documentation (SPEC 7.21)
    • Root README.md links SPEC.md
    • Root README and AGENTS.md document bootstrap and bootstrap.ps1
    • Refresh librarian/README.md and mdoctor/README.md for the current subcommands (list, toc, chklint, fix)
    • Refresh src/README.md structure (wiki/, python_ruff/, run_tests.py, run_sanity.py, settings_dev.json). Full gap list is the last item of the P0 2026-09-29 structure audit, including the root README link to ./llm-wiki/AGENTS.md
  • Process hygiene: use semantic commit prefixes (feat:, fix:, docs:, ...) per project agent rule 8.1
  • fiona-assistant wiki graph (SPEC 7.4): link parser, graph, HTML template, and CLI in src/wiki/graph_generator.py
  • Draw only important graph nodes until a parent is clicked (SPEC 7.4.16)
    • Setting fiona.wiki.graph.node_importance_min_point
    • 0, or a missing key, draws every node
    • A positive integer hides a node whose unique_points is below that integer until one of its parents is clicked

P4 — Design Decisions Needed

  • Librarian: version tracking for non-MD files (SPEC 7.14)
    • Python files — format TBD
    • Bash scripts — format TBD
    • Decide the tagging format before implementing
  • Define LLM wiki tool roles (SPEC 7.22)
    • Document purpose of src/ tools
    • Classify tools by role:
      • Ingestion: moving files from ingest/ to raw/
      • Processing: converting raw/ to wiki/concepts/
      • Maintenance: updating index.md / log.md
    • Add scripts to utils/ for these tasks (e.g. ingest.py, process.py)

Open Questions (SPEC)

  • Tagging format for python files and bash scripts (SPEC 7.14)
  • Git wrapper location: implemented at src/gittool/git_tool.py (SPEC 7.5)
  • Wiki root discovery (SPEC 7.23): the nearest fiona-settings.json wins; error when two at the same depth are equally near. Ancestor search is still open.