diff --git a/todo.md b/todo.md index 0b64db3..41e9edc 100644 --- a/todo.md +++ b/todo.md @@ -23,8 +23,9 @@ Ordered by priority. Each item references its section in [SPEC.md](./SPEC.md) ## P0 — Packaging and Structure Fixes -From the 2026-09-23 structure audit. These are repo-hygiene items that -block a correct install or make the docs contradict the code. +From the 2026-09-23 structure audit, plus the 2026-09-29 layout audit +below. These are repo-hygiene items that block a correct install or make +the docs contradict the code. - [x] Fix PyPI packaging so a non-editable install works (wheel build confirmed `python_ruff` is missing entirely) @@ -79,7 +80,175 @@ block a correct install or make the docs contradict the code. - [x] same thing for ruff - [ ] Remove the phantom `html_js_lint` package and `html-js-lint` console script from `pyproject.toml` until SPEC 4.12 is implemented. The - directory `src/html_js_lint/` does not exist + directory `src/html_js_lint/` does not exist. Detail is item 2 in + the 2026-09-29 audit below + +### 2026-09-29 structure audit + +`pyproject.toml` declares a PEP 621 project and the code lives under +`src/`, but the tree is nine sibling import packages plus loose scripts. +A src layout is meant to install one import package. Target shape when +this cluster is done: + +```text +src/fiona/ +├── __main__.py +├── librarian/ +├── mdoctor/ +├── customlinter/ +├── python_ruff/ +├── gittool/ +├── sanity_runner/ +├── wiki/ +└── data/llm_wiki/ # templates, no fake package +tests/ # repo root, not installed +``` + +Keep a single required entry point: +`fiona-assistant = fiona.__main__:main`. Optional thin scripts can stay +as extra entry points that call into those submodules. Do this as one +change set: import paths, tests, entry points, and docs move together. +Half-moving a package leaves both layouts on `sys.path`. + +- [ ] Collapse the nine top-level packages into `fiona` subpackages + - [ ] Current install names, from `packages.find` `include` and from + `src/fiona_assistant.egg-info/top_level.txt`: `customlinter`, + `fiona`, `gittool`, `librarian`, `llm-wiki`, `mdoctor`, + `python_ruff`, `sanity_runner`, `wiki` + - [ ] `fiona/__main__.py` imports them as top-level modules + (`from librarian.librarian import main`, `from wiki.wiki import + wiki_init`, and the same for `customlinter`, `gittool`, + `mdoctor`, `python_ruff`) + - [ ] Same-name module pattern to preserve while moving: + `librarian/librarian.py`, `mdoctor/mdoctor.py`, + `python_ruff/python_ruff.py`, `sanity_runner/sanity_runner.py`. + `gittool` is the odd one: the module is `git_tool.py` + - [ ] `wiki` is a generic top-level name and will collide with any + other installed project that also ships a `wiki` package + - [ ] After the move, `packages.find` `include` should list `fiona` + only. An allow-list is how `python_ruff` was shipped late and + how `html_js_lint` stayed declared after the directory vanished +- [ ] Remove the phantom `html_js_lint` declaration (same checkbox as + the item above this section) + - [ ] `[project.scripts]` maps `html-js-lint` to + `html_js_lint.html_js_lint:main` + - [ ] `packages.find` `include` lists `html_js_lint` + - [ ] `src/fiona_assistant.egg-info/entry_points.txt` still has that + script from the last editable install + - [ ] `src/html_js_lint/` does not exist. Invoking `html-js-lint` + fails at import. SPEC 4.12 is the future feature; the entry + point should not ship until that directory exists + - [ ] `html5validator` is in runtime `dependencies` and is not + imported anywhere under `src/`. Drop it with the phantom + script, or keep it only when SPEC 4.12 lands +- [ ] Delete `src/__init__.py` + - [ ] The file's only content is the comment `# Fiona tools package` + - [ ] An `__init__.py` in `src/` makes `src` itself a package whenever + the repo root is on `sys.path`. The src layout exists so + packages are importable only after install + - [ ] That is why bytecode for `run_sanity.py` and `run_tests.py` + lands in `src/__pycache__/` (`run_sanity.cpython-313.pyc`, + `run_tests.cpython-313.pyc`, `__init__.cpython-313.pyc`) +- [ ] Stop tests from editing `sys.path` + - [ ] `src/tests/conftest.py` does + `sys.path.insert(0, os.path.dirname(os.path.dirname(...)))`, + which puts `src/` on the path, then imports `librarian`, + `mdoctor`, `python_ruff`, and `wiki` as top-level names + - [ ] `src/tests/__init__.py` exists, so `tests` is a package sitting + inside the source root. Standard layout is a repo-root `tests/` + directory that is not installed + - [ ] `[tool.pytest.ini_options]` sets `testpaths = ["src/tests"]` and + has no `pythonpath`. Coverage `source = ["src"]` treats the + mixed script-and-package directory as the measured tree + - [ ] Replacement: `pip install -e .` in the dev env, or + `pythonpath = ["src"]` once there is a single `fiona` package. + The path edit hides "package not installed" failures + - [ ] The 2026-09-27 conftest consolidation (outer `src/conftest.py` + removed) stays valid. This item replaces the remaining + `sys.path` bootstrap, it does not undo that merge +- [ ] Decide where the loose `src/` files live + - [ ] `src/run_sanity.py` and `src/run_tests.py` are modules, not + packages. They are absent from `packages.find`, so a wheel + does not install them. They are how this repo runs its own + sanity and tests (`python src/run_sanity.py`) + - [ ] `src/install_packages.sh`, `src/utils.sh`, and + `src/settings_dev.json` sit in the same directory. Dev config + and shell helpers are not package source + - [ ] Either give `run_sanity` / `run_tests` real entry points under + `fiona`, or move them to a repo-root `scripts/` (or `tools/`) + directory that is not on the package path +- [ ] Replace the hyphenated `llm-wiki` package with package data + - [ ] `import llm-wiki` is a `SyntaxError`. `src/llm-wiki/__init__.py` + exists only so setuptools detects the directory. The + `pyproject.toml` comment already documents that hack + - [ ] `[tool.setuptools.package-data]` maps `"llm-wiki" = ["*", "**/*"]` + - [ ] `src/llm-wiki/__init__.py` sets `__version__ = "0.1.0"` while + `[project]` `version` is `0.36.0`. Two versions for one + distribution + - [ ] Move the template tree to something importable, for example + `src/fiona/data/llm_wiki/`, and load files with + `importlib.resources`. Update `wiki init` / `wiki update` + path helpers (`get_llm_wiki_source_path`) in the same change + - [ ] Root `README.md` links `./llm-wiki/AGENTS.md`. The file is + `src/llm-wiki/AGENTS.md` +- [ ] Get setuptools metadata out of `src/` + - [ ] `src/fiona_assistant.egg-info/` holds `PKG-INFO`, + `entry_points.txt`, `requires.txt`, `SOURCES.txt`, + `top_level.txt`. That is install metadata. It belongs at the + repo root (or nowhere, if it is only a local editable-install + artifact) + - [ ] `.gitignore` already ignores `*.egg-info/`, so this directory + should not be committed, but `where = ["src"]` can still see it + during package discovery + - [ ] The egg name `fiona_assistant` is the setuptools normalization + of project name `fiona-assistant`. Leave the normalization + alone; only the directory location is wrong +- [ ] Keep a single coverage config, and point it at the code that exists + - [ ] Both `.coveragerc` and `[tool.coverage]` in `pyproject.toml` + exist. Coverage reads `.coveragerc` first, so the pyproject + block is unused + - [ ] `.coveragerc` still omits `src/mdoctor/fmlint.py` and + `src/python_ruff/pycustomlint.py`. Those modules live under + `src/customlinter/` (`fmlint.py`, `pycustomlint.py`) + - [ ] Both files share the same `exclude_lines` list. After deleting + `.coveragerc`, confirm `pytest --cov` still honors the + pyproject `source`, `omit`, and `exclude_lines` + - [ ] `coverage.xml` at the repo root is a generated report. It is + not matched by `.gitignore` (`*.log` and `.coverage` are; + `coverage.xml` is not) +- [ ] Raise the setuptools build pin so `[project]` is valid + - [ ] `[build-system]` `requires` is `setuptools>=42` + - [ ] The `[project]` table (PEP 621) needs setuptools 61 or newer. + An older setuptools that satisfies `>=42` can fail the build + - [ ] Set `requires = ["setuptools>=61"]` (or a current floor) and + rebuild the wheel +- [ ] Sort runtime dependencies from repo-only tools + - [ ] `ruff` is imported by `src/python_ruff/python_ruff.py`, so it + stays in `dependencies` + - [ ] `pylint` is in `dependencies` with no version pin, and nothing + under `src/` imports it. Move it to the `dev` extra or drop it + - [ ] There is no `[tool.ruff]` table and no `ruff.toml`. If this + repo is linted with Ruff, the rule set belongs in + `pyproject.toml` + - [ ] `requires-python` is `>=3.8`. Local bytecode in + `src/__pycache__/` is CPython 3.13. Classifiers only say + `Programming Language :: Python :: 3`. The bootstrap examples + use 3.11. Record the versions this repo actually runs, and + confirm syntax such as `str | None` is either covered by + `from __future__ import annotations` or by a higher + `requires-python` +- [ ] Make the structure docs match the tree (also listed under P3) + - [ ] `src/README.md` "Project Structure" lists only `fiona/`, + `librarian/`, `mdoctor/`, `tests/`, and `README.md`. Missing + from that diagram: `customlinter/`, `python_ruff/`, `gittool/`, + `sanity_runner/`, `wiki/`, `llm-wiki/`, `run_sanity.py`, + `run_tests.py`, `settings_dev.json`, `utils.sh`, + `install_packages.sh` + - [ ] MDoctor's description in that file is still + `[Description to be added]`, and Contributing is still + `[Contribution guidelines to be added]` + - [ ] Refresh this diagram again after the package collapse, so the + docs describe `src/fiona/` rather than today's sibling packages ## P1 — Wiki Lifecycle and Version Control @@ -265,7 +434,9 @@ WARNING run_tests: pytest short summary missing from output; failure details com - [ ] Refresh `librarian/README.md` and `mdoctor/README.md` for the current subcommands (`list`, `toc`, `chklint`, `fix`) - [ ] Refresh `src/README.md` structure (`wiki/`, `python_ruff/`, - `run_tests.py`, `run_sanity.py`, `settings_dev.json`) + `run_tests.py`, `run_sanity.py`, `settings_dev.json`). Full + gap list is the last item of the P0 2026-09-29 structure audit, + including the root README link to `./llm-wiki/AGENTS.md` - [ ] Process hygiene: use semantic commit prefixes (`feat:`, `fix:`, `docs:`, ...) per project agent rule 8.1 - [x] `fiona-assistant wiki graph` (SPEC 3.1.4): link parser, graph,