Files
daedalus/SPEC.md
T
nvaldez bd1a10487c feat: PyPI distribution renamed daedalus-tools; PEP 625 artifact names (v0.13.0)
PyPI policy-blocks the plain name: the 0.12.1 upload was rejected
with "The name 'daedalus' isn't allowed" (near the existing
daedalus-cli and pydaedalus projects). The distribution is now
daedalus-tools; command, imports, registry, and the repository
name stay daedalus. hello's installed-version lookup follows.
publish now derives artifact file names from the PEP 625
normalized name (daedalus_tools-<version>.tar.gz) - with a
hyphenated project name the previously constructed names would
have missed what setuptools writes to dist/ (publish spec 3.2,
new regression test).

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
2026-11-19 10:58:33 +01:00

755 lines
38 KiB
Markdown

# daedalus specification
How this repository is laid out, how scripts are registered and run, and
the rules for maintaining it. This file alone is enough to rebuild the
tool: it states every observable behavior (arguments, output, exit codes,
environment variables, files written, packaging, checks). Change this file
first when the behavior changes, then change the code and tests to match.
This checkout is an **empty template**: the CLI and quality gates are
ready; add scripts per section 6.
<!-- toc-start: headings 2..6 -->
- [1. Purpose](#1-purpose)
- [2. Layout](#2-layout)
- [3. Command line](#3-command-line)
- [4. Registry](#4-registry)
- [5. Runners](#5-runners)
- [6. Adding a script](#6-adding-a-script)
- [7. Quality gates](#7-quality-gates)
- [8. Conventions](#8-conventions)
- [9. Packaging](#9-packaging)
- [10. Library imports](#10-library-imports)
- [11. Logging for Python scripts](#11-logging-for-python-scripts)
- [12. Implementation plan](#12-implementation-plan)
<!-- toc-end -->
## 1. Purpose
- **1.1** Every heading and list item has a stable id. Cite an item as
`SPEC item 3.7.1`. New items get the next free number; numbers of
removed items are not reused.
- **1.2** daedalus keeps loose personal scripts in one repository, each
with tests, and runs any of them through one command:
```text
daedalus <name> [args...]
daedalus <category> [category...] list scripts (AND when multiple)
daedalus <category>… <name> [args...] run when the script has every category
```
In this template checkout, `SCRIPTS` is empty until the first script is
added (section 6).
- **1.3** The same tool runs on a Windows workstation and on a Linux
host, either installed with pipx (section 9) or run from a checkout
(SPEC item 3.7.2).
- **1.4** Scripts are written in Python, bash, PowerShell, or batch. The
CLI picks the interpreter from the file suffix and passes arguments,
terminal, and exit code straight through.
- **1.5** Reuse first: when a popular, widely tested, trusted
third-party package already solves a problem, use it instead of
writing our own code; the Python standard library covers the rest.
Python 3.11 or newer. Dependency policy: SPEC item 9.6. Today the one
dependency is argcomplete, for shell completion (SPEC item 3.13);
its import is guarded, so the CLI also runs where it is not
installed — completion is then off.
## 2. Layout
- **2.1** Repository tree:
```text
daedalus/
├── bin/
│ ├── daedalus # Linux launcher (bash), no install needed; PYTHON_ARGCOMPLETE_OK marker
│ └── daedalus.bat # Windows launcher (cmd / PowerShell)
├── daedalus/ # The CLI package
│ ├── __init__.py # PACKAGE_DIR, REPO_ROOT, LIBRARY_ROOT, checkout imports
│ ├── __main__.py # python -m daedalus
│ ├── cli.py # argparse parser, list, path, dispatch
│ ├── completions.py # Completion candidates for argcomplete
│ ├── registry.py # Script, SCRIPTS, category helpers
│ ├── runners.py # Suffix -> interpreter command, run
│ ├── run_python_script.py # Child entry for .py scripts (logging, argv)
│ ├── daedalus_logging/ # File + console logging (section 11)
│ └── tests/
├── python/
│ ├── src/<name>/ # Python scripts, one folder per script,
│ │ # holding <name>.py and spec.md (SPEC item 6.11)
│ └── tests/ # pytest tests for python/src, .ps1, .bat
├── bash/ # *.sh scripts (SPEC item 6.10 interface),
│ # each with a <name>.spec.md sibling
│ ├── daedalus_common.sh # shared helper: daedalus_bash_dispatch, help() routing
│ └── tests/ # pytest tests that run them
├── powershell/ # *.ps1 scripts, each with <name>.spec.md
├── batch/ # *.bat scripts (CRLF), each with <name>.spec.md
├── ci/ # sanity.py, checks.py, with tests
├── autotest.py # Runs every check (local and CI)
├── pyproject.toml # Package, version, entry point, package data
├── requirements.txt # Test and lint tools
├── .gitea/workflows/ci.yml # CI job that runs autotest.py
├── .gitattributes # LF everywhere, CRLF for *.bat
├── .editorconfig # UTF-8, LF, final newline, 4 spaces (2 for yml/json/md)
├── CHANGELOG.md # Release notes, newest first
├── README.md # Script catalog (user-facing)
├── INSTALL.md # Install, usage, development
├── AGENTS.md # Rules for AI agents
└── SPEC.md # This file
```
- **2.2** Three paths are computed in `daedalus/__init__.py`:
- **2.2.1** `PACKAGE_DIR`: the resolved folder of the `daedalus`
package.
- **2.2.2** `REPO_ROOT`: the parent of `PACKAGE_DIR` (the checkout root,
or `site-packages` in an install).
- **2.2.3** `LIBRARY_ROOT`: see SPEC item 9.3.
## 3. Command line
| Call | Result | Exit code |
| --- | --- | --- |
| `daedalus <name> [args...]` | Runs the registered script with `args` passed through unchanged | The script's exit code |
| `daedalus <category> [category...]` | Prints the version line, then scripts that have **every** listed category | 0 |
| `daedalus <category>… <name> [args...]` | Runs `<name>` when it has every category prefix; same pass-through as the flat form | The script's exit code |
| `daedalus`, `daedalus -l`, `daedalus --list` | Prints the version line, then every script (SPEC item 3.3) | 0 |
| `daedalus -h`, `daedalus --help` | Prints usage (SPEC item 3.8) | 0 |
| `daedalus --path <name>` | Prints the absolute path of the script | 0 |
| `daedalus --version` | Prints `daedalus <version>` (SPEC item 3.9) | 0 |
| Unknown option, category, or script; script not in category | Prints an error and close matches to stderr | 2 |
| Script file missing, unsupported suffix, or no interpreter found | Prints the reason to stderr | 2 |
- **3.1** Everything after the script name belongs to the script,
including `--help`; these forms show the script help:
- **3.1.1** `daedalus sample_script --help`
- **3.1.2** `daedalus general sample_script --help`
- **3.1.3** `daedalus media convert_video --help`
- **3.2** The flat form `daedalus <name>` stays valid; categories are
optional grouping.
- **3.3** Listing format (full list and category-filtered list):
- **3.3.1** First line `Daedalus Version: <version>`. When at least one
script is listed, a blank line and the table follow; otherwise nothing
follows.
- **3.3.2** The table has a header row `script`, `type`, `categories`,
then a rule row of `-` per column. Columns are left-aligned, padded to
the widest cell (header included) and separated by two spaces. The
`type` column is at least 4 wide.
- **3.3.3** One row per script, sorted by name. `type` is the file
suffix without the dot (`py`, `sh`, `ps1`, `bat`). `categories` is the
labels joined with a comma and a space (`media, linux`).
- **3.3.4** Under each row, the description is wrapped to the terminal
width (at least 40 columns; 80 when the width cannot be read), every
line indented by 8 spaces. Long words and hyphenated words are not
split.
- **3.3.5** Color: when enabled, the padded name cell is green
(`ESC[32m`), the type cell yellow (`ESC[33m`), the categories cell
bright blue (`ESC[94m`), each followed by `ESC[0m`.
- **3.3.6** Color is enabled when `NO_COLOR` is empty or unset and
either `FORCE_COLOR` is non-empty or stdout is a terminal. A non-empty
`NO_COLOR` always wins.
```text
Daedalus Version: 0.1.0
script type categories
-------------------- ---- ------------------
sample_script_name py general, linux
sample script description
```
- **3.4** A token is treated as a **category** only when it matches a
category in use and is **not** also a registered script name (names and
category labels must not collide, SPEC item 4.3).
- **3.5** The script runs in the caller's current directory, with the
caller's stdin, stdout, and stderr, so interactive scripts work.
- **3.6** Ctrl+C reaches the script. daedalus waits for it to exit and
returns its exit code.
- **3.7** Ways to run the CLI:
- **3.7.1** Install it with pipx or pip (section 9). `daedalus` is then
on `PATH` everywhere, with every registered script bundled.
- **3.7.2** `bin/daedalus` (Linux) or `bin\daedalus.bat` (Windows). Add
`bin/` to `PATH` to call plain `daedalus`. Needs no install.
- **3.7.2.1** `bin/daedalus` resolves its own real path (following
symlinks), prepends the checkout root to `PYTHONPATH`, and `exec`s
`python3 -m daedalus "$@"`.
- **3.7.2.2** `bin\daedalus.bat` prepends `<bin>\..` to `PYTHONPATH`,
runs `python -m daedalus %*`, and exits with its `ERRORLEVEL`.
- **3.7.3** `python -m daedalus` from the repository root.
- **3.7.4** The installed command is the entry point
`daedalus = daedalus.cli:main`.
- **3.8** `-h` / `--help` prints the argparse rendering of the parser in
SPEC item 3.10 (80 columns shown; wrapped to the terminal width):
```text
usage: daedalus [-h] [-l | --list] [--version] [--path <name>] [<category> ...] [<name> [args ...]]
One CLI, many small scripts, each with tests (SPEC.md).
positional arguments:
<category|name> category labels to filter or list, then the script name to
run
[args ...] arguments passed to the script
options:
-h, --help show this help message and exit
-l, --list list all registered scripts
--version print the installed version
--path <name> print the path of a script
Forms: daedalus <name> [args...] runs a script; daedalus <category>
[category...] lists the scripts with every category (AND when several);
daedalus <category>... <name> [args...] runs a script that has every category.
```
- **3.9** The version is resolved in this order:
- **3.9.1** The installed distribution metadata for `daedalus`.
- **3.9.2** `project.version` from `REPO_ROOT/pyproject.toml` (a
checkout that is not installed).
- **3.9.3** The string `unknown`.
- **3.10** Argument resolution. One `argparse.ArgumentParser`
(`prog="daedalus"`, `allow_abbrev=False`, subclass whose `error()`
raises `UsageError` instead of exiting):
```text
parser arguments:
-l | --list store_true flag
--version store_true flag
--path <name>
words positional, nargs "*", metavar <category|name>
script_args positional, argparse.REMAINDER, metavar [args ...]
main(argv = sys.argv[1:]):
parser = build_parser()
argcomplete.autocomplete(parser) when importable (SPEC item 3.13)
try:
parsed = parser.parse_args(argv)
return dispatch(parsed)
on usage or runner error: print "daedalus: <message>" to stderr; return 2
dispatch(parsed):
if --list or ((words and script_args both empty) and no option set):
list(all scripts); return 0
if --version: print "daedalus " + version; return 0
if --path:
if words or script_args is not empty: usage error "usage: daedalus --path <name>"
print absolute path of lookup(--path value) # file need not exist
return 0
prefix = leading words entries while is_category_label(entry)
rest = remaining words entries
if prefix and rest is empty:
scripts = scripts having every label in prefix
if none: usage error "no scripts for categories"
list(scripts); return 0
if rest: script = lookup_in(prefix, rest[0]) when prefix else lookup(rest[0])
return run(script, rest[1:] + script_args)
```
With argparse, options are parsed wherever argparse sees them before
the positionals start (the old first-token-only rule is gone), but
everything after the first positional word that argparse cannot bind
to `words` is captured by `script_args` (REMAINDER) and passed through
(SPEC item 3.1): `daedalus <name> --help` still shows the **script's**
help.
- **3.11** Error messages: parser errors carry the argparse message,
registry errors keep their exact texts; each is printed as
`daedalus: <message>` on stderr with exit code 2:
| Case | Message |
| --- | --- |
| `--path` without a value | `argument --path: expected one argument` |
| `--path` with extra words | `usage: daedalus --path <name>` |
| Unknown option | `unrecognized arguments: <opt>` |
| Unknown script, close matches exist | `unknown script '<name>'; did you mean: <m1>, <m2>` |
| Unknown script, no close match | `unknown script '<name>'; run 'daedalus --list'` |
| Categories match no script | `no scripts for categories: <a>, <b>` |
| Unknown script after a category prefix | `unknown script '<name>' for '<a b>'; did you mean: …` or `…; run 'daedalus <a b>'` |
| Script lacks a prefix category | `script '<name>' has categories '<x, y>', not [<missing>]; run 'daedalus <name>' or pick another prefix` |
- **3.12** Close matches are up to 3 names with similarity ratio at least
0.6 (Python `difflib.get_close_matches` defaults). After a category
prefix, only scripts in those categories are candidates.
- **3.13** Shell completion for the parser above (SPEC item 3.10) comes
from argcomplete (SPEC item 1.5):
- **3.13.1** `main` calls `argparse` completion via
`argcomplete.autocomplete(parser)` before `parse_args`, when
argcomplete is importable. Without argcomplete the CLI runs
unchanged, with completion off.
- **3.13.2** Activation (bash): `eval "$(register-python-argcomplete
daedalus)"` in `~/.bashrc`; `register-python-argcomplete` ships with
argcomplete. Other shells argcomplete supports (zsh, fish, tcsh,
powershell) register the same way.
- **3.13.3** Completing a `words` entry (SPEC item 3.10):
`daedalus/completions.py` offers every registered script name and
every category label in use, narrowed by prefix; category labels
already typed are not offered again, and script candidates narrow to
the scripts that have every typed label (SPEC item 3.4): `daedalus
test h<TAB>` completes `hello`. Candidates are sorted.
- **3.13.4** `--path <name>` completes script names.
- **3.13.5** After a registered script name has been typed (completed,
before the word being completed), the rest of the line belongs to
the script (SPEC item 3.1): daedalus offers nothing of its own —
not its options, and never the working-directory file listing (the
`script_args` action keeps its empty completer, SPEC 3.10).
- **3.13.5.1** A `.py` script is asked to complete its own
arguments: daedalus re-execs the interpreter command of its
runner (SPEC item 5.8) with the argument words as the script's
argv, `COMP_LINE` rewritten to start at the script name,
`COMP_POINT` adjusted, the completion environment passed
through, and the completion channel (fd 8) inherited. The script
completes its arguments (SPEC item 6.9); quoting in the words is
not preserved in the rewrite.
- **3.13.5.2** A `.sh` script that implements the interface (SPEC item
6.10) completes the help words `--help`, `-h`, and `help`, filtered
by the word being typed; daedalus writes them itself on the
completion channel — no script runs on TAB. A `.sh` script without
the interface, and `.ps1` / `.bat` scripts, get no candidates: the
shell default (file names) applies.
- **3.13.6** The completion hook re-runs the `daedalus` command on
`PATH`: the program that runs on TAB must import argcomplete
(installed dependency, or the dev venv, or the system package) and
must reach `autocomplete` before parsing.
- **3.13.7** `bin/daedalus` carries a `# PYTHON_ARGCOMPLETE_OK` comment
in its first kilobyte: argcomplete versions that scan for the
marker require it.
## 4. Registry
`daedalus/registry.py` holds one dictionary. In this template checkout it
is empty until the first script is added:
```python
SCRIPTS: dict[str, Script] = {
"<name>": Script(
path="python/src/<name>/<name>.py",
description="One-line summary",
categories="general",
),
"<other_name>": Script(
path="bash/<other_name>.sh",
description="One-line summary",
categories=("category_a", "category_b"),
),
}
```
- **4.1** `Script` is a frozen (immutable) record:
- **4.1.1** `path` is relative to `LIBRARY_ROOT`, with forward slashes.
- **4.1.2** `description` is one line.
- **4.1.3** `categories` is one or more labels, default `general`. A
string becomes a one-item tuple, a list becomes a tuple, and an empty
list or tuple is rejected with `ValueError`.
- **4.1.4** `absolute_path` is `LIBRARY_ROOT / path`.
- **4.1.5** `categories_display` is the labels joined with a comma and
a space.
- **4.2** The key is the command name and matches `^[a-z][a-z0-9_-]*$`.
- **4.3** Each category label uses the same pattern as script names. No
script name may equal any label in `used_categories()` (so
`daedalus media` is unambiguous).
- **4.4** Helpers:
- **4.4.1** `used_categories()`: the set of every label of every script.
- **4.4.2** `is_category_label(token)`: false when `token` is a script
name; otherwise true when it is in `used_categories()`.
- **4.4.3** `scripts_matching_categories(labels)`: scripts that have
every label (AND), sorted by name; all scripts when `labels` is empty.
- **4.5** Leading argv tokens that are category labels form a prefix; the
script must include **every** prefix label in `categories`.
- **4.6** Names that start with `-` are reserved for daedalus options.
- **4.7** The runner is chosen from the suffix of `path`. No other field
selects it.
- **4.8** Tests enforce, for every entry:
- **4.8.1** The name matches the pattern.
- **4.8.2** The file exists, and `path` has no backslash, does not start
with `/`, and resolves inside `LIBRARY_ROOT`.
- **4.8.3** The suffix has a runner.
- **4.8.4** The description is non-empty and has no newline.
- **4.8.5** Categories are non-empty and every label matches the
pattern; no script name equals a label.
- **4.8.6** `README.md` contains ``| `<name>` |``.
## 5. Runners
| Suffix | Command | Platforms |
| --- | --- | --- |
| `.py` | `<python> <PACKAGE_DIR>/run_python_script.py <path> args...` | All |
| `.sh` | `<bash> <path> args...` | All with bash |
| `.ps1` | `<pwsh or powershell> -NoProfile -ExecutionPolicy Bypass -File <path> args...` | All with PowerShell |
| `.bat` | `<COMSPEC or cmd.exe> /c <path> args...` | Windows only |
- **5.1** Building the command, in order:
- **5.1.1** Look up the runner by the lowercased suffix. None found:
error `unsupported script type '<suffix>': <path>`.
- **5.1.2** The file must exist: else error `script not found: <path>`.
- **5.1.3** Find the interpreter (SPEC items 5.3 to 5.6), then append
the script arguments unchanged.
- **5.2** `<python>` is `DAEDALUS_PYTHON` when set and non-empty, else
the interpreter running daedalus. Set it to a virtual-environment
interpreter when a script needs packages that are not in the pipx
environment daedalus was installed into.
- **5.3** bash lookup order:
- **5.3.1** `DAEDALUS_BASH` when set and non-empty.
- **5.3.2** On Windows only: resolve `git` on `PATH`, then use
`<git root>/bin/bash.exe` when that file exists (`git` lives in
`<git root>/cmd/`).
- **5.3.3** `bash` on `PATH`. On Windows, `bash` on `PATH` is often the
WSL launcher, which cannot open Windows paths, so Git Bash comes first.
- **5.3.4** None: error `bash not found; install it or set DAEDALUS_BASH`.
- **5.4** PowerShell lookup order: `pwsh`, then `powershell` on `PATH`.
None: error `PowerShell not found; install pwsh`.
- **5.5** `.bat` on a non-Windows host: error
`<file name>: .bat scripts run on Windows only`.
- **5.6** Every error above exits daedalus with code 2 (SPEC item 3.10),
never a silent skip.
- **5.7** Running:
- **5.7.1** The child gets the caller's environment, with `REPO_ROOT`
(the folder that contains the `daedalus` package) prepended to
`PYTHONPATH` using the platform path separator. This applies to every
script type, so Python code started from a shell script can also
import `daedalus.*` (section 10).
- **5.7.2** The child inherits the current directory and the terminal
(stdin, stdout, stderr are not redirected).
- **5.7.3** daedalus waits for the child and returns its exit code.
When Ctrl+C interrupts the wait, it waits again and returns the
child's exit code.
- **5.8** `run_python_script.py <script.py> [args...]` runs in the child
process:
- **5.8.1** No arguments: print
`usage: run_python_script.py <script.py> [args...]` to stderr, exit 2.
- **5.8.2** Script path (resolved) is not a file: print
`script not found: <path>` to stderr, exit 2.
- **5.8.3** Put the parent of its own folder and its own folder at the
front of `sys.path` (when missing), so `import daedalus` works.
- **5.8.4** Unless `DAEDALUS_NO_LOGGING` is non-empty, configure
logging (section 11) with the console level from `DAEDALUS_LOGLEVEL`
(default `INFO`).
- **5.8.5** Set `sys.argv` to `[<script path>, *args]`, execute the file
as `__main__`, and restore `sys.argv` afterwards.
- **5.8.6** Exit code: 0 when the script returns normally or calls
`sys.exit()` / `sys.exit(None)`; the integer for `sys.exit(<int>)`;
1 for any other exit value. An uncaught exception prints a traceback
and exits 1.
## 6. Adding a script
- **6.1** Put the file in the folder for its type (section 2). A Python
script gets its own folder: `python/src/<name>/<name>.py`. The command
name, file stem, and (for Python) folder name are the same string.
- **6.2** The script must run without daedalus.
- **6.2.1** Python scripts expose `main(argv: list[str] | None)`.
- **6.2.2** They end with
`if __name__ == "__main__": sys.exit(main())`.
- **6.2.3** Shared Python code is imported as `daedalus.<folder>`
(section 10). A script that does so runs on its own when daedalus is
installed or the checkout root is on `PYTHONPATH`.
- **6.2.4** The script never imports the CLI modules (`cli`, `registry`,
`runners`) and never reads `LIBRARY_ROOT`.
- **6.3** Add one entry to `SCRIPTS` in `daedalus/registry.py` (set
`categories` when the script belongs to media, linux, or other
groups).
- **6.4** Add a row to the "Available scripts" table in `README.md`
(columns Script, Type, Categories, What it does), and a short section
with usage when the script takes arguments.
- **6.4.1** A test fails when a registered name has no row.
- **6.5** A new `python/src/<folder>/` is added to `packages` and
`package-dir` in `pyproject.toml` as `daedalus.<folder>` (SPEC item
9.4), so installed code can import it.
- **6.6** Add tests:
- **6.6.1** Python: `python/tests/test_<name>.py`, importing with
`from daedalus.<name> import <name>`.
- **6.6.2** bash: `bash/tests/test_<name>.py`, running the script with
`subprocess` and skipping when bash is missing.
- **6.6.3** PowerShell and batch: a test under `python/tests/` that runs
the script, skipped when the interpreter or Windows is missing.
- **6.7** Bump `version` in `pyproject.toml` and add a `CHANGELOG.md`
entry.
- **6.8** Run `python autotest.py`. It must pass before pushing.
- **6.9** A Python script opts in to completing its own arguments
(SPEC item 3.13.5.1): right after building its parser it calls
`argcomplete.autocomplete(parser)` behind a guarded import (SPEC item
1.5). `daedalus <category>... <name> -<TAB>` then completes the
script's own options; without the call, such words fall back to the
shell default, and a TAB can run the script itself (argcomplete
semantics — scripts that must not run on TAB do not opt in or keep
their parser behind a subcommand).
- **6.10** Bash script interface. Every `*.sh` script registered in
`SCRIPTS` implements the shared interface through
`bash/daedalus_common.sh`:
- **6.10.1** Near the top, the script sources the sibling helper
exactly once:
```bash
# shellcheck source=/dev/null
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/daedalus_common.sh"
```
- **6.10.2** The script defines `help()`, which prints its usage and
option list: the words `daedalus <name> <TAB>` completes (SPEC item
3.13.5.2) route to it.
- **6.10.3** The script defines `main()` with the body; `main "$@"`
receives the script arguments unchanged.
- **6.10.4** The last statement is `daedalus_bash_dispatch "$@"`: the
helper routes `help`, `-h`, and `--help` to `help()` (exit status of
`help`), every other invocation to `main() "$@"` (exit status of
`main`). The help words are reserved for the dispatch in every
bash script.
- **6.10.5** `bash/tests/` enforces the interface for every
registered `.sh` script (source line, `help()`, `main()`, the
dispatch call) and tests the helper behavior directly.
- **6.11** Every script has a local spec that completely
describes it.
- **6.11.1** A Python script's spec is `spec.md` inside its
own folder: `python/src/<name>/spec.md`.
- **6.11.2** A bash, PowerShell, or batch script's spec is a
sibling file named `<name>.spec.md`:
`bash/<name>.spec.md`, `powershell/<name>.spec.md`,
`batch/<name>.spec.md`.
- **6.11.3** The local spec follows the sectionized format of
this file: every heading and list item has a stable numbered
id (`- **N.M**`), sub-items continue the numbering
(`N.M.K`), and items are cited as
`<name> spec item N.M`.
- **6.11.4** The local spec is complete enough to rebuild the
script from it alone: purpose, commands and arguments,
observable behavior (steps, validation, output), exit
codes, files written, error handling, and testing approach.
- **6.11.5** The local spec does not repeat this file; where it
relies on a repository rule it cites the SPEC item instead.
- **6.12** The change order for script work is spec first, then
tests, then code.
- **6.12.1** Create or modify the script's local spec (SPEC
item 6.11) until it describes the intended behavior - a
change that contradicts an existing spec item updates that
item in the same change.
- **6.12.2** Add or change tests that fail without the
behavior.
- **6.12.3** Make the smallest change that makes the tests
pass, then refactor with them green.
- **6.12.4** Spec, tests, code, README, and CHANGELOG land in
the same change; the version bump follows SPEC item 9.8.
## 7. Quality gates
- **7.1** `python autotest.py [CHECK...]` runs the named checks (default:
all). An unknown name is an argument error (exit 2) listing the unknown
names.
- **7.2** Selected checks run concurrently in threads. Each returns an
error count; 0 is a pass. Failures are logged as
`ERROR [FAIL] <check detail>` while the checks run.
- **7.3** After all finish, autotest prints a summary and exits 1 when any
check failed, else 0:
```text
########################################
autotest results
########################################
[ PASS ] 0.42s sanity
[ FAIL ] 12.07s pytest
```
- **7.4** Run it before every commit; do not commit unless every check
passes ([AGENTS.md](./AGENTS.md) section 2.3).
- **7.5** File discovery walks the repository, skipping folders named
`.git`, `.venv`, `venv`, `.cache`, `.pytest_cache`, `.ruff_cache`,
`__pycache__`, `htmlcov`, `node_modules`, `build`, `dist`. Suffixes
match exactly.
| Check | Rule |
| --- | --- |
| `sanity` | Changed files (SPEC item 7.6): line endings (section 8), no private key, at most 2,000,000 bytes |
| `python_syntax` | Every `*.py` parses |
| `pycodestyle` | PEP 8, ignoring `E501 E121 E123 E126 E226 E24 E704 W503 W504`; error count is the violation count |
| `shellcheck` | `shellcheck --external-sources` on every `*.sh`, `*.bash`, and suffixless file whose first line matches `^#!.*\b(ba\|da\|k\|z)?sh\b`; a missing `shellcheck` fails each file |
| `json` | Every `*.json` parses as UTF-8 JSON |
| `yaml` | Every document in every `*.yaml` / `*.yml` loads (safe loader) |
| `markdown` | PyMarkdown with the front-matter extension on every `*.md`; MD013 and MD033 ignored |
| `pytest` | `python -m pytest --cov=ci --cov=daedalus --cov=python/src --cov-report=term-missing:skip-covered --cov-fail-under=<MIN_COVERAGE_PERCENT>` in a subprocess |
- **7.6** Changed files for `sanity`:
- **7.6.1** Base ref: the first that resolves to a commit of
`SANITY_DIFF_BASE`, `origin/<target branch>`, `origin/main`,
`origin/master`.
- **7.6.2** With a base: files added, copied, modified, or renamed
against it (`git diff --name-only --diff-filter=ACMR <ref>`), plus
untracked files not ignored.
- **7.6.3** Without a base: every tracked and untracked, not ignored
file.
- **7.6.4** Deleted paths are skipped.
- **7.7** Sanity rules per file:
- **7.7.1** A file containing a NUL byte is binary and skips the line
ending rule.
- **7.7.2** `*.bat` fails when any `\n` is not preceded by `\r`. Any
other file fails when it contains `\r`.
- **7.7.3** A file fails when it contains
`-----BEGIN <uppercase words> PRIVATE KEY-----`.
- **7.8** `MIN_COVERAGE_PERCENT` in `autotest.py` is 75 and only goes up.
- **7.9** pytest configuration in `pyproject.toml`: test paths
`ci/tests`, `daedalus/tests`, `python/tests`, `bash/tests`; `.` on the
Python path; coverage omits `*/tests/*` and `*/__main__.py`.
- **7.10** CI (`.gitea/workflows/ci.yml`) runs one job on pull request
pipelines and on the default branch: a Python 3.11 image with git and
shellcheck, full git history (for SPEC item 7.6),
`pip install -r requirements.txt`, then `python autotest.py`. The
sanity diff base is passed as `SANITY_DIFF_BASE`.
- **7.11** `requirements.txt` holds the tools: `pytest`, `pytest-cov`,
`pycodestyle`, `pymarkdownlnt`, `pyyaml`, `shellcheck-py`, `setuptools`,
and `argcomplete` (the runtime dependency, so the completion tests run).
## 8. Conventions
- **8.1** Line endings are LF everywhere except `*.bat`, which must be
CRLF.
- **8.1.1** cmd.exe mishandles labels and `goto` in LF-only batch files.
- **8.1.2** `.gitattributes` enforces both on checkout.
- **8.2** Python targets 3.11. Standard library imports use `import module`.
- **8.3** Logging uses lazy `%` formatting.
- **8.6** Markdown lists use dashes.
- **8.7** `bin/daedalus` and every `*.sh` are committed executable:
`git add --chmod=+x <file>`.
- **8.8** `MIN_COVERAGE_PERCENT` in `autotest.py` only goes up. Raise it
when tests are added.
- **8.9** `CHANGELOG.md` follows Keep a Changelog, newest release first,
with release numbers equal to `version` in `pyproject.toml`.
## 9. Packaging
- **9.1** The PyPI distribution is named `daedalus-tools` (plain
`daedalus` is policy-blocked on PyPI - near the existing
`daedalus-cli`/`pydaedalus` projects; command and import names
stay `daedalus`). `pyproject.toml` builds one wheel (setuptools 68
or newer) that holds the CLI and every script. It requires Python
3.11 or newer.
| Command | Use |
| --- | --- |
| `pipx install git+https://example.com/your-org/daedalus.git` | Own isolated environment, `daedalus` on `PATH` (recommended; works on Debian 12, where plain `pip install` is blocked) |
| `pip install --user .` | From a checkout, into the user site |
| `pip install -e .` | Development: the installed command runs the scripts in the checkout |
| `pipx upgrade daedalus` / `pip install --upgrade .` | Pick up new scripts |
- **9.2** The script folders are copied into the wheel under
`daedalus/_library/`, with the same relative paths as in the repository:
| Repository folder | Installed folder | Files |
| --- | --- | --- |
| `python/src/` | `daedalus/_library/python/src/` | Everything except `__pycache__`, `*.pyc`, `.gitkeep` |
| `bash/` | `daedalus/_library/bash/` | `*.sh` |
| `powershell/` | `daedalus/_library/powershell/` | `*.ps1` |
| `batch/` | `daedalus/_library/batch/` | `*.bat` |
- **9.2.1** The source tree must be able to rebuild its wheel from
the sdist: every folder the packaging declares (including empty
`powershell/` and `batch/`) carries at least a `README.md`
placeholder, and `MANIFEST.in` ships both folders in the sdist.
A missing declared package directory is a release-blocking
defect: `python -m build` builds sdist and wheel chained, so a
broken sdist fails the release before it ships.
- **9.3** `LIBRARY_ROOT` is `PACKAGE_DIR/_library` when that folder exists
(a normal install), otherwise `REPO_ROOT` (a checkout or an editable
install).
- **9.3.1** Registry paths resolve against it, so `SCRIPTS` entries are
the same in both cases.
- **9.4** Each `python/src/<folder>/` is also installed as the package
`daedalus.<folder>` (an explicit `packages` and `package-dir` entry per
folder). `daedalus` and `daedalus.daedalus_logging` are packages too.
The wheel therefore holds each Python script twice: the `_library` copy
is what the CLI runs, the `daedalus.<folder>` copy is what imports
resolve to (section 10).
- **9.5** Tests under `*/tests/`, bytecode, and `.gitkeep` are not
packaged.
- **9.6** Runtime dependencies follow SPEC item 1.5 (reuse first): the
wheel declares `argcomplete >= 3` for shell completion (SPEC item
3.13); everything else is the standard library. A script that needs
other third-party packages says so in its own help text and fails
with a clear message when they are missing;
`DAEDALUS_PYTHON` (SPEC item 5.2) points it at their environment.
Test and lint tools stay in `requirements.txt`.
- **9.7** Tests build the wheel from a copy of the checkout and check:
- **9.7.1** Every registered script is at `daedalus/_library/<path>`.
- **9.7.2** No `/tests/`, `*.pyc`, or `.gitkeep` entry is present.
- **9.7.3** The entry point `daedalus = daedalus.cli:main` is declared.
- **9.7.4** Unpacked on `PYTHONPATH`, `daedalus --path <name>` points
into `_library` for every registered `<name>`, `--list` lists it,
`<name> --help` runs, and every registered `daedalus.<folder>`
import resolves.
- **9.8** Bump `version` in `pyproject.toml` when scripts or the CLI
change, so `pipx upgrade` picks them up.
## 10. Library imports
- **10.1** Library Python code imports shared modules under the
`daedalus` namespace, never as top-level names:
`from daedalus.<folder> import <module>`.
- **10.2** In an install, `daedalus.<folder>` is a real installed package
(SPEC item 9.4).
- **10.3** In a checkout, `import daedalus` registers every direct
subfolder of `REPO_ROOT/python/src` (except `__pycache__`) as a module
`daedalus.<folder>` whose package search path is that folder, unless a
module of that name is already loaded. Files are not copied.
```text
on import daedalus:
if REPO_ROOT/python/src is a folder:
for each subfolder F, sorted by name:
if "daedalus.F" not already loaded:
create empty package "daedalus.F" with search path [F]
```
- **10.4** `REPO_ROOT` on `PYTHONPATH` (SPEC item 5.7.1) and the
`sys.path` setup of `run_python_script.py` (SPEC item 5.8.3) make
`import daedalus` succeed in the child, under `DAEDALUS_PYTHON` too.
## 11. Logging for Python scripts
- **11.1** `daedalus.daedalus_logging.configure_logging()` is called by
`run_python_script.py` before the script starts (SPEC item 5.8.4).
Library scripts may call it too. It replaces any existing root handlers.
- **11.2** Root logger level is `DEBUG`, with two handlers:
- **11.2.1** File handler, appending UTF-8 to `~/.daedalus/debug.log`
(parent folders created), all levels, format
`%(asctime)s|%(levelname)-5.5s|%(module)s.%(funcName)s:%(lineno)d:%(message)s`,
date format `%Y-%m-%d %H:%M:%S`.
- **11.2.2** Console handler on stderr, level from
`DAEDALUS_LOGLEVEL` (a level name such as `DEBUG`, `INFO`,
`WARNING`), format `%(levelname)-5.5s|%(message)s`.
- **11.4** Console color is on when `NO_COLOR` is empty or unset and
either `FORCE_COLOR` is non-empty or stderr is a terminal.
- **11.5** With color on:
- **11.5.1** `WARNING` lines are yellow (`ESC[33m`), `ERROR` bold red
(`ESC[1;31m`), `CRITICAL` bold bright red (`ESC[1;91m`); the whole
line is colored and reset at the end. `DEBUG` and `INFO` are plain.
- **11.5.2** An IPv4 address in square brackets, `[10.0.0.1]` or a
range `[10.0.0.1 - 10.0.0.9]`, has its first address in bold blue
(`ESC[1;34m`); the line's level color resumes after it.
- **11.6** `configure_logging` accepts overrides for the log file path,
console level, file level, and color, and returns the resolved log
file path.
## 12. Implementation plan
| Step | Work | Status |
| --- | --- | --- |
| 1 | This SPEC | Done |
| 2 | `daedalus/registry.py` with `Script` and `SCRIPTS` (empty in template) | Done |
| 3 | `daedalus/runners.py`: suffix table, interpreter lookup, `build_command`, `run` | Done |
| 4 | `daedalus/cli.py` and `__main__.py`: list, help, `--path`, dispatch, close-match errors | Done |
| 5 | `bin/daedalus`, `bin/daedalus.bat`, `[project.scripts]` in `pyproject.toml` | Done |
| 6 | Tests: registry consistency, every runner, CLI paths, launchers | Done |
| 7 | Per-script tests under `python/tests/` or `bash/tests/` when scripts exist | Template |
| 8 | CI: CRLF rule for `*.bat`, shellcheck on suffixless shell files, `daedalus` in coverage | Done |
| 9 | README: how to call `daedalus` | Done |
| 10 | `pyproject.toml`: map the script folders into `daedalus/_library` as package data | Done |
| 11 | `LIBRARY_ROOT` in `daedalus/__init__.py`; registry and runners resolve against it | Done |
| 12 | `daedalus --version` | Done |
| 13 | Tests: `LIBRARY_ROOT` selection, wheel contains every registered script, installed CLI | Done |
| 14 | README: install with pipx or pip | Done |
| 15 | Categories in `Script`, CLI `daedalus <category>` and `daedalus <category> <name>` | Done |
| 16 | `run_python_script.py` and `daedalus_logging` (section 11) | Done |
| 17 | `daedalus.<folder>` imports in checkouts and installs (section 10) | Done |
| 18 | Listing table with version line, wrapped descriptions, and color (SPEC item 3.3) | Done |
| 19 | argparse parser and argcomplete shell completion (SPEC items 3.10, 3.13) | Done |