daedalus 0.1.0: scriptman spec update ported - categories, new listing format with colors, run_python_script child entry, daedalus_logging, daedalus.<folder> namespace imports; SPEC rewritten with stable item numbers

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
nvaldez
2026-11-18 21:13:57 +01:00
parent 792beca2c5
commit b71a513c36
19 changed files with 1606 additions and 202 deletions
+481 -119
View File
@@ -1,8 +1,15 @@
# daedalus specification
How this repository is laid out, how scripts are registered and run, and
the rules for maintaining it. Change this file first when the behavior
changes, then change the code and tests to match.
the rules for maintaining it. This file alone is enough to rebuild the
tool: it states every observable behavior (arguments, output, exit codes,
environment variables, files written, packaging, checks). Change this file
first when the behavior changes, then change the code and tests to match.
This checkout is an **empty template**: the CLI and quality gates are
ready; add scripts per section 6.
<!-- toc-start: headings 2..6 -->
- [1. Purpose](#1-purpose)
- [2. Layout](#2-layout)
@@ -13,201 +20,552 @@ changes, then change the code and tests to match.
- [7. Quality gates](#7-quality-gates)
- [8. Conventions](#8-conventions)
- [9. Packaging](#9-packaging)
- [10. Implementation plan](#10-implementation-plan)
- [10. Library imports](#10-library-imports)
- [11. Logging for Python scripts](#11-logging-for-python-scripts)
- [12. Implementation plan](#12-implementation-plan)
<!-- toc-end -->
## 1. Purpose
This repository is an **empty template**: the CLI and quality gates are
ready; add scripts per section 6. A filled checkout keeps loose scripts in
one repository, each with tests, and runs any of them through one command:
- **1.1** Every heading and list item has a stable id. Cite an item as
`SPEC item 3.7.1`. New items get the next free number; numbers of
removed items are not reused.
- **1.2** daedalus keeps loose personal scripts in one repository, each
with tests, and runs any of them through one command:
```text
daedalus <name> [args...]
daedalus <category> [category...] list scripts (AND when multiple)
daedalus <category>… <name> [args...] run when the script has every category
```
The same checkout is used on a Windows workstation and on a Linux host.
In this template checkout, `SCRIPTS` is empty until the first script is
added (section 6).
- **1.3** The same tool runs on a Windows workstation and on a Linux
host, either installed with pipx (section 9) or run from a checkout
(SPEC item 3.7.2).
- **1.4** Scripts are written in Python, bash, PowerShell, or batch. The
CLI picks the interpreter from the file suffix and passes arguments,
terminal, and exit code straight through.
- **1.5** The CLI itself uses only the Python standard library. Python
3.11 or newer.
## 2. Layout
- **2.1** Repository tree:
```text
daedalus/
├── bin/
│ ├── daedalus # Linux launcher (bash), no install needed
│ └── daedalus.bat # Windows launcher (cmd / PowerShell)
├── daedalus/ # The CLI package
│ ├── __main__.py # python -m daedalus
│ ├── cli.py # Argument handling, list, path, dispatch
│ ├── registry.py # SCRIPTS: name -> Script
│ ├── runners.py # Suffix -> interpreter command
│ ├── daedalus # Linux launcher (bash), no install needed
│ └── daedalus.bat # Windows launcher (cmd / PowerShell)
├── daedalus/ # The CLI package
│ ├── __init__.py # PACKAGE_DIR, REPO_ROOT, LIBRARY_ROOT, checkout imports
│ ├── __main__.py # python -m daedalus
│ ├── cli.py # Argument handling, list, path, dispatch
│ ├── registry.py # Script, SCRIPTS, category helpers
│ ├── runners.py # Suffix -> interpreter command, run
│ ├── run_python_script.py # Child entry for .py scripts (logging, argv)
│ ├── daedalus_logging/ # File + console logging (section 11)
│ └── tests/
├── python/
│ ├── src/<name>/ # Python scripts, one folder per script
│ └── tests/ # pytest tests for python/src
├── bash/ # *.sh scripts
│ └── tests/ # pytest tests that run them
├── powershell/ # *.ps1 scripts
├── batch/ # *.bat scripts
├── ci/ # Sanity, syntax, and lint checks, with tests
├── autotest.py # Runs every check (local and CI)
├── README.md # Script catalog (user-facing)
├── INSTALL.md # Install, usage, development
└── SPEC.md # This file
│ ├── src/<name>/ # Python scripts, one folder per script
│ └── tests/ # pytest tests for python/src, .ps1, .bat
├── bash/ # *.sh scripts
│ └── tests/ # pytest tests that run them
├── powershell/ # *.ps1 scripts
├── batch/ # *.bat scripts (CRLF)
├── ci/ # sanity.py, checks.py, with tests
├── autotest.py # Runs every check (local and CI)
├── pyproject.toml # Package, version, entry point, package data
├── requirements.txt # Test and lint tools
├── .gitea/workflows/ci.yml # CI job that runs autotest.py
├── .gitattributes # LF everywhere, CRLF for *.bat
├── .editorconfig # UTF-8, LF, final newline, 4 spaces (2 for yml/json/md)
├── CHANGELOG.md # Release notes, newest first
├── README.md # Script catalog (user-facing)
├── INSTALL.md # Install, usage, development
├── AGENTS.md # Rules for AI agents
└── SPEC.md # This file
```
- **2.2** Three paths are computed in `daedalus/__init__.py`:
- **2.2.1** `PACKAGE_DIR`: the resolved folder of the `daedalus`
package.
- **2.2.2** `REPO_ROOT`: the parent of `PACKAGE_DIR` (the checkout root,
or `site-packages` in an install).
- **2.2.3** `LIBRARY_ROOT`: see SPEC item 9.3.
## 3. Command line
| Call | Result | Exit code |
| --- | --- | --- |
| `daedalus <name> [args...]` | Runs the registered script with `args` passed through unchanged | The script's exit code |
| `daedalus`, `daedalus -l`, `daedalus --list` | Prints every registered name, its type, and description | 0 |
| `daedalus -h`, `daedalus --help` | Prints usage | 0 |
| `daedalus <category> [category...]` | Prints the version line, then scripts that have **every** listed category | 0 |
| `daedalus <category>… <name> [args...]` | Runs `<name>` when it has every category prefix; same pass-through as the flat form | The script's exit code |
| `daedalus`, `daedalus -l`, `daedalus --list` | Prints the version line, then every script (SPEC item 3.3) | 0 |
| `daedalus -h`, `daedalus --help` | Prints usage (SPEC item 3.8) | 0 |
| `daedalus --path <name>` | Prints the absolute path of the script | 0 |
| `daedalus --version` | Prints the installed version, or `project.version` from `pyproject.toml` in a checkout | 0 |
| Unknown name | Prints an error and close matches to stderr | 2 |
| Script file missing, or no interpreter found | Prints the reason to stderr | 2 |
| `daedalus --version` | Prints `daedalus <version>` (SPEC item 3.9) | 0 |
| Unknown option, category, or script; script not in category | Prints an error and close matches to stderr | 2 |
| Script file missing, unsupported suffix, or no interpreter found | Prints the reason to stderr | 2 |
- Everything after `<name>` belongs to the script, including `--help`:
`daedalus my_tool --help` shows the help of `my_tool`.
- The script runs in the caller's current directory, with the caller's
stdin, stdout, and stderr, so interactive scripts work.
- Ctrl+C reaches the script. daedalus waits for it to exit and returns its
exit code.
- **3.1** Everything after the script name belongs to the script,
including `--help`; these forms show the script help:
- **3.1.1** `daedalus sample_script --help`
- **3.1.2** `daedalus general sample_script --help`
- **3.1.3** `daedalus media convert_video --help`
- **3.2** The flat form `daedalus <name>` stays valid; categories are
optional grouping.
- **3.3** Listing format (full list and category-filtered list):
- **3.3.1** First line `Daedalus Version: <version>`. When at least one
script is listed, a blank line and the table follow; otherwise nothing
follows.
- **3.3.2** The table has a header row `script`, `type`, `categories`,
then a rule row of `-` per column. Columns are left-aligned, padded to
the widest cell (header included) and separated by two spaces. The
`type` column is at least 4 wide.
- **3.3.3** One row per script, sorted by name. `type` is the file
suffix without the dot (`py`, `sh`, `ps1`, `bat`). `categories` is the
labels joined with a comma and a space (`media, linux`).
- **3.3.4** Under each row, the description is wrapped to the terminal
width (at least 40 columns; 80 when the width cannot be read), every
line indented by 8 spaces. Long words and hyphenated words are not
split.
- **3.3.5** Color: when enabled, the padded name cell is green
(`ESC[32m`), the type cell yellow (`ESC[33m`), the categories cell
bright blue (`ESC[94m`), each followed by `ESC[0m`.
- **3.3.6** Color is enabled when `NO_COLOR` is empty or unset and
either `FORCE_COLOR` is non-empty or stdout is a terminal. A non-empty
`NO_COLOR` always wins.
Ways to run the CLI:
```text
Daedalus Version: 0.1.0
- Install it with pip (section 9). `daedalus` is then on `PATH`
everywhere, with every registered script bundled.
- `bin/daedalus` (Linux) or `bin\daedalus.bat` (Windows). Add `bin/` to
`PATH` to call plain `daedalus`. Needs no install.
- `python -m daedalus` from the repository root.
script type categories
-------------------- ---- ------------------
sample_script_name py general, linux
sample script description
```
- **3.4** A token is treated as a **category** only when it matches a
category in use and is **not** also a registered script name (names and
category labels must not collide, SPEC item 4.3).
- **3.5** The script runs in the caller's current directory, with the
caller's stdin, stdout, and stderr, so interactive scripts work.
- **3.6** Ctrl+C reaches the script. daedalus waits for it to exit and
returns its exit code.
- **3.7** Ways to run the CLI:
- **3.7.1** Install it with pipx or pip (section 9). `daedalus` is then
on `PATH` everywhere, with every registered script bundled.
- **3.7.2** `bin/daedalus` (Linux) or `bin\daedalus.bat` (Windows). Add
`bin/` to `PATH` to call plain `daedalus`. Needs no install.
- **3.7.2.1** `bin/daedalus` resolves its own real path (following
symlinks), prepends the checkout root to `PYTHONPATH`, and `exec`s
`python3 -m daedalus "$@"`.
- **3.7.2.2** `bin\daedalus.bat` prepends `<bin>\..` to `PYTHONPATH`,
runs `python -m daedalus %*`, and exits with its `ERRORLEVEL`.
- **3.7.3** `python -m daedalus` from the repository root.
- **3.7.4** The installed command is the entry point
`daedalus = daedalus.cli:main`.
- **3.8** `--help` prints this text:
```text
usage: daedalus <name> [args...] run a script; args go to the script
daedalus <category> [category...] list scripts (AND when multiple)
daedalus <category>… <name> [args...] run when script has every category
daedalus [-l | --list] list all registered scripts
daedalus --path <name> print the path of a script
daedalus --version print the installed version
daedalus -h | --help show this help
```
- **3.9** The version is resolved in this order:
- **3.9.1** The installed distribution metadata for `daedalus`.
- **3.9.2** `project.version` from `REPO_ROOT/pyproject.toml` (a
checkout that is not installed).
- **3.9.3** The string `unknown`.
- **3.10** Argument resolution. Options are recognized only as the first
token:
```text
dispatch(argv):
if argv is empty or argv[0] in ("-l", "--list"): list(all scripts); return 0
if argv[0] in ("-h", "--help"): print usage; return 0
if argv[0] == "--version": print "daedalus " + version; return 0
if argv[0] == "--path":
require exactly one more token, else usage error
print absolute path of lookup(argv[1]) # file need not exist
return 0
if argv[0] starts with "-": usage error "unknown option"
prefix = leading tokens while is_category_label(token)
rest = remaining tokens
if prefix and rest is empty:
scripts = scripts having every label in prefix
if none: usage error "no scripts for categories"
list(scripts); return 0
if prefix: script = lookup_in(prefix, rest[0]); return run(script, rest[1:])
return run(lookup(argv[0]), argv[1:])
main(argv = sys.argv[1:]):
try dispatch(argv)
on usage or runner error: print "daedalus: <message>" to stderr; return 2
```
- **3.11** Error messages (each printed as `daedalus: <message>`):
| Case | Message |
| --- | --- |
| `--path` without exactly one name | `usage: daedalus --path <name>` |
| Unknown option | `unknown option '<opt>'; run 'daedalus --help'` |
| Unknown script, close matches exist | `unknown script '<name>'; did you mean: <m1>, <m2>` |
| Unknown script, no close match | `unknown script '<name>'; run 'daedalus --list'` |
| Categories match no script | `no scripts for categories: <a>, <b>` |
| Unknown script after a category prefix | `unknown script '<name>' for '<a b>'; did you mean: …` or `…; run 'daedalus <a b>'` |
| Script lacks a prefix category | `script '<name>' has categories '<x, y>', not [<missing>]; run 'daedalus <name>' or pick another prefix` |
- **3.12** Close matches are up to 3 names with similarity ratio at least
0.6 (Python `difflib.get_close_matches` defaults). After a category
prefix, only scripts in those categories are candidates.
## 4. Registry
`daedalus/registry.py` holds one dictionary:
`daedalus/registry.py` holds one dictionary. In this template checkout it
is empty until the first script is added:
```python
SCRIPTS: dict[str, Script] = {
"my_tool": Script(
path="python/src/my_tool/my_tool.py",
description="One-line summary for daedalus --list",
"<name>": Script(
path="python/src/<name>/<name>.py",
description="One-line summary",
categories="general",
),
"<other_name>": Script(
path="bash/<other_name>.sh",
description="One-line summary",
categories=("category_a", "category_b"),
),
}
```
In this template checkout, `SCRIPTS` is empty until you add the first
script.
- `Script` is a frozen dataclass with `path` (repo-relative, forward
slashes) and `description` (one line).
- The key is the command name: lowercase letters, digits, `_`, and `-`,
starting with a letter.
- Names that start with `-` are reserved for daedalus options.
- The runner is chosen from the suffix of `path`. No other field selects it.
- Tests enforce, for every entry: the name is valid, the file exists, the
suffix has a runner, and the description is not empty.
- **4.1** `Script` is a frozen (immutable) record:
- **4.1.1** `path` is relative to `LIBRARY_ROOT`, with forward slashes.
- **4.1.2** `description` is one line.
- **4.1.3** `categories` is one or more labels, default `general`. A
string becomes a one-item tuple, a list becomes a tuple, and an empty
list or tuple is rejected with `ValueError`.
- **4.1.4** `absolute_path` is `LIBRARY_ROOT / path`.
- **4.1.5** `categories_display` is the labels joined with a comma and
a space.
- **4.2** The key is the command name and matches `^[a-z][a-z0-9_-]*$`.
- **4.3** Each category label uses the same pattern as script names. No
script name may equal any label in `used_categories()` (so
`daedalus media` is unambiguous).
- **4.4** Helpers:
- **4.4.1** `used_categories()`: the set of every label of every script.
- **4.4.2** `is_category_label(token)`: false when `token` is a script
name; otherwise true when it is in `used_categories()`.
- **4.4.3** `scripts_matching_categories(labels)`: scripts that have
every label (AND), sorted by name; all scripts when `labels` is empty.
- **4.5** Leading argv tokens that are category labels form a prefix; the
script must include **every** prefix label in `categories`.
- **4.6** Names that start with `-` are reserved for daedalus options.
- **4.7** The runner is chosen from the suffix of `path`. No other field
selects it.
- **4.8** Tests enforce, for every entry:
- **4.8.1** The name matches the pattern.
- **4.8.2** The file exists, and `path` has no backslash, does not start
with `/`, and resolves inside `LIBRARY_ROOT`.
- **4.8.3** The suffix has a runner.
- **4.8.4** The description is non-empty and has no newline.
- **4.8.5** Categories are non-empty and every label matches the
pattern; no script name equals a label.
- **4.8.6** `README.md` contains ``| `<name>` |``.
## 5. Runners
| Suffix | Command | Platforms |
| --- | --- | --- |
| `.py` | `<python> <path> args...` (`DAEDALUS_PYTHON` or daedalus's interpreter) | All |
| `.py` | `<python> <PACKAGE_DIR>/run_python_script.py <path> args...` | All |
| `.sh` | `<bash> <path> args...` | All with bash |
| `.ps1` | `<pwsh or powershell> -NoProfile -ExecutionPolicy Bypass -File <path> args...` | All with PowerShell |
| `.bat` | `cmd /c <path> args...` | Windows only |
| `.bat` | `<COMSPEC or cmd.exe> /c <path> args...` | Windows only |
- Python scripts get `python/src` prepended to `PYTHONPATH`, so they can
import shared modules from there.
- Set `DAEDALUS_PYTHON` to a virtual-environment interpreter when a
script needs packages that are not in the pipx venv.
- bash lookup order: `DAEDALUS_BASH`, then Git Bash next to `git` on
Windows (`<git root>/bin/bash.exe`), then `bash` on `PATH`. On Windows,
`bash` on `PATH` is often the WSL launcher, which cannot open Windows
paths, so Git Bash comes first.
- PowerShell lookup order: `pwsh`, then `powershell`.
- A missing interpreter or a `.bat` on a non-Windows host is an error with a
clear message (exit code 2), never a silent skip.
- **5.1** Building the command, in order:
- **5.1.1** Look up the runner by the lowercased suffix. None found:
error `unsupported script type '<suffix>': <path>`.
- **5.1.2** The file must exist: else error `script not found: <path>`.
- **5.1.3** Find the interpreter (SPEC items 5.3 to 5.6), then append
the script arguments unchanged.
- **5.2** `<python>` is `DAEDALUS_PYTHON` when set and non-empty, else
the interpreter running daedalus. Set it to a virtual-environment
interpreter when a script needs packages that are not in the pipx
environment daedalus was installed into.
- **5.3** bash lookup order:
- **5.3.1** `DAEDALUS_BASH` when set and non-empty.
- **5.3.2** On Windows only: resolve `git` on `PATH`, then use
`<git root>/bin/bash.exe` when that file exists (`git` lives in
`<git root>/cmd/`).
- **5.3.3** `bash` on `PATH`. On Windows, `bash` on `PATH` is often the
WSL launcher, which cannot open Windows paths, so Git Bash comes first.
- **5.3.4** None: error `bash not found; install it or set DAEDALUS_BASH`.
- **5.4** PowerShell lookup order: `pwsh`, then `powershell` on `PATH`.
None: error `PowerShell not found; install pwsh`.
- **5.5** `.bat` on a non-Windows host: error
`<file name>: .bat scripts run on Windows only`.
- **5.6** Every error above exits daedalus with code 2 (SPEC item 3.10),
never a silent skip.
- **5.7** Running:
- **5.7.1** The child gets the caller's environment, with `REPO_ROOT`
(the folder that contains the `daedalus` package) prepended to
`PYTHONPATH` using the platform path separator. This applies to every
script type, so Python code started from a shell script can also
import `daedalus.*` (section 10).
- **5.7.2** The child inherits the current directory and the terminal
(stdin, stdout, stderr are not redirected).
- **5.7.3** daedalus waits for the child and returns its exit code.
When Ctrl+C interrupts the wait, it waits again and returns the
child's exit code.
- **5.8** `run_python_script.py <script.py> [args...]` runs in the child
process:
- **5.8.1** No arguments: print
`usage: run_python_script.py <script.py> [args...]` to stderr, exit 2.
- **5.8.2** Script path (resolved) is not a file: print
`script not found: <path>` to stderr, exit 2.
- **5.8.3** Put the parent of its own folder and its own folder at the
front of `sys.path` (when missing), so `import daedalus` works.
- **5.8.4** Unless `DAEDALUS_NO_LOGGING` is non-empty, configure
logging (section 11) with the console level from `DAEDALUS_LOGLEVEL`
(default `INFO`).
- **5.8.5** Set `sys.argv` to `[<script path>, *args]`, execute the file
as `__main__`, and restore `sys.argv` afterwards.
- **5.8.6** Exit code: 0 when the script returns normally or calls
`sys.exit()` / `sys.exit(None)`; the integer for `sys.exit(<int>)`;
1 for any other exit value. An uncaught exception prints a traceback
and exits 1.
## 6. Adding a script
- Put the file in the folder for its type (section 2). A Python script gets
its own folder: `python/src/<name>/<name>.py`.
- The script must run on its own (`python python/src/<name>/<name>.py`)
without daedalus. Python scripts expose `main(argv: list[str] | None)`
and end with `if __name__ == "__main__": sys.exit(main())`.
- Add one entry to `SCRIPTS` in `daedalus/registry.py`.
- Add a row to the "Available scripts" table in `README.md`, and a short
section with usage when the script takes arguments. A test fails when a
registered name has no row.
- Add tests:
- Python: `python/tests/test_<name>.py`, importing with
`from <name> import <name>`.
- bash: `bash/tests/test_<name>.py`, running the script with
- **6.1** Put the file in the folder for its type (section 2). A Python
script gets its own folder: `python/src/<name>/<name>.py`. The command
name, file stem, and (for Python) folder name are the same string.
- **6.2** The script must run without daedalus.
- **6.2.1** Python scripts expose `main(argv: list[str] | None)`.
- **6.2.2** They end with
`if __name__ == "__main__": sys.exit(main())`.
- **6.2.3** Shared Python code is imported as `daedalus.<folder>`
(section 10). A script that does so runs on its own when daedalus is
installed or the checkout root is on `PYTHONPATH`.
- **6.2.4** The script never imports the CLI modules (`cli`, `registry`,
`runners`) and never reads `LIBRARY_ROOT`.
- **6.3** Add one entry to `SCRIPTS` in `daedalus/registry.py` (set
`categories` when the script belongs to media, linux, or other
groups).
- **6.4** Add a row to the "Available scripts" table in `README.md`
(columns Script, Type, Categories, What it does), and a short section
with usage when the script takes arguments.
- **6.4.1** A test fails when a registered name has no row.
- **6.5** A new `python/src/<folder>/` is added to `packages` and
`package-dir` in `pyproject.toml` as `daedalus.<folder>` (SPEC item
9.4), so installed code can import it.
- **6.6** Add tests:
- **6.6.1** Python: `python/tests/test_<name>.py`, importing with
`from daedalus.<name> import <name>`.
- **6.6.2** bash: `bash/tests/test_<name>.py`, running the script with
`subprocess` and skipping when bash is missing.
- PowerShell and batch: a test under `python/tests/` that runs the
script, skipped when the interpreter or Windows is missing.
- Run `python autotest.py`. It must pass before pushing.
- **6.6.3** PowerShell and batch: a test under `python/tests/` that runs
the script, skipped when the interpreter or Windows is missing.
- **6.7** Bump `version` in `pyproject.toml` and add a `CHANGELOG.md`
entry.
- **6.8** Run `python autotest.py`. It must pass before pushing.
## 7. Quality gates
`python autotest.py` runs every check; CI runs the same command.
- **7.1** `python autotest.py [CHECK...]` runs the named checks (default:
all). An unknown name is an argument error (exit 2) listing the unknown
names.
- **7.2** Selected checks run concurrently in threads. Each returns an
error count; 0 is a pass. Failures are logged as
`ERROR [FAIL] <check detail>` while the checks run.
- **7.3** After all finish, autotest prints a summary and exits 1 when any
check failed, else 0:
```text
########################################
autotest results
########################################
[ PASS ] 0.42s sanity
[ FAIL ] 12.07s pytest
```
- **7.4** Run it before every commit; do not commit unless every check
passes ([AGENTS.md](./AGENTS.md) section 2.3).
- **7.5** File discovery walks the repository, skipping folders named
`.git`, `.venv`, `venv`, `.cache`, `.pytest_cache`, `.ruff_cache`,
`__pycache__`, `htmlcov`, `node_modules`, `build`, `dist`. Suffixes
match exactly.
| Check | Rule |
| --- | --- |
| `sanity` | Changed files: line endings (section 8), no private keys, under 2 MB |
| `sanity` | Changed files (SPEC item 7.6): line endings (section 8), no private key, at most 2,000,000 bytes |
| `python_syntax` | Every `*.py` parses |
| `pycodestyle` | PEP 8 with the repository's ignore list (`PYCODESTYLE_IGNORE` in `ci/checks.py`) |
| `shellcheck` | Every `*.sh`, `*.bash`, and suffixless file with a shell shebang |
| `json`, `yaml` | Every file loads |
| `markdown` | markdownlint, MD013 and MD033 disabled |
| `pytest` | All tests pass; coverage of `ci`, `daedalus`, and `python/src` at least `MIN_COVERAGE_PERCENT` |
| `pycodestyle` | PEP 8, ignoring `E501 E121 E123 E126 E226 E24 E704 W503 W504`; error count is the violation count |
| `shellcheck` | `shellcheck --external-sources` on every `*.sh`, `*.bash`, and suffixless file whose first line matches `^#!.*\b(ba\|da\|k\|z)?sh\b`; a missing `shellcheck` fails each file |
| `json` | Every `*.json` parses as UTF-8 JSON |
| `yaml` | Every document in every `*.yaml` / `*.yml` loads (safe loader) |
| `markdown` | PyMarkdown with the front-matter extension on every `*.md`; MD013 and MD033 ignored |
| `pytest` | `python -m pytest --cov=ci --cov=daedalus --cov=python/src --cov-report=term-missing:skip-covered --cov-fail-under=<MIN_COVERAGE_PERCENT>` in a subprocess |
- **7.6** Changed files for `sanity`:
- **7.6.1** Base ref: the first that resolves to a commit of
`SANITY_DIFF_BASE`, `origin/<target branch>`, `origin/main`,
`origin/master`.
- **7.6.2** With a base: files added, copied, modified, or renamed
against it (`git diff --name-only --diff-filter=ACMR <ref>`), plus
untracked files not ignored.
- **7.6.3** Without a base: every tracked and untracked, not ignored
file.
- **7.6.4** Deleted paths are skipped.
- **7.7** Sanity rules per file:
- **7.7.1** A file containing a NUL byte is binary and skips the line
ending rule.
- **7.7.2** `*.bat` fails when any `\n` is not preceded by `\r`. Any
other file fails when it contains `\r`.
- **7.7.3** A file fails when it contains
`-----BEGIN <uppercase words> PRIVATE KEY-----`.
- **7.8** `MIN_COVERAGE_PERCENT` in `autotest.py` is 75 and only goes up.
- **7.9** pytest configuration in `pyproject.toml`: test paths
`ci/tests`, `daedalus/tests`, `python/tests`, `bash/tests`; `.` on the
Python path; coverage omits `*/tests/*` and `*/__main__.py`.
- **7.10** CI (`.gitea/workflows/ci.yml`) runs one job on pull request
pipelines and on the default branch: a Python 3.11 image with git and
shellcheck, full git history (for SPEC item 7.6),
`pip install -r requirements.txt`, then `python autotest.py`. The
sanity diff base is passed as `SANITY_DIFF_BASE`.
- **7.11** `requirements.txt` holds the tools: `pytest`, `pytest-cov`,
`pycodestyle`, `pymarkdownlnt`, `pyyaml`, `shellcheck-py`, `setuptools`.
## 8. Conventions
- Line endings are LF everywhere except `*.bat`, which must be CRLF:
cmd.exe mishandles labels and `goto` in LF-only batch files.
`.gitattributes` enforces both on checkout.
- Python targets 3.11. Standard library imports use `import module`.
Logging uses lazy `%` formatting.
- Markdown lists use dashes.
- `bin/daedalus` and every `*.sh` are committed executable:
- **8.1** Line endings are LF everywhere except `*.bat`, which must be
CRLF.
- **8.1.1** cmd.exe mishandles labels and `goto` in LF-only batch files.
- **8.1.2** `.gitattributes` enforces both on checkout.
- **8.2** Python targets 3.11. Standard library imports use `import module`.
- **8.3** Logging uses lazy `%` formatting.
- **8.6** Markdown lists use dashes.
- **8.7** `bin/daedalus` and every `*.sh` are committed executable:
`git add --chmod=+x <file>`.
- `MIN_COVERAGE_PERCENT` in `autotest.py` only goes up. Raise it when
tests are added.
- **8.8** `MIN_COVERAGE_PERCENT` in `autotest.py` only goes up. Raise it
when tests are added.
- **8.9** `CHANGELOG.md` follows Keep a Changelog, newest release first,
with release numbers equal to `version` in `pyproject.toml`.
## 9. Packaging
`pyproject.toml` builds one wheel that holds the CLI and every script.
- **9.1** `pyproject.toml` builds one wheel (setuptools 68 or newer) named
`daedalus` that holds the CLI and every script. It requires Python
3.11 or newer.
| Command | Use |
| --- | --- |
| `pipx install git+ssh://git@example.com/your-org/daedalus.git` | Own isolated environment, `daedalus` on `PATH` (recommended; works on Debian 12, where plain `pip install` is blocked) |
| `pipx install git+https://example.com/your-org/daedalus.git` | Own isolated environment, `daedalus` on `PATH` (recommended; works on Debian 12, where plain `pip install` is blocked) |
| `pip install --user .` | From a checkout, into the user site |
| `pip install -e .` | Development: the installed command runs the scripts in the checkout |
| `pipx upgrade daedalus` / `pip install --upgrade .` | Pick up new scripts |
The script folders are copied into the wheel under
`daedalus/_library/`, with the same relative paths as in the repository:
- **9.2** The script folders are copied into the wheel under
`daedalus/_library/`, with the same relative paths as in the repository:
| Repository folder | Installed folder | Files |
| --- | --- | --- |
| `python/src/` | `daedalus/_library/python/src/` | Everything except `__pycache__` and `*.pyc` |
| `python/src/` | `daedalus/_library/python/src/` | Everything except `__pycache__`, `*.pyc`, `.gitkeep` |
| `bash/` | `daedalus/_library/bash/` | `*.sh` |
| `powershell/` | `daedalus/_library/powershell/` | `*.ps1` |
| `batch/` | `daedalus/_library/batch/` | `*.bat` |
- `LIBRARY_ROOT` in `daedalus/__init__.py` is `daedalus/_library` when
that folder exists (a normal install), otherwise the repository root (a
checkout or an editable install). Registry paths resolve against it, so
`SCRIPTS` entries are the same in both cases.
- Tests under `*/tests/` are not packaged.
- The CLI has no runtime dependencies. Test and lint tools stay in
`requirements.txt`.
- A test builds the wheel and fails when any registered script is missing
from it.
- Bump `version` in `pyproject.toml` when scripts or the CLI change, so
`pipx upgrade` picks them up.
- **9.3** `LIBRARY_ROOT` is `PACKAGE_DIR/_library` when that folder exists
(a normal install), otherwise `REPO_ROOT` (a checkout or an editable
install).
- **9.3.1** Registry paths resolve against it, so `SCRIPTS` entries are
the same in both cases.
- **9.4** Each `python/src/<folder>/` is also installed as the package
`daedalus.<folder>` (an explicit `packages` and `package-dir` entry per
folder). `daedalus` and `daedalus.daedalus_logging` are packages too.
The wheel therefore holds each Python script twice: the `_library` copy
is what the CLI runs, the `daedalus.<folder>` copy is what imports
resolve to (section 10).
- **9.5** Tests under `*/tests/`, bytecode, and `.gitkeep` are not
packaged.
- **9.6** Runtime dependencies: none; the CLI uses only the standard
library. A script that needs third-party packages says so in its own
help text and fails with a clear message when they are missing;
`DAEDALUS_PYTHON` (SPEC item 5.2) points it at their environment.
Test and lint tools stay in `requirements.txt`.
- **9.7** Tests build the wheel from a copy of the checkout and check:
- **9.7.1** Every registered script is at `daedalus/_library/<path>`.
- **9.7.2** No `/tests/`, `*.pyc`, or `.gitkeep` entry is present.
- **9.7.3** The entry point `daedalus = daedalus.cli:main` is declared.
- **9.7.4** Unpacked on `PYTHONPATH`, `daedalus --path <name>` points
into `_library` for every registered `<name>`, `--list` lists it,
`<name> --help` runs, and every registered `daedalus.<folder>`
import resolves.
- **9.8** Bump `version` in `pyproject.toml` when scripts or the CLI
change, so `pipx upgrade` picks them up.
## 10. Implementation plan
## 10. Library imports
- **10.1** Library Python code imports shared modules under the
`daedalus` namespace, never as top-level names:
`from daedalus.<folder> import <module>`.
- **10.2** In an install, `daedalus.<folder>` is a real installed package
(SPEC item 9.4).
- **10.3** In a checkout, `import daedalus` registers every direct
subfolder of `REPO_ROOT/python/src` (except `__pycache__`) as a module
`daedalus.<folder>` whose package search path is that folder, unless a
module of that name is already loaded. Files are not copied.
```text
on import daedalus:
if REPO_ROOT/python/src is a folder:
for each subfolder F, sorted by name:
if "daedalus.F" not already loaded:
create empty package "daedalus.F" with search path [F]
```
- **10.4** `REPO_ROOT` on `PYTHONPATH` (SPEC item 5.7.1) and the
`sys.path` setup of `run_python_script.py` (SPEC item 5.8.3) make
`import daedalus` succeed in the child, under `DAEDALUS_PYTHON` too.
## 11. Logging for Python scripts
- **11.1** `daedalus.daedalus_logging.configure_logging()` is called by
`run_python_script.py` before the script starts (SPEC item 5.8.4).
Library scripts may call it too. It replaces any existing root handlers.
- **11.2** Root logger level is `DEBUG`, with two handlers:
- **11.2.1** File handler, appending UTF-8 to `~/.daedalus/debug.log`
(parent folders created), all levels, format
`%(asctime)s|%(levelname)-5.5s|%(module)s.%(funcName)s:%(lineno)d:%(message)s`,
date format `%Y-%m-%d %H:%M:%S`.
- **11.2.2** Console handler on stderr, level from
`DAEDALUS_LOGLEVEL` (a level name such as `DEBUG`, `INFO`,
`WARNING`), format `%(levelname)-5.5s|%(message)s`.
- **11.4** Console color is on when `NO_COLOR` is empty or unset and
either `FORCE_COLOR` is non-empty or stderr is a terminal.
- **11.5** With color on:
- **11.5.1** `WARNING` lines are yellow (`ESC[33m`), `ERROR` bold red
(`ESC[1;31m`), `CRITICAL` bold bright red (`ESC[1;91m`); the whole
line is colored and reset at the end. `DEBUG` and `INFO` are plain.
- **11.5.2** An IPv4 address in square brackets, `[10.0.0.1]` or a
range `[10.0.0.1 - 10.0.0.9]`, has its first address in bold blue
(`ESC[1;34m`); the line's level color resumes after it.
- **11.6** `configure_logging` accepts overrides for the log file path,
console level, file level, and color, and returns the resolved log
file path.
## 12. Implementation plan
| Step | Work | Status |
| --- | --- | --- |
@@ -225,3 +583,7 @@ The script folders are copied into the wheel under
| 12 | `daedalus --version` | Done |
| 13 | Tests: `LIBRARY_ROOT` selection, wheel contains every registered script, installed CLI | Done |
| 14 | README: install with pipx or pip | Done |
| 15 | Categories in `Script`, CLI `daedalus <category>` and `daedalus <category> <name>` | Done |
| 16 | `run_python_script.py` and `daedalus_logging` (section 11) | Done |
| 17 | `daedalus.<folder>` imports in checkouts and installs (section 10) | Done |
| 18 | Listing table with version line, wrapped descriptions, and color (SPEC item 3.3) | Done |