daedalus 0.1.0: scriptman spec update ported - categories, new listing format with colors, run_python_script child entry, daedalus_logging, daedalus.<folder> namespace imports; SPEC rewritten with stable item numbers
Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -1,8 +1,15 @@
|
||||
# daedalus specification
|
||||
|
||||
How this repository is laid out, how scripts are registered and run, and
|
||||
the rules for maintaining it. Change this file first when the behavior
|
||||
changes, then change the code and tests to match.
|
||||
the rules for maintaining it. This file alone is enough to rebuild the
|
||||
tool: it states every observable behavior (arguments, output, exit codes,
|
||||
environment variables, files written, packaging, checks). Change this file
|
||||
first when the behavior changes, then change the code and tests to match.
|
||||
|
||||
This checkout is an **empty template**: the CLI and quality gates are
|
||||
ready; add scripts per section 6.
|
||||
|
||||
<!-- toc-start: headings 2..6 -->
|
||||
|
||||
- [1. Purpose](#1-purpose)
|
||||
- [2. Layout](#2-layout)
|
||||
@@ -13,201 +20,552 @@ changes, then change the code and tests to match.
|
||||
- [7. Quality gates](#7-quality-gates)
|
||||
- [8. Conventions](#8-conventions)
|
||||
- [9. Packaging](#9-packaging)
|
||||
- [10. Implementation plan](#10-implementation-plan)
|
||||
- [10. Library imports](#10-library-imports)
|
||||
- [11. Logging for Python scripts](#11-logging-for-python-scripts)
|
||||
- [12. Implementation plan](#12-implementation-plan)
|
||||
|
||||
<!-- toc-end -->
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This repository is an **empty template**: the CLI and quality gates are
|
||||
ready; add scripts per section 6. A filled checkout keeps loose scripts in
|
||||
one repository, each with tests, and runs any of them through one command:
|
||||
- **1.1** Every heading and list item has a stable id. Cite an item as
|
||||
`SPEC item 3.7.1`. New items get the next free number; numbers of
|
||||
removed items are not reused.
|
||||
- **1.2** daedalus keeps loose personal scripts in one repository, each
|
||||
with tests, and runs any of them through one command:
|
||||
|
||||
```text
|
||||
daedalus <name> [args...]
|
||||
daedalus <category> [category...] list scripts (AND when multiple)
|
||||
daedalus <category>… <name> [args...] run when the script has every category
|
||||
```
|
||||
|
||||
The same checkout is used on a Windows workstation and on a Linux host.
|
||||
In this template checkout, `SCRIPTS` is empty until the first script is
|
||||
added (section 6).
|
||||
|
||||
- **1.3** The same tool runs on a Windows workstation and on a Linux
|
||||
host, either installed with pipx (section 9) or run from a checkout
|
||||
(SPEC item 3.7.2).
|
||||
- **1.4** Scripts are written in Python, bash, PowerShell, or batch. The
|
||||
CLI picks the interpreter from the file suffix and passes arguments,
|
||||
terminal, and exit code straight through.
|
||||
- **1.5** The CLI itself uses only the Python standard library. Python
|
||||
3.11 or newer.
|
||||
|
||||
## 2. Layout
|
||||
|
||||
- **2.1** Repository tree:
|
||||
|
||||
```text
|
||||
daedalus/
|
||||
├── bin/
|
||||
│ ├── daedalus # Linux launcher (bash), no install needed
|
||||
│ └── daedalus.bat # Windows launcher (cmd / PowerShell)
|
||||
├── daedalus/ # The CLI package
|
||||
│ ├── __main__.py # python -m daedalus
|
||||
│ ├── cli.py # Argument handling, list, path, dispatch
|
||||
│ ├── registry.py # SCRIPTS: name -> Script
|
||||
│ ├── runners.py # Suffix -> interpreter command
|
||||
│ ├── daedalus # Linux launcher (bash), no install needed
|
||||
│ └── daedalus.bat # Windows launcher (cmd / PowerShell)
|
||||
├── daedalus/ # The CLI package
|
||||
│ ├── __init__.py # PACKAGE_DIR, REPO_ROOT, LIBRARY_ROOT, checkout imports
|
||||
│ ├── __main__.py # python -m daedalus
|
||||
│ ├── cli.py # Argument handling, list, path, dispatch
|
||||
│ ├── registry.py # Script, SCRIPTS, category helpers
|
||||
│ ├── runners.py # Suffix -> interpreter command, run
|
||||
│ ├── run_python_script.py # Child entry for .py scripts (logging, argv)
|
||||
│ ├── daedalus_logging/ # File + console logging (section 11)
|
||||
│ └── tests/
|
||||
├── python/
|
||||
│ ├── src/<name>/ # Python scripts, one folder per script
|
||||
│ └── tests/ # pytest tests for python/src
|
||||
├── bash/ # *.sh scripts
|
||||
│ └── tests/ # pytest tests that run them
|
||||
├── powershell/ # *.ps1 scripts
|
||||
├── batch/ # *.bat scripts
|
||||
├── ci/ # Sanity, syntax, and lint checks, with tests
|
||||
├── autotest.py # Runs every check (local and CI)
|
||||
├── README.md # Script catalog (user-facing)
|
||||
├── INSTALL.md # Install, usage, development
|
||||
└── SPEC.md # This file
|
||||
│ ├── src/<name>/ # Python scripts, one folder per script
|
||||
│ └── tests/ # pytest tests for python/src, .ps1, .bat
|
||||
├── bash/ # *.sh scripts
|
||||
│ └── tests/ # pytest tests that run them
|
||||
├── powershell/ # *.ps1 scripts
|
||||
├── batch/ # *.bat scripts (CRLF)
|
||||
├── ci/ # sanity.py, checks.py, with tests
|
||||
├── autotest.py # Runs every check (local and CI)
|
||||
├── pyproject.toml # Package, version, entry point, package data
|
||||
├── requirements.txt # Test and lint tools
|
||||
├── .gitea/workflows/ci.yml # CI job that runs autotest.py
|
||||
├── .gitattributes # LF everywhere, CRLF for *.bat
|
||||
├── .editorconfig # UTF-8, LF, final newline, 4 spaces (2 for yml/json/md)
|
||||
├── CHANGELOG.md # Release notes, newest first
|
||||
├── README.md # Script catalog (user-facing)
|
||||
├── INSTALL.md # Install, usage, development
|
||||
├── AGENTS.md # Rules for AI agents
|
||||
└── SPEC.md # This file
|
||||
```
|
||||
|
||||
- **2.2** Three paths are computed in `daedalus/__init__.py`:
|
||||
- **2.2.1** `PACKAGE_DIR`: the resolved folder of the `daedalus`
|
||||
package.
|
||||
- **2.2.2** `REPO_ROOT`: the parent of `PACKAGE_DIR` (the checkout root,
|
||||
or `site-packages` in an install).
|
||||
- **2.2.3** `LIBRARY_ROOT`: see SPEC item 9.3.
|
||||
|
||||
## 3. Command line
|
||||
|
||||
| Call | Result | Exit code |
|
||||
| --- | --- | --- |
|
||||
| `daedalus <name> [args...]` | Runs the registered script with `args` passed through unchanged | The script's exit code |
|
||||
| `daedalus`, `daedalus -l`, `daedalus --list` | Prints every registered name, its type, and description | 0 |
|
||||
| `daedalus -h`, `daedalus --help` | Prints usage | 0 |
|
||||
| `daedalus <category> [category...]` | Prints the version line, then scripts that have **every** listed category | 0 |
|
||||
| `daedalus <category>… <name> [args...]` | Runs `<name>` when it has every category prefix; same pass-through as the flat form | The script's exit code |
|
||||
| `daedalus`, `daedalus -l`, `daedalus --list` | Prints the version line, then every script (SPEC item 3.3) | 0 |
|
||||
| `daedalus -h`, `daedalus --help` | Prints usage (SPEC item 3.8) | 0 |
|
||||
| `daedalus --path <name>` | Prints the absolute path of the script | 0 |
|
||||
| `daedalus --version` | Prints the installed version, or `project.version` from `pyproject.toml` in a checkout | 0 |
|
||||
| Unknown name | Prints an error and close matches to stderr | 2 |
|
||||
| Script file missing, or no interpreter found | Prints the reason to stderr | 2 |
|
||||
| `daedalus --version` | Prints `daedalus <version>` (SPEC item 3.9) | 0 |
|
||||
| Unknown option, category, or script; script not in category | Prints an error and close matches to stderr | 2 |
|
||||
| Script file missing, unsupported suffix, or no interpreter found | Prints the reason to stderr | 2 |
|
||||
|
||||
- Everything after `<name>` belongs to the script, including `--help`:
|
||||
`daedalus my_tool --help` shows the help of `my_tool`.
|
||||
- The script runs in the caller's current directory, with the caller's
|
||||
stdin, stdout, and stderr, so interactive scripts work.
|
||||
- Ctrl+C reaches the script. daedalus waits for it to exit and returns its
|
||||
exit code.
|
||||
- **3.1** Everything after the script name belongs to the script,
|
||||
including `--help`; these forms show the script help:
|
||||
- **3.1.1** `daedalus sample_script --help`
|
||||
- **3.1.2** `daedalus general sample_script --help`
|
||||
- **3.1.3** `daedalus media convert_video --help`
|
||||
- **3.2** The flat form `daedalus <name>` stays valid; categories are
|
||||
optional grouping.
|
||||
- **3.3** Listing format (full list and category-filtered list):
|
||||
- **3.3.1** First line `Daedalus Version: <version>`. When at least one
|
||||
script is listed, a blank line and the table follow; otherwise nothing
|
||||
follows.
|
||||
- **3.3.2** The table has a header row `script`, `type`, `categories`,
|
||||
then a rule row of `-` per column. Columns are left-aligned, padded to
|
||||
the widest cell (header included) and separated by two spaces. The
|
||||
`type` column is at least 4 wide.
|
||||
- **3.3.3** One row per script, sorted by name. `type` is the file
|
||||
suffix without the dot (`py`, `sh`, `ps1`, `bat`). `categories` is the
|
||||
labels joined with a comma and a space (`media, linux`).
|
||||
- **3.3.4** Under each row, the description is wrapped to the terminal
|
||||
width (at least 40 columns; 80 when the width cannot be read), every
|
||||
line indented by 8 spaces. Long words and hyphenated words are not
|
||||
split.
|
||||
- **3.3.5** Color: when enabled, the padded name cell is green
|
||||
(`ESC[32m`), the type cell yellow (`ESC[33m`), the categories cell
|
||||
bright blue (`ESC[94m`), each followed by `ESC[0m`.
|
||||
- **3.3.6** Color is enabled when `NO_COLOR` is empty or unset and
|
||||
either `FORCE_COLOR` is non-empty or stdout is a terminal. A non-empty
|
||||
`NO_COLOR` always wins.
|
||||
|
||||
Ways to run the CLI:
|
||||
```text
|
||||
Daedalus Version: 0.1.0
|
||||
|
||||
- Install it with pip (section 9). `daedalus` is then on `PATH`
|
||||
everywhere, with every registered script bundled.
|
||||
- `bin/daedalus` (Linux) or `bin\daedalus.bat` (Windows). Add `bin/` to
|
||||
`PATH` to call plain `daedalus`. Needs no install.
|
||||
- `python -m daedalus` from the repository root.
|
||||
script type categories
|
||||
-------------------- ---- ------------------
|
||||
sample_script_name py general, linux
|
||||
sample script description
|
||||
```
|
||||
|
||||
- **3.4** A token is treated as a **category** only when it matches a
|
||||
category in use and is **not** also a registered script name (names and
|
||||
category labels must not collide, SPEC item 4.3).
|
||||
- **3.5** The script runs in the caller's current directory, with the
|
||||
caller's stdin, stdout, and stderr, so interactive scripts work.
|
||||
- **3.6** Ctrl+C reaches the script. daedalus waits for it to exit and
|
||||
returns its exit code.
|
||||
- **3.7** Ways to run the CLI:
|
||||
- **3.7.1** Install it with pipx or pip (section 9). `daedalus` is then
|
||||
on `PATH` everywhere, with every registered script bundled.
|
||||
- **3.7.2** `bin/daedalus` (Linux) or `bin\daedalus.bat` (Windows). Add
|
||||
`bin/` to `PATH` to call plain `daedalus`. Needs no install.
|
||||
- **3.7.2.1** `bin/daedalus` resolves its own real path (following
|
||||
symlinks), prepends the checkout root to `PYTHONPATH`, and `exec`s
|
||||
`python3 -m daedalus "$@"`.
|
||||
- **3.7.2.2** `bin\daedalus.bat` prepends `<bin>\..` to `PYTHONPATH`,
|
||||
runs `python -m daedalus %*`, and exits with its `ERRORLEVEL`.
|
||||
- **3.7.3** `python -m daedalus` from the repository root.
|
||||
- **3.7.4** The installed command is the entry point
|
||||
`daedalus = daedalus.cli:main`.
|
||||
- **3.8** `--help` prints this text:
|
||||
|
||||
```text
|
||||
usage: daedalus <name> [args...] run a script; args go to the script
|
||||
daedalus <category> [category...] list scripts (AND when multiple)
|
||||
daedalus <category>… <name> [args...] run when script has every category
|
||||
daedalus [-l | --list] list all registered scripts
|
||||
daedalus --path <name> print the path of a script
|
||||
daedalus --version print the installed version
|
||||
daedalus -h | --help show this help
|
||||
```
|
||||
|
||||
- **3.9** The version is resolved in this order:
|
||||
- **3.9.1** The installed distribution metadata for `daedalus`.
|
||||
- **3.9.2** `project.version` from `REPO_ROOT/pyproject.toml` (a
|
||||
checkout that is not installed).
|
||||
- **3.9.3** The string `unknown`.
|
||||
- **3.10** Argument resolution. Options are recognized only as the first
|
||||
token:
|
||||
|
||||
```text
|
||||
dispatch(argv):
|
||||
if argv is empty or argv[0] in ("-l", "--list"): list(all scripts); return 0
|
||||
if argv[0] in ("-h", "--help"): print usage; return 0
|
||||
if argv[0] == "--version": print "daedalus " + version; return 0
|
||||
if argv[0] == "--path":
|
||||
require exactly one more token, else usage error
|
||||
print absolute path of lookup(argv[1]) # file need not exist
|
||||
return 0
|
||||
if argv[0] starts with "-": usage error "unknown option"
|
||||
prefix = leading tokens while is_category_label(token)
|
||||
rest = remaining tokens
|
||||
if prefix and rest is empty:
|
||||
scripts = scripts having every label in prefix
|
||||
if none: usage error "no scripts for categories"
|
||||
list(scripts); return 0
|
||||
if prefix: script = lookup_in(prefix, rest[0]); return run(script, rest[1:])
|
||||
return run(lookup(argv[0]), argv[1:])
|
||||
|
||||
main(argv = sys.argv[1:]):
|
||||
try dispatch(argv)
|
||||
on usage or runner error: print "daedalus: <message>" to stderr; return 2
|
||||
```
|
||||
|
||||
- **3.11** Error messages (each printed as `daedalus: <message>`):
|
||||
|
||||
| Case | Message |
|
||||
| --- | --- |
|
||||
| `--path` without exactly one name | `usage: daedalus --path <name>` |
|
||||
| Unknown option | `unknown option '<opt>'; run 'daedalus --help'` |
|
||||
| Unknown script, close matches exist | `unknown script '<name>'; did you mean: <m1>, <m2>` |
|
||||
| Unknown script, no close match | `unknown script '<name>'; run 'daedalus --list'` |
|
||||
| Categories match no script | `no scripts for categories: <a>, <b>` |
|
||||
| Unknown script after a category prefix | `unknown script '<name>' for '<a b>'; did you mean: …` or `…; run 'daedalus <a b>'` |
|
||||
| Script lacks a prefix category | `script '<name>' has categories '<x, y>', not [<missing>]; run 'daedalus <name>' or pick another prefix` |
|
||||
|
||||
- **3.12** Close matches are up to 3 names with similarity ratio at least
|
||||
0.6 (Python `difflib.get_close_matches` defaults). After a category
|
||||
prefix, only scripts in those categories are candidates.
|
||||
|
||||
## 4. Registry
|
||||
|
||||
`daedalus/registry.py` holds one dictionary:
|
||||
`daedalus/registry.py` holds one dictionary. In this template checkout it
|
||||
is empty until the first script is added:
|
||||
|
||||
```python
|
||||
SCRIPTS: dict[str, Script] = {
|
||||
"my_tool": Script(
|
||||
path="python/src/my_tool/my_tool.py",
|
||||
description="One-line summary for daedalus --list",
|
||||
"<name>": Script(
|
||||
path="python/src/<name>/<name>.py",
|
||||
description="One-line summary",
|
||||
categories="general",
|
||||
),
|
||||
"<other_name>": Script(
|
||||
path="bash/<other_name>.sh",
|
||||
description="One-line summary",
|
||||
categories=("category_a", "category_b"),
|
||||
),
|
||||
}
|
||||
```
|
||||
|
||||
In this template checkout, `SCRIPTS` is empty until you add the first
|
||||
script.
|
||||
|
||||
- `Script` is a frozen dataclass with `path` (repo-relative, forward
|
||||
slashes) and `description` (one line).
|
||||
- The key is the command name: lowercase letters, digits, `_`, and `-`,
|
||||
starting with a letter.
|
||||
- Names that start with `-` are reserved for daedalus options.
|
||||
- The runner is chosen from the suffix of `path`. No other field selects it.
|
||||
- Tests enforce, for every entry: the name is valid, the file exists, the
|
||||
suffix has a runner, and the description is not empty.
|
||||
- **4.1** `Script` is a frozen (immutable) record:
|
||||
- **4.1.1** `path` is relative to `LIBRARY_ROOT`, with forward slashes.
|
||||
- **4.1.2** `description` is one line.
|
||||
- **4.1.3** `categories` is one or more labels, default `general`. A
|
||||
string becomes a one-item tuple, a list becomes a tuple, and an empty
|
||||
list or tuple is rejected with `ValueError`.
|
||||
- **4.1.4** `absolute_path` is `LIBRARY_ROOT / path`.
|
||||
- **4.1.5** `categories_display` is the labels joined with a comma and
|
||||
a space.
|
||||
- **4.2** The key is the command name and matches `^[a-z][a-z0-9_-]*$`.
|
||||
- **4.3** Each category label uses the same pattern as script names. No
|
||||
script name may equal any label in `used_categories()` (so
|
||||
`daedalus media` is unambiguous).
|
||||
- **4.4** Helpers:
|
||||
- **4.4.1** `used_categories()`: the set of every label of every script.
|
||||
- **4.4.2** `is_category_label(token)`: false when `token` is a script
|
||||
name; otherwise true when it is in `used_categories()`.
|
||||
- **4.4.3** `scripts_matching_categories(labels)`: scripts that have
|
||||
every label (AND), sorted by name; all scripts when `labels` is empty.
|
||||
- **4.5** Leading argv tokens that are category labels form a prefix; the
|
||||
script must include **every** prefix label in `categories`.
|
||||
- **4.6** Names that start with `-` are reserved for daedalus options.
|
||||
- **4.7** The runner is chosen from the suffix of `path`. No other field
|
||||
selects it.
|
||||
- **4.8** Tests enforce, for every entry:
|
||||
- **4.8.1** The name matches the pattern.
|
||||
- **4.8.2** The file exists, and `path` has no backslash, does not start
|
||||
with `/`, and resolves inside `LIBRARY_ROOT`.
|
||||
- **4.8.3** The suffix has a runner.
|
||||
- **4.8.4** The description is non-empty and has no newline.
|
||||
- **4.8.5** Categories are non-empty and every label matches the
|
||||
pattern; no script name equals a label.
|
||||
- **4.8.6** `README.md` contains ``| `<name>` |``.
|
||||
|
||||
## 5. Runners
|
||||
|
||||
| Suffix | Command | Platforms |
|
||||
| --- | --- | --- |
|
||||
| `.py` | `<python> <path> args...` (`DAEDALUS_PYTHON` or daedalus's interpreter) | All |
|
||||
| `.py` | `<python> <PACKAGE_DIR>/run_python_script.py <path> args...` | All |
|
||||
| `.sh` | `<bash> <path> args...` | All with bash |
|
||||
| `.ps1` | `<pwsh or powershell> -NoProfile -ExecutionPolicy Bypass -File <path> args...` | All with PowerShell |
|
||||
| `.bat` | `cmd /c <path> args...` | Windows only |
|
||||
| `.bat` | `<COMSPEC or cmd.exe> /c <path> args...` | Windows only |
|
||||
|
||||
- Python scripts get `python/src` prepended to `PYTHONPATH`, so they can
|
||||
import shared modules from there.
|
||||
- Set `DAEDALUS_PYTHON` to a virtual-environment interpreter when a
|
||||
script needs packages that are not in the pipx venv.
|
||||
- bash lookup order: `DAEDALUS_BASH`, then Git Bash next to `git` on
|
||||
Windows (`<git root>/bin/bash.exe`), then `bash` on `PATH`. On Windows,
|
||||
`bash` on `PATH` is often the WSL launcher, which cannot open Windows
|
||||
paths, so Git Bash comes first.
|
||||
- PowerShell lookup order: `pwsh`, then `powershell`.
|
||||
- A missing interpreter or a `.bat` on a non-Windows host is an error with a
|
||||
clear message (exit code 2), never a silent skip.
|
||||
- **5.1** Building the command, in order:
|
||||
- **5.1.1** Look up the runner by the lowercased suffix. None found:
|
||||
error `unsupported script type '<suffix>': <path>`.
|
||||
- **5.1.2** The file must exist: else error `script not found: <path>`.
|
||||
- **5.1.3** Find the interpreter (SPEC items 5.3 to 5.6), then append
|
||||
the script arguments unchanged.
|
||||
- **5.2** `<python>` is `DAEDALUS_PYTHON` when set and non-empty, else
|
||||
the interpreter running daedalus. Set it to a virtual-environment
|
||||
interpreter when a script needs packages that are not in the pipx
|
||||
environment daedalus was installed into.
|
||||
- **5.3** bash lookup order:
|
||||
- **5.3.1** `DAEDALUS_BASH` when set and non-empty.
|
||||
- **5.3.2** On Windows only: resolve `git` on `PATH`, then use
|
||||
`<git root>/bin/bash.exe` when that file exists (`git` lives in
|
||||
`<git root>/cmd/`).
|
||||
- **5.3.3** `bash` on `PATH`. On Windows, `bash` on `PATH` is often the
|
||||
WSL launcher, which cannot open Windows paths, so Git Bash comes first.
|
||||
- **5.3.4** None: error `bash not found; install it or set DAEDALUS_BASH`.
|
||||
- **5.4** PowerShell lookup order: `pwsh`, then `powershell` on `PATH`.
|
||||
None: error `PowerShell not found; install pwsh`.
|
||||
- **5.5** `.bat` on a non-Windows host: error
|
||||
`<file name>: .bat scripts run on Windows only`.
|
||||
- **5.6** Every error above exits daedalus with code 2 (SPEC item 3.10),
|
||||
never a silent skip.
|
||||
- **5.7** Running:
|
||||
- **5.7.1** The child gets the caller's environment, with `REPO_ROOT`
|
||||
(the folder that contains the `daedalus` package) prepended to
|
||||
`PYTHONPATH` using the platform path separator. This applies to every
|
||||
script type, so Python code started from a shell script can also
|
||||
import `daedalus.*` (section 10).
|
||||
- **5.7.2** The child inherits the current directory and the terminal
|
||||
(stdin, stdout, stderr are not redirected).
|
||||
- **5.7.3** daedalus waits for the child and returns its exit code.
|
||||
When Ctrl+C interrupts the wait, it waits again and returns the
|
||||
child's exit code.
|
||||
- **5.8** `run_python_script.py <script.py> [args...]` runs in the child
|
||||
process:
|
||||
- **5.8.1** No arguments: print
|
||||
`usage: run_python_script.py <script.py> [args...]` to stderr, exit 2.
|
||||
- **5.8.2** Script path (resolved) is not a file: print
|
||||
`script not found: <path>` to stderr, exit 2.
|
||||
- **5.8.3** Put the parent of its own folder and its own folder at the
|
||||
front of `sys.path` (when missing), so `import daedalus` works.
|
||||
- **5.8.4** Unless `DAEDALUS_NO_LOGGING` is non-empty, configure
|
||||
logging (section 11) with the console level from `DAEDALUS_LOGLEVEL`
|
||||
(default `INFO`).
|
||||
- **5.8.5** Set `sys.argv` to `[<script path>, *args]`, execute the file
|
||||
as `__main__`, and restore `sys.argv` afterwards.
|
||||
- **5.8.6** Exit code: 0 when the script returns normally or calls
|
||||
`sys.exit()` / `sys.exit(None)`; the integer for `sys.exit(<int>)`;
|
||||
1 for any other exit value. An uncaught exception prints a traceback
|
||||
and exits 1.
|
||||
|
||||
## 6. Adding a script
|
||||
|
||||
- Put the file in the folder for its type (section 2). A Python script gets
|
||||
its own folder: `python/src/<name>/<name>.py`.
|
||||
- The script must run on its own (`python python/src/<name>/<name>.py`)
|
||||
without daedalus. Python scripts expose `main(argv: list[str] | None)`
|
||||
and end with `if __name__ == "__main__": sys.exit(main())`.
|
||||
- Add one entry to `SCRIPTS` in `daedalus/registry.py`.
|
||||
- Add a row to the "Available scripts" table in `README.md`, and a short
|
||||
section with usage when the script takes arguments. A test fails when a
|
||||
registered name has no row.
|
||||
- Add tests:
|
||||
- Python: `python/tests/test_<name>.py`, importing with
|
||||
`from <name> import <name>`.
|
||||
- bash: `bash/tests/test_<name>.py`, running the script with
|
||||
- **6.1** Put the file in the folder for its type (section 2). A Python
|
||||
script gets its own folder: `python/src/<name>/<name>.py`. The command
|
||||
name, file stem, and (for Python) folder name are the same string.
|
||||
- **6.2** The script must run without daedalus.
|
||||
- **6.2.1** Python scripts expose `main(argv: list[str] | None)`.
|
||||
- **6.2.2** They end with
|
||||
`if __name__ == "__main__": sys.exit(main())`.
|
||||
- **6.2.3** Shared Python code is imported as `daedalus.<folder>`
|
||||
(section 10). A script that does so runs on its own when daedalus is
|
||||
installed or the checkout root is on `PYTHONPATH`.
|
||||
- **6.2.4** The script never imports the CLI modules (`cli`, `registry`,
|
||||
`runners`) and never reads `LIBRARY_ROOT`.
|
||||
- **6.3** Add one entry to `SCRIPTS` in `daedalus/registry.py` (set
|
||||
`categories` when the script belongs to media, linux, or other
|
||||
groups).
|
||||
- **6.4** Add a row to the "Available scripts" table in `README.md`
|
||||
(columns Script, Type, Categories, What it does), and a short section
|
||||
with usage when the script takes arguments.
|
||||
- **6.4.1** A test fails when a registered name has no row.
|
||||
- **6.5** A new `python/src/<folder>/` is added to `packages` and
|
||||
`package-dir` in `pyproject.toml` as `daedalus.<folder>` (SPEC item
|
||||
9.4), so installed code can import it.
|
||||
- **6.6** Add tests:
|
||||
- **6.6.1** Python: `python/tests/test_<name>.py`, importing with
|
||||
`from daedalus.<name> import <name>`.
|
||||
- **6.6.2** bash: `bash/tests/test_<name>.py`, running the script with
|
||||
`subprocess` and skipping when bash is missing.
|
||||
- PowerShell and batch: a test under `python/tests/` that runs the
|
||||
script, skipped when the interpreter or Windows is missing.
|
||||
- Run `python autotest.py`. It must pass before pushing.
|
||||
- **6.6.3** PowerShell and batch: a test under `python/tests/` that runs
|
||||
the script, skipped when the interpreter or Windows is missing.
|
||||
- **6.7** Bump `version` in `pyproject.toml` and add a `CHANGELOG.md`
|
||||
entry.
|
||||
- **6.8** Run `python autotest.py`. It must pass before pushing.
|
||||
|
||||
## 7. Quality gates
|
||||
|
||||
`python autotest.py` runs every check; CI runs the same command.
|
||||
- **7.1** `python autotest.py [CHECK...]` runs the named checks (default:
|
||||
all). An unknown name is an argument error (exit 2) listing the unknown
|
||||
names.
|
||||
- **7.2** Selected checks run concurrently in threads. Each returns an
|
||||
error count; 0 is a pass. Failures are logged as
|
||||
`ERROR [FAIL] <check detail>` while the checks run.
|
||||
- **7.3** After all finish, autotest prints a summary and exits 1 when any
|
||||
check failed, else 0:
|
||||
|
||||
```text
|
||||
########################################
|
||||
autotest results
|
||||
########################################
|
||||
[ PASS ] 0.42s sanity
|
||||
[ FAIL ] 12.07s pytest
|
||||
```
|
||||
|
||||
- **7.4** Run it before every commit; do not commit unless every check
|
||||
passes ([AGENTS.md](./AGENTS.md) section 2.3).
|
||||
- **7.5** File discovery walks the repository, skipping folders named
|
||||
`.git`, `.venv`, `venv`, `.cache`, `.pytest_cache`, `.ruff_cache`,
|
||||
`__pycache__`, `htmlcov`, `node_modules`, `build`, `dist`. Suffixes
|
||||
match exactly.
|
||||
|
||||
| Check | Rule |
|
||||
| --- | --- |
|
||||
| `sanity` | Changed files: line endings (section 8), no private keys, under 2 MB |
|
||||
| `sanity` | Changed files (SPEC item 7.6): line endings (section 8), no private key, at most 2,000,000 bytes |
|
||||
| `python_syntax` | Every `*.py` parses |
|
||||
| `pycodestyle` | PEP 8 with the repository's ignore list (`PYCODESTYLE_IGNORE` in `ci/checks.py`) |
|
||||
| `shellcheck` | Every `*.sh`, `*.bash`, and suffixless file with a shell shebang |
|
||||
| `json`, `yaml` | Every file loads |
|
||||
| `markdown` | markdownlint, MD013 and MD033 disabled |
|
||||
| `pytest` | All tests pass; coverage of `ci`, `daedalus`, and `python/src` at least `MIN_COVERAGE_PERCENT` |
|
||||
| `pycodestyle` | PEP 8, ignoring `E501 E121 E123 E126 E226 E24 E704 W503 W504`; error count is the violation count |
|
||||
| `shellcheck` | `shellcheck --external-sources` on every `*.sh`, `*.bash`, and suffixless file whose first line matches `^#!.*\b(ba\|da\|k\|z)?sh\b`; a missing `shellcheck` fails each file |
|
||||
| `json` | Every `*.json` parses as UTF-8 JSON |
|
||||
| `yaml` | Every document in every `*.yaml` / `*.yml` loads (safe loader) |
|
||||
| `markdown` | PyMarkdown with the front-matter extension on every `*.md`; MD013 and MD033 ignored |
|
||||
| `pytest` | `python -m pytest --cov=ci --cov=daedalus --cov=python/src --cov-report=term-missing:skip-covered --cov-fail-under=<MIN_COVERAGE_PERCENT>` in a subprocess |
|
||||
|
||||
- **7.6** Changed files for `sanity`:
|
||||
- **7.6.1** Base ref: the first that resolves to a commit of
|
||||
`SANITY_DIFF_BASE`, `origin/<target branch>`, `origin/main`,
|
||||
`origin/master`.
|
||||
- **7.6.2** With a base: files added, copied, modified, or renamed
|
||||
against it (`git diff --name-only --diff-filter=ACMR <ref>`), plus
|
||||
untracked files not ignored.
|
||||
- **7.6.3** Without a base: every tracked and untracked, not ignored
|
||||
file.
|
||||
- **7.6.4** Deleted paths are skipped.
|
||||
- **7.7** Sanity rules per file:
|
||||
- **7.7.1** A file containing a NUL byte is binary and skips the line
|
||||
ending rule.
|
||||
- **7.7.2** `*.bat` fails when any `\n` is not preceded by `\r`. Any
|
||||
other file fails when it contains `\r`.
|
||||
- **7.7.3** A file fails when it contains
|
||||
`-----BEGIN <uppercase words> PRIVATE KEY-----`.
|
||||
- **7.8** `MIN_COVERAGE_PERCENT` in `autotest.py` is 75 and only goes up.
|
||||
- **7.9** pytest configuration in `pyproject.toml`: test paths
|
||||
`ci/tests`, `daedalus/tests`, `python/tests`, `bash/tests`; `.` on the
|
||||
Python path; coverage omits `*/tests/*` and `*/__main__.py`.
|
||||
- **7.10** CI (`.gitea/workflows/ci.yml`) runs one job on pull request
|
||||
pipelines and on the default branch: a Python 3.11 image with git and
|
||||
shellcheck, full git history (for SPEC item 7.6),
|
||||
`pip install -r requirements.txt`, then `python autotest.py`. The
|
||||
sanity diff base is passed as `SANITY_DIFF_BASE`.
|
||||
- **7.11** `requirements.txt` holds the tools: `pytest`, `pytest-cov`,
|
||||
`pycodestyle`, `pymarkdownlnt`, `pyyaml`, `shellcheck-py`, `setuptools`.
|
||||
|
||||
## 8. Conventions
|
||||
|
||||
- Line endings are LF everywhere except `*.bat`, which must be CRLF:
|
||||
cmd.exe mishandles labels and `goto` in LF-only batch files.
|
||||
`.gitattributes` enforces both on checkout.
|
||||
- Python targets 3.11. Standard library imports use `import module`.
|
||||
Logging uses lazy `%` formatting.
|
||||
- Markdown lists use dashes.
|
||||
- `bin/daedalus` and every `*.sh` are committed executable:
|
||||
- **8.1** Line endings are LF everywhere except `*.bat`, which must be
|
||||
CRLF.
|
||||
- **8.1.1** cmd.exe mishandles labels and `goto` in LF-only batch files.
|
||||
- **8.1.2** `.gitattributes` enforces both on checkout.
|
||||
- **8.2** Python targets 3.11. Standard library imports use `import module`.
|
||||
- **8.3** Logging uses lazy `%` formatting.
|
||||
- **8.6** Markdown lists use dashes.
|
||||
- **8.7** `bin/daedalus` and every `*.sh` are committed executable:
|
||||
`git add --chmod=+x <file>`.
|
||||
- `MIN_COVERAGE_PERCENT` in `autotest.py` only goes up. Raise it when
|
||||
tests are added.
|
||||
- **8.8** `MIN_COVERAGE_PERCENT` in `autotest.py` only goes up. Raise it
|
||||
when tests are added.
|
||||
- **8.9** `CHANGELOG.md` follows Keep a Changelog, newest release first,
|
||||
with release numbers equal to `version` in `pyproject.toml`.
|
||||
|
||||
## 9. Packaging
|
||||
|
||||
`pyproject.toml` builds one wheel that holds the CLI and every script.
|
||||
- **9.1** `pyproject.toml` builds one wheel (setuptools 68 or newer) named
|
||||
`daedalus` that holds the CLI and every script. It requires Python
|
||||
3.11 or newer.
|
||||
|
||||
| Command | Use |
|
||||
| --- | --- |
|
||||
| `pipx install git+ssh://git@example.com/your-org/daedalus.git` | Own isolated environment, `daedalus` on `PATH` (recommended; works on Debian 12, where plain `pip install` is blocked) |
|
||||
| `pipx install git+https://example.com/your-org/daedalus.git` | Own isolated environment, `daedalus` on `PATH` (recommended; works on Debian 12, where plain `pip install` is blocked) |
|
||||
| `pip install --user .` | From a checkout, into the user site |
|
||||
| `pip install -e .` | Development: the installed command runs the scripts in the checkout |
|
||||
| `pipx upgrade daedalus` / `pip install --upgrade .` | Pick up new scripts |
|
||||
|
||||
The script folders are copied into the wheel under
|
||||
`daedalus/_library/`, with the same relative paths as in the repository:
|
||||
- **9.2** The script folders are copied into the wheel under
|
||||
`daedalus/_library/`, with the same relative paths as in the repository:
|
||||
|
||||
| Repository folder | Installed folder | Files |
|
||||
| --- | --- | --- |
|
||||
| `python/src/` | `daedalus/_library/python/src/` | Everything except `__pycache__` and `*.pyc` |
|
||||
| `python/src/` | `daedalus/_library/python/src/` | Everything except `__pycache__`, `*.pyc`, `.gitkeep` |
|
||||
| `bash/` | `daedalus/_library/bash/` | `*.sh` |
|
||||
| `powershell/` | `daedalus/_library/powershell/` | `*.ps1` |
|
||||
| `batch/` | `daedalus/_library/batch/` | `*.bat` |
|
||||
|
||||
- `LIBRARY_ROOT` in `daedalus/__init__.py` is `daedalus/_library` when
|
||||
that folder exists (a normal install), otherwise the repository root (a
|
||||
checkout or an editable install). Registry paths resolve against it, so
|
||||
`SCRIPTS` entries are the same in both cases.
|
||||
- Tests under `*/tests/` are not packaged.
|
||||
- The CLI has no runtime dependencies. Test and lint tools stay in
|
||||
`requirements.txt`.
|
||||
- A test builds the wheel and fails when any registered script is missing
|
||||
from it.
|
||||
- Bump `version` in `pyproject.toml` when scripts or the CLI change, so
|
||||
`pipx upgrade` picks them up.
|
||||
- **9.3** `LIBRARY_ROOT` is `PACKAGE_DIR/_library` when that folder exists
|
||||
(a normal install), otherwise `REPO_ROOT` (a checkout or an editable
|
||||
install).
|
||||
- **9.3.1** Registry paths resolve against it, so `SCRIPTS` entries are
|
||||
the same in both cases.
|
||||
- **9.4** Each `python/src/<folder>/` is also installed as the package
|
||||
`daedalus.<folder>` (an explicit `packages` and `package-dir` entry per
|
||||
folder). `daedalus` and `daedalus.daedalus_logging` are packages too.
|
||||
The wheel therefore holds each Python script twice: the `_library` copy
|
||||
is what the CLI runs, the `daedalus.<folder>` copy is what imports
|
||||
resolve to (section 10).
|
||||
- **9.5** Tests under `*/tests/`, bytecode, and `.gitkeep` are not
|
||||
packaged.
|
||||
- **9.6** Runtime dependencies: none; the CLI uses only the standard
|
||||
library. A script that needs third-party packages says so in its own
|
||||
help text and fails with a clear message when they are missing;
|
||||
`DAEDALUS_PYTHON` (SPEC item 5.2) points it at their environment.
|
||||
Test and lint tools stay in `requirements.txt`.
|
||||
- **9.7** Tests build the wheel from a copy of the checkout and check:
|
||||
- **9.7.1** Every registered script is at `daedalus/_library/<path>`.
|
||||
- **9.7.2** No `/tests/`, `*.pyc`, or `.gitkeep` entry is present.
|
||||
- **9.7.3** The entry point `daedalus = daedalus.cli:main` is declared.
|
||||
- **9.7.4** Unpacked on `PYTHONPATH`, `daedalus --path <name>` points
|
||||
into `_library` for every registered `<name>`, `--list` lists it,
|
||||
`<name> --help` runs, and every registered `daedalus.<folder>`
|
||||
import resolves.
|
||||
- **9.8** Bump `version` in `pyproject.toml` when scripts or the CLI
|
||||
change, so `pipx upgrade` picks them up.
|
||||
|
||||
## 10. Implementation plan
|
||||
## 10. Library imports
|
||||
|
||||
- **10.1** Library Python code imports shared modules under the
|
||||
`daedalus` namespace, never as top-level names:
|
||||
`from daedalus.<folder> import <module>`.
|
||||
- **10.2** In an install, `daedalus.<folder>` is a real installed package
|
||||
(SPEC item 9.4).
|
||||
- **10.3** In a checkout, `import daedalus` registers every direct
|
||||
subfolder of `REPO_ROOT/python/src` (except `__pycache__`) as a module
|
||||
`daedalus.<folder>` whose package search path is that folder, unless a
|
||||
module of that name is already loaded. Files are not copied.
|
||||
|
||||
```text
|
||||
on import daedalus:
|
||||
if REPO_ROOT/python/src is a folder:
|
||||
for each subfolder F, sorted by name:
|
||||
if "daedalus.F" not already loaded:
|
||||
create empty package "daedalus.F" with search path [F]
|
||||
```
|
||||
|
||||
- **10.4** `REPO_ROOT` on `PYTHONPATH` (SPEC item 5.7.1) and the
|
||||
`sys.path` setup of `run_python_script.py` (SPEC item 5.8.3) make
|
||||
`import daedalus` succeed in the child, under `DAEDALUS_PYTHON` too.
|
||||
|
||||
## 11. Logging for Python scripts
|
||||
|
||||
- **11.1** `daedalus.daedalus_logging.configure_logging()` is called by
|
||||
`run_python_script.py` before the script starts (SPEC item 5.8.4).
|
||||
Library scripts may call it too. It replaces any existing root handlers.
|
||||
- **11.2** Root logger level is `DEBUG`, with two handlers:
|
||||
- **11.2.1** File handler, appending UTF-8 to `~/.daedalus/debug.log`
|
||||
(parent folders created), all levels, format
|
||||
`%(asctime)s|%(levelname)-5.5s|%(module)s.%(funcName)s:%(lineno)d:%(message)s`,
|
||||
date format `%Y-%m-%d %H:%M:%S`.
|
||||
- **11.2.2** Console handler on stderr, level from
|
||||
`DAEDALUS_LOGLEVEL` (a level name such as `DEBUG`, `INFO`,
|
||||
`WARNING`), format `%(levelname)-5.5s|%(message)s`.
|
||||
- **11.4** Console color is on when `NO_COLOR` is empty or unset and
|
||||
either `FORCE_COLOR` is non-empty or stderr is a terminal.
|
||||
- **11.5** With color on:
|
||||
- **11.5.1** `WARNING` lines are yellow (`ESC[33m`), `ERROR` bold red
|
||||
(`ESC[1;31m`), `CRITICAL` bold bright red (`ESC[1;91m`); the whole
|
||||
line is colored and reset at the end. `DEBUG` and `INFO` are plain.
|
||||
- **11.5.2** An IPv4 address in square brackets, `[10.0.0.1]` or a
|
||||
range `[10.0.0.1 - 10.0.0.9]`, has its first address in bold blue
|
||||
(`ESC[1;34m`); the line's level color resumes after it.
|
||||
- **11.6** `configure_logging` accepts overrides for the log file path,
|
||||
console level, file level, and color, and returns the resolved log
|
||||
file path.
|
||||
|
||||
## 12. Implementation plan
|
||||
|
||||
| Step | Work | Status |
|
||||
| --- | --- | --- |
|
||||
@@ -225,3 +583,7 @@ The script folders are copied into the wheel under
|
||||
| 12 | `daedalus --version` | Done |
|
||||
| 13 | Tests: `LIBRARY_ROOT` selection, wheel contains every registered script, installed CLI | Done |
|
||||
| 14 | README: install with pipx or pip | Done |
|
||||
| 15 | Categories in `Script`, CLI `daedalus <category>` and `daedalus <category> <name>` | Done |
|
||||
| 16 | `run_python_script.py` and `daedalus_logging` (section 11) | Done |
|
||||
| 17 | `daedalus.<folder>` imports in checkouts and installs (section 10) | Done |
|
||||
| 18 | Listing table with version line, wrapped descriptions, and color (SPEC item 3.3) | Done |
|
||||
|
||||
Reference in New Issue
Block a user