202 lines
4.5 KiB
Markdown
202 lines
4.5 KiB
Markdown
# shout
|
|
|
|
A transcript-based shell integration test runner.
|
|
|
|
## Format
|
|
|
|
A `.shout` file is a plain text transcript of a shell session. Lines starting
|
|
with `$ ` are commands. Everything after — until the next `$` or end of file
|
|
— is the expected output (stdout and stderr combined).
|
|
|
|
```
|
|
$ dev new "add auth"
|
|
created draft 1 "add auth"
|
|
|
|
$ dev save
|
|
saved draft 1 (v1)
|
|
```
|
|
|
|
Blank lines within expected output are significant. Trailing newline on the
|
|
file is ignored.
|
|
|
|
### Comments
|
|
|
|
`#` after a command is a comment and is stripped before execution. Comments
|
|
in expected output are matched literally.
|
|
|
|
```
|
|
$ dev new "add auth" # create a draft from timeline HEAD
|
|
created draft 1 "add auth"
|
|
```
|
|
|
|
### Wildcards
|
|
|
|
A `...` on its own line in expected output matches any number of lines
|
|
(including zero).
|
|
|
|
```
|
|
$ dev log
|
|
...
|
|
draft 1 "add auth"
|
|
```
|
|
|
|
A `...` inline matches any sequence of characters on that line.
|
|
|
|
```
|
|
$ dev status
|
|
draft 1 "add auth" (v...)
|
|
```
|
|
|
|
### Environment
|
|
|
|
Each `.shout` file runs in a fresh temporary directory. The directory is
|
|
created before the first command and removed after the last (unless
|
|
`--keep` is passed).
|
|
|
|
The following environment variables are set for every command:
|
|
|
|
| Variable | Value |
|
|
|---|---|
|
|
| `HOME` | the temp directory |
|
|
| `PATH` | prepended with the directory containing the binary under test |
|
|
| `CUE_DIR` | the temp directory |
|
|
|
|
All other environment variables are inherited from the host unless explicitly
|
|
cleared with `--clean-env`.
|
|
|
|
### Setup blocks
|
|
|
|
Commands before the first blank line + command sequence are run as setup and
|
|
their output is not asserted.
|
|
|
|
Alternatively, a `# ---` line separates setup from the test body explicitly:
|
|
|
|
```
|
|
$ export TOKEN=abc
|
|
$ cd myproject
|
|
# ---
|
|
$ dev status
|
|
on timeline @ change 0
|
|
```
|
|
|
|
### Exit codes
|
|
|
|
By default, a non-zero exit code fails the test regardless of output. To
|
|
assert a specific exit code, append `[N]` on the last line of expected output:
|
|
|
|
```
|
|
$ dev rm
|
|
error: draft 1 has children. use dev rm -f to cascade.
|
|
[1]
|
|
```
|
|
|
|
`[*]` accepts any non-zero exit code without asserting the value.
|
|
|
|
---
|
|
|
|
## CLI
|
|
|
|
```
|
|
shout [options] [files|dirs...]
|
|
```
|
|
|
|
If no files are given, shout runs all `*.shout` files in the current directory
|
|
and subdirectories. Each command in each shout file is run sequentially
|
|
(unless `--parallel` is passed).
|
|
|
|
### Options
|
|
|
|
| Flag | Description |
|
|
|---|---|
|
|
| `--update` / `-u` | Rewrite expected output in-place with actual output |
|
|
| `--keep` / `-k` | Keep temp directories after run (printed to stderr) |
|
|
| `--clean-env` | Start with empty environment (only `PATH` and `CUE_DIR` set) |
|
|
| `--bin <path>` | Prepend `<path>` to `PATH` instead of auto-detecting |
|
|
| `--timeout <dur>` | Per-command timeout (default: `10s`) |
|
|
| `--verbose` / `-v` | Print each command as it runs |
|
|
| `--parallel` | Run files in parallel (implies all files run regardless of failures) |
|
|
|
|
### Output
|
|
|
|
Passing files print a single `.` per file. Failing files print a unified diff:
|
|
|
|
```
|
|
FAIL tests/auth.shout
|
|
|
|
$ dev rm
|
|
- error: draft 1 has children. use dev rm -f to cascade.
|
|
+ error: draft 1 has dependents. use dev rm -f to cascade.
|
|
[1]
|
|
```
|
|
|
|
Summary line at the end:
|
|
|
|
```
|
|
12 passed, 1 failed in 340ms
|
|
```
|
|
|
|
---
|
|
|
|
## Update mode
|
|
|
|
`--update` rewrites the expected output sections of each `.shout` file with the
|
|
actual output from the run. Commands, comments, and whitespace are preserved.
|
|
Wildcard lines are left in place if the actual output matches them; they are
|
|
only replaced if the match fails.
|
|
|
|
This makes it safe to run `shout --update` routinely after intentional output
|
|
changes — review the diff, commit if correct.
|
|
|
|
---
|
|
|
|
## File layout
|
|
|
|
```
|
|
tests/
|
|
auth.shout
|
|
drafts.shout
|
|
stack.shout
|
|
```
|
|
|
|
No special directory structure is required. `.shout` files can live anywhere.
|
|
|
|
---
|
|
|
|
## Implementation notes
|
|
|
|
- Bun + TypeScript
|
|
- Commands run via `Bun.spawn` with a shell (`/bin/sh -c`)
|
|
- Stdout and stderr merged (same as a terminal)
|
|
- Each command in a file shares a working directory but runs in a fresh
|
|
process — no persistent shell state between commands
|
|
- For persistent state (e.g. `cd`, `export`), users wrap in a shell block or
|
|
use a setup script
|
|
|
|
---
|
|
|
|
## Example
|
|
|
|
```
|
|
$ dev new "add auth"
|
|
created draft 1 "add auth"
|
|
|
|
$ echo 'export function auth() {}' > auth.ts
|
|
$ dev save
|
|
saved draft 1 (v1)
|
|
|
|
$ echo 'export function auth(token: string) {}' > auth.ts
|
|
$ dev save
|
|
saved draft 1 (v2)
|
|
|
|
$ dev status
|
|
draft 1 "add auth" (v2)
|
|
modified: (none)
|
|
|
|
$ dev new "add db"
|
|
created draft 2 "add db"
|
|
|
|
$ dev rm 1
|
|
error: draft 1 has children. use dev rm -f to cascade.
|
|
[1]
|
|
```
|