forked from defunkt/toes
Add comprehensive documentation for releases and deployment
This commit is contained in:
parent
3baf943a53
commit
e5e82f3085
47
CLAUDE.md
47
CLAUDE.md
|
|
@ -23,12 +23,36 @@ Personal web appliance that auto-discovers and runs multiple web apps on your ho
|
|||
## Running
|
||||
|
||||
```bash
|
||||
bun run dev # Hot reload (deletes pub/client/index.js first)
|
||||
bun run start # Production
|
||||
bun run check # Type check
|
||||
bun run test # Tests
|
||||
bun run dev # Hot reload (rebuilds client bundle on change)
|
||||
bun run start # Production (generates templates, then runs server)
|
||||
bun run check # Type check
|
||||
bun run test # Tests
|
||||
bun run build # Build client JS bundle (pub/client/index.js)
|
||||
bun run release # Build release tarball for the Pi
|
||||
```
|
||||
|
||||
## Building & Releasing
|
||||
|
||||
`bun run release` (runs `scripts/release.sh`) produces a self-contained tarball the Pi can install without building anything:
|
||||
|
||||
1. Client JS bundle → `pub/client/index.js`
|
||||
2. Embedded templates → `src/lib/templates.data.ts` (generated from `templates/` by `scripts/embed-templates.ts`)
|
||||
3. Pre-built bare git repos for bundled apps → `dist/repos/*.git` (built by `scripts/build-repos.sh`)
|
||||
4. Cross-compiled CLI binary → `dist/toes` (linux-arm64, built by `scripts/build.ts`)
|
||||
5. Everything staged and packed into `dist/toes-<version>.tar.gz`
|
||||
|
||||
The Pi installer (`install/install.sh`) downloads this tarball, runs `bun install` for the server and all bundled apps (in parallel), copies pre-built repos and CLI binary into place, and starts the systemd service. No git commands, no compilation on the Pi.
|
||||
|
||||
### Key scripts
|
||||
|
||||
- `scripts/build.sh` -- Builds client JS bundle only (used during dev)
|
||||
- `scripts/build-repos.sh` -- Pre-builds bare git repos for bundled apps (excludes node_modules, snapshots, logs)
|
||||
- `scripts/release.sh` -- Full release pipeline: client + templates + repos + CLI → tarball
|
||||
- `scripts/build.ts` -- CLI binary compiler (current platform or `--all` for cross-compile)
|
||||
- `scripts/embed-templates.ts` -- Generates `src/lib/templates.data.ts` from `templates/`
|
||||
- `install/install.sh` -- Pi installer, downloads release tarball and sets everything up
|
||||
- `scripts/remote-install.sh` -- Runs the installer on a remote Pi over SSH
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
|
|
@ -235,6 +259,21 @@ function start(app: App): void {
|
|||
}
|
||||
```
|
||||
|
||||
## Install & Deployment
|
||||
|
||||
The install script (`install/install.sh`) is designed to run on a fresh Pi or as an updater:
|
||||
|
||||
1. Installs system packages (git, fish, avahi-utils, etc.) via apt
|
||||
2. Installs Bun and grants `cap_net_bind_service`
|
||||
3. Downloads and extracts the release tarball into `~/toes`
|
||||
4. Runs `bun install` for the server
|
||||
5. Copies bundled apps to `~/apps/` and runs `bun install` for each (in parallel)
|
||||
6. Copies pre-built bare repos to `~/data/repos/` (for git-based versioning)
|
||||
7. Installs the pre-built CLI binary to `/usr/local/bin/toes`
|
||||
8. Sets up SSH access and the systemd service
|
||||
|
||||
The release tarball URL is configured as `RELEASE_URL` at the top of `install/install.sh`.
|
||||
|
||||
## Writing Apps and Tools
|
||||
|
||||
See `docs/GUIDE.md` for the guide to writing toes apps and tools.
|
||||
|
|
|
|||
85
README.md
85
README.md
|
|
@ -1,42 +1,69 @@
|
|||
# 🐾 Toes
|
||||
|
||||
Toes is a personal web appliance you run on your home network.
|
||||
Personal web appliance you run on your home network.
|
||||
|
||||
Plug it in, turn it on, and forget about the cloud.
|
||||
|
||||
## setup
|
||||
## Development
|
||||
|
||||
Toes runs on a Raspberry Pi. You'll need:
|
||||
```bash
|
||||
bun run dev # Hot reload (rebuilds client bundle on change)
|
||||
bun run start # Production mode
|
||||
bun run check # Type check
|
||||
bun run test # Tests
|
||||
bun run build # Build client JS bundle
|
||||
bun run release # Build a release tarball for the Pi
|
||||
```
|
||||
|
||||
- A Raspberry Pi 5 running the latest Raspberry Pi OS
|
||||
- A `toes` user with passwordless sudo
|
||||
### Releasing
|
||||
|
||||
SSH into your Pi as the `toes` user and run:
|
||||
`bun run release` builds everything the Pi needs into a single tarball:
|
||||
|
||||
1. Client JS bundle (`pub/client/index.js`)
|
||||
2. Embedded templates (`src/lib/templates.data.ts`)
|
||||
3. Pre-built bare git repos for bundled apps (`dist/repos/`)
|
||||
4. Cross-compiled CLI binary for linux-arm64 (`dist/toes`)
|
||||
|
||||
Output: `dist/toes-<version>.tar.gz`
|
||||
|
||||
The Pi does zero building — it untars, runs `bun install`, and starts. Upload the tarball to wherever `RELEASE_URL` in `install/install.sh` points (currently `https://toes.dev/release/latest.tar.gz`).
|
||||
|
||||
### Scripts
|
||||
|
||||
| Script | What it does |
|
||||
|--------|-------------|
|
||||
| `scripts/build.sh` | Builds the client JS bundle into `pub/client/index.js` |
|
||||
| `scripts/build-repos.sh` | Pre-builds bare git repos for bundled apps in `dist/repos/` |
|
||||
| `scripts/release.sh` | Full release: client + templates + repos + CLI → tarball |
|
||||
| `scripts/build.ts` | Builds the CLI binary (current platform or cross-compile) |
|
||||
| `scripts/embed-templates.ts` | Generates `src/lib/templates.data.ts` from `templates/` |
|
||||
| `scripts/setup-ssh.sh` | Configures SSH access for the `cli` user on the Pi |
|
||||
| `scripts/remote-install.sh` | Runs the installer on a remote Pi over SSH |
|
||||
|
||||
## Setup
|
||||
|
||||
Toes runs on a Raspberry Pi 5 with a `toes` user and passwordless sudo.
|
||||
|
||||
```bash
|
||||
curl -fsSL https://toes.dev/install | bash
|
||||
```
|
||||
|
||||
This will:
|
||||
The installer downloads the release tarball, installs bun and system packages, runs `bun install` for the server and all bundled apps (in parallel), copies the pre-built CLI and git repos into place, and starts the systemd service.
|
||||
|
||||
1. Install system dependencies (git, fish shell, networking tools)
|
||||
2. Install Bun and grant it network binding capabilities
|
||||
3. Clone and build the toes server
|
||||
4. Set up bundled apps and tools (clock, code, cron, env, stats)
|
||||
5. Install and enable a systemd service for auto-start
|
||||
Dashboard: `http://<hostname>.local`
|
||||
|
||||
Once complete, visit `http://<hostname>.local` on your local network.
|
||||
## Features
|
||||
|
||||
## features
|
||||
- Hosts Bun/Hype webapps (SSR and SPA)
|
||||
- `git push` Heroku-style deploys
|
||||
- Web dashboard with real-time status, logs, and tools
|
||||
- `toes` CLI (local install or SSH)
|
||||
- Per-app environment variables, cron jobs, metrics
|
||||
- Public sharing via tunnels
|
||||
|
||||
- Effortlessly hosts bun/hype webapps - both SSR and SPA.
|
||||
- `git push`, Heroku-style deploys
|
||||
- https://toes.local web UI for managing your projects.
|
||||
- `toes` CLI for managing your projects.
|
||||
## SSH CLI
|
||||
|
||||
## ssh cli
|
||||
|
||||
You can manage your toes server from any machine on your network over SSH — no install required.
|
||||
Manage your server from any machine on the network — no install required.
|
||||
|
||||
```bash
|
||||
ssh cli@toes.local # interactive shell with tab completion
|
||||
|
|
@ -44,24 +71,12 @@ ssh cli@toes.local list # run a single command
|
|||
ssh cli@toes.local logs fog # stream logs for an app
|
||||
```
|
||||
|
||||
The `cli` user's login shell is the `toes` binary itself. No password is needed. With no arguments, you get an interactive REPL. With arguments, it runs the command and exits.
|
||||
## CLI Configuration
|
||||
|
||||
## cli configuration
|
||||
|
||||
by default, the CLI connects to `localhost:3000` in dev and `toes.local:80` in production.
|
||||
By default, the CLI connects to `localhost:3000` in dev and `toes.local:80` in production.
|
||||
|
||||
```bash
|
||||
toes config # show current host
|
||||
TOES_URL=http://192.168.1.50:3000 toes list # connect to IP
|
||||
TOES_URL=http://mypi.local toes list # connect to hostname
|
||||
```
|
||||
|
||||
## fun stuff
|
||||
|
||||
- textOS (TODO, more?)
|
||||
- Claude that knows about all your toes APIS and your projects.
|
||||
- non-webapps
|
||||
|
||||
## february goal
|
||||
|
||||
- [ ] Corey and Chris are running Toes servers on their home networks, hosting personal projects and games.
|
||||
|
|
|
|||
81
docs/WEBHOOKS.md
Normal file
81
docs/WEBHOOKS.md
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
# Rev Webhooks for Toes
|
||||
|
||||
Deploy Toes apps by saving to rev.host — no manual deploy step, no rsync scripts.
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
rev save "fix combat" → rev.host → relay (sneaker.toes.space) → toes.local pulls + deploys
|
||||
```
|
||||
|
||||
toes.local can't receive inbound connections (home NAT), so it maintains an outbound connection to a relay — same tunnel infrastructure used by `toes share`.
|
||||
|
||||
## Setup Flow
|
||||
|
||||
1. In the Toes dashboard (or `toes` CLI via SSH), enable rev webhooks for an app
|
||||
2. Toes connects to the relay and gets a stable webhook URL
|
||||
3. Add that URL in rev.host project settings as a webhook endpoint
|
||||
4. Done — `rev save` and `rev merge` now trigger deploys
|
||||
|
||||
## What Toes Does on Webhook
|
||||
|
||||
1. Receives event from relay (repo, ref, timestamp)
|
||||
2. Pulls latest from rev.host (needs a rev auth token stored in Toes env)
|
||||
3. Runs `scripts.predeploy` if defined in package.json (type-check, build, etc.)
|
||||
4. Runs `bun install`
|
||||
5. Restarts the app
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
# Enable/disable rev webhooks
|
||||
toes webhook enable [name] # Shows the relay URL to paste into rev.host
|
||||
toes webhook disable [name]
|
||||
|
||||
# Manual trigger (pull latest and deploy now)
|
||||
toes deploy [name]
|
||||
|
||||
# Check webhook status
|
||||
toes webhook status [name]
|
||||
```
|
||||
|
||||
## Settings UI
|
||||
|
||||
App settings page gets a "Rev Webhooks" section:
|
||||
- Toggle to enable/disable
|
||||
- Displays the relay URL (copy button)
|
||||
- Field for rev.host auth token
|
||||
- Last deploy timestamp + status
|
||||
- "Deploy Now" button (manual trigger)
|
||||
|
||||
## Auth
|
||||
|
||||
Toes needs read access to pull from rev.host. Store a rev API token per-app (or globally):
|
||||
|
||||
```bash
|
||||
toes env set -g REV_TOKEN rt_abc123
|
||||
# or per-app
|
||||
toes env set my-app REV_TOKEN rt_abc123
|
||||
```
|
||||
|
||||
## Predeploy Scripts
|
||||
|
||||
Project-specific build steps go in package.json:
|
||||
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"toes": "bun run --watch index.tsx",
|
||||
"predeploy": "bunx tsc --noEmit && bun build client/main.tsx --outdir dist --minify"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Toes runs `predeploy` after pulling but before restarting. If it exits non-zero, the deploy is aborted and the previous version stays running.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should the relay URL be per-app or per-Toes-instance? (Per-instance with app routing via path seems simpler: `https://sneaker.toes.space/hooks/<instance-id>/<app-name>`)
|
||||
- Webhook secret/signature verification — rev.host should sign payloads so the relay can't be spoofed
|
||||
- Should `toes deploy` work without webhooks enabled? (Just pull from rev.host on demand — useful as a migration path from deploy.sh)
|
||||
- Rollback: `toes rollback [name]` to revert to previous rev version?
|
||||
|
|
@ -8,7 +8,7 @@ set -euo pipefail
|
|||
# Installs or updates toes on a Raspberry Pi.
|
||||
# Must be run as the 'toes' user with passwordless sudo.
|
||||
|
||||
ARCHIVE="https://git.nose.space/defunkt/toes/archive/main.tar.gz"
|
||||
RELEASE_URL="https://toes.dev/release/latest.tar.gz"
|
||||
DEST=~/toes
|
||||
APPS_DIR=~/apps
|
||||
DATA_DIR=~/data
|
||||
|
|
@ -63,23 +63,19 @@ sudo setcap 'cap_net_bind_service=+ep' "$BUN"
|
|||
|
||||
info "Downloading toes"
|
||||
mkdir -p "$DEST"
|
||||
curl -fsSL "$ARCHIVE" | tar xz --strip-components=1 -C "$DEST"
|
||||
curl -fsSL "$RELEASE_URL" | tar xz --strip-components=1 -C "$DEST"
|
||||
|
||||
# ── Directories ──────────────────────────────────────────
|
||||
|
||||
mkdir -p "$APPS_DIR" "$DATA_DIR" "$DATA_DIR/toes"
|
||||
|
||||
# ── Dependencies & build ─────────────────────────────────
|
||||
# ── Dependencies ─────────────────────────────────────────
|
||||
|
||||
cd "$DEST"
|
||||
|
||||
info "Installing dependencies"
|
||||
quiet bun install
|
||||
|
||||
info "Building"
|
||||
rm -rf "$DEST/dist"
|
||||
quiet bun run build
|
||||
|
||||
# ── Bundled apps ─────────────────────────────────────────
|
||||
|
||||
REPOS_DIR="$DATA_DIR/repos"
|
||||
|
|
@ -94,19 +90,6 @@ for app_dir in "$DEST"/apps/*/; do
|
|||
(
|
||||
cp -a "$app_dir" "$APPS_DIR/$app"
|
||||
quiet bun install --frozen-lockfile --cwd "$APPS_DIR/$app" || quiet bun install --cwd "$APPS_DIR/$app"
|
||||
|
||||
# Seed bare repo for git-based versioning
|
||||
bare="$REPOS_DIR/$app.git"
|
||||
quiet git -C "$APPS_DIR/$app" init -b main
|
||||
quiet git -C "$APPS_DIR/$app" add -A
|
||||
quiet git -C "$APPS_DIR/$app" -c user.name=toes -c user.email=toes@localhost commit -m "install"
|
||||
if [ -d "$bare" ]; then
|
||||
quiet git -C "$APPS_DIR/$app" push --force "$bare" main
|
||||
else
|
||||
quiet git clone --bare "$APPS_DIR/$app" "$bare"
|
||||
quiet git -C "$bare" config http.receivepack true
|
||||
fi
|
||||
rm -rf "$APPS_DIR/$app/.git"
|
||||
) &
|
||||
pids+=("$!")
|
||||
done
|
||||
|
|
@ -115,15 +98,16 @@ for pid in "${pids[@]}"; do
|
|||
wait "$pid" || fail "A bundled app failed to install."
|
||||
done
|
||||
|
||||
# Copy pre-built bare repos for git-based versioning
|
||||
cp -a "$DEST"/dist/repos/*.git "$REPOS_DIR/"
|
||||
|
||||
# ── CLI + SSH ────────────────────────────────────────────
|
||||
|
||||
info "Setting up SSH access"
|
||||
sudo bash "$DEST/scripts/setup-ssh.sh"
|
||||
|
||||
info "Installing CLI"
|
||||
sudo rm -f /usr/local/bin/toes
|
||||
bun run cli:build
|
||||
sudo cp dist/toes /usr/local/bin/toes
|
||||
sudo install -m 755 "$DEST/dist/toes" /usr/local/bin/toes
|
||||
|
||||
# ── Systemd ──────────────────────────────────────────────
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@
|
|||
"scripts": {
|
||||
"check": "bun run templates && bunx tsc --noEmit",
|
||||
"build": "./scripts/build.sh",
|
||||
"release": "./scripts/release.sh",
|
||||
"cli:build": "bun run scripts/build.ts",
|
||||
"cli:build:all": "bun run scripts/build.ts --all",
|
||||
"cli:install": "bun cli:build && sudo cp dist/toes /usr/local/bin",
|
||||
|
|
|
|||
36
scripts/build-repos.sh
Executable file
36
scripts/build-repos.sh
Executable file
|
|
@ -0,0 +1,36 @@
|
|||
#!/usr/bin/env bash
|
||||
# Pre-builds bare git repos for bundled apps so the install script
|
||||
# doesn't need to run any git commands.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT="$SCRIPT_DIR/.."
|
||||
APPS_DIR="$ROOT/apps"
|
||||
OUT_DIR="$ROOT/dist/repos"
|
||||
|
||||
rm -rf "$OUT_DIR"
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
for app_dir in "$APPS_DIR"/*/; do
|
||||
app=$(basename "$app_dir")
|
||||
[ -f "$app_dir/package.json" ] || continue
|
||||
|
||||
tmp=$(mktemp -d)
|
||||
tar -C "$app_dir" \
|
||||
--exclude='node_modules' \
|
||||
--exclude='logs' \
|
||||
--exclude='current' \
|
||||
--exclude='[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9]' \
|
||||
-cf - . | tar -C "$tmp" -xf -
|
||||
|
||||
git -C "$tmp" init -b main -q
|
||||
git -C "$tmp" add -A
|
||||
git -C "$tmp" -c user.name=toes -c user.email=toes@localhost commit -q -m "install"
|
||||
git clone --bare -q "$tmp" "$OUT_DIR/$app.git"
|
||||
git -C "$OUT_DIR/$app.git" config http.receivepack true
|
||||
|
||||
rm -rf "$tmp"
|
||||
echo " $app"
|
||||
done
|
||||
|
||||
echo ">> Bare repos built in dist/repos/"
|
||||
|
|
@ -16,3 +16,4 @@ bun build src/client/index.tsx \
|
|||
|
||||
echo ">> Client bundle created at pub/client/index.js"
|
||||
ls -lh pub/client/index.js
|
||||
|
||||
|
|
|
|||
85
scripts/release.sh
Executable file
85
scripts/release.sh
Executable file
|
|
@ -0,0 +1,85 @@
|
|||
#!/usr/bin/env bash
|
||||
# Builds a release tarball with all artifacts pre-built.
|
||||
# The Pi just untars, runs bun install, and starts.
|
||||
#
|
||||
# Usage: bun run release
|
||||
# Output: dist/toes-<version>.tar.gz
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
VERSION=$(grep '"version"' package.json | head -1 | sed 's/.*"version": *"\(.*\)".*/\1/')
|
||||
STAGING="$ROOT/dist/toes-$VERSION"
|
||||
|
||||
echo ">> Building toes $VERSION release"
|
||||
|
||||
# ── Clean ────────────────────────────────────────────────
|
||||
|
||||
rm -rf dist
|
||||
mkdir -p dist
|
||||
|
||||
# ── Client bundle ────────────────────────────────────────
|
||||
|
||||
echo ">> Building client bundle"
|
||||
rm -rf pub/client
|
||||
mkdir -p pub/client
|
||||
bun build src/client/index.tsx \
|
||||
--outfile pub/client/index.js \
|
||||
--target browser \
|
||||
--minify
|
||||
|
||||
# ── Embedded templates ───────────────────────────────────
|
||||
|
||||
echo ">> Embedding templates"
|
||||
bun run scripts/embed-templates.ts
|
||||
|
||||
# ── Bare repos ───────────────────────────────────────────
|
||||
|
||||
echo ">> Building bundled app repos"
|
||||
bash scripts/build-repos.sh
|
||||
|
||||
# ── CLI binary (linux-arm64 for Pi) ──────────────────────
|
||||
|
||||
echo ">> Building CLI for linux-arm64"
|
||||
bun build src/cli/index.ts \
|
||||
--compile \
|
||||
--target bun-linux-arm64 \
|
||||
--minify \
|
||||
--sourcemap=external \
|
||||
--define="__GIT_SHA__=\"$VERSION\"" \
|
||||
--outfile dist/toes-linux-arm64
|
||||
|
||||
# ── Stage release ────────────────────────────────────────
|
||||
|
||||
echo ">> Staging release"
|
||||
mkdir -p "$STAGING"
|
||||
|
||||
# Source (excluding dev artifacts)
|
||||
tar -C "$ROOT" \
|
||||
--exclude='node_modules' \
|
||||
--exclude='.git' \
|
||||
--exclude='dist' \
|
||||
--exclude='apps/*/node_modules' \
|
||||
--exclude='apps/*/logs' \
|
||||
--exclude='apps/*/current' \
|
||||
--exclude='apps/*/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9]' \
|
||||
-cf - . | tar -C "$STAGING" -xf -
|
||||
|
||||
# Pre-built artifacts
|
||||
cp -a pub/client "$STAGING/pub/client"
|
||||
cp -a dist/repos "$STAGING/dist/repos"
|
||||
mkdir -p "$STAGING/dist"
|
||||
cp dist/toes-linux-arm64 "$STAGING/dist/toes"
|
||||
|
||||
# Generated templates (so the server doesn't need to generate them)
|
||||
cp src/lib/templates.data.ts "$STAGING/src/lib/templates.data.ts"
|
||||
|
||||
# ── Tarball ──────────────────────────────────────────────
|
||||
|
||||
echo ">> Creating tarball"
|
||||
tar -C dist -czf "dist/toes-$VERSION.tar.gz" "toes-$VERSION"
|
||||
rm -rf "$STAGING"
|
||||
|
||||
SIZE=$(du -h "dist/toes-$VERSION.tar.gz" | cut -f1)
|
||||
echo ">> dist/toes-$VERSION.tar.gz ($SIZE)"
|
||||
Loading…
Reference in New Issue
Block a user