diff --git a/CLAUDE.md b/CLAUDE.md index b826cf2..da25f89 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,12 +23,36 @@ Personal web appliance that auto-discovers and runs multiple web apps on your ho ## Running ```bash -bun run dev # Hot reload (deletes pub/client/index.js first) -bun run start # Production -bun run check # Type check -bun run test # Tests +bun run dev # Hot reload (rebuilds client bundle on change) +bun run start # Production (generates templates, then runs server) +bun run check # Type check +bun run test # Tests +bun run build # Build client JS bundle (pub/client/index.js) +bun run release # Build release tarball for the Pi ``` +## Building & Releasing + +`bun run release` (runs `scripts/release.sh`) produces a self-contained tarball the Pi can install without building anything: + +1. Client JS bundle → `pub/client/index.js` +2. Embedded templates → `src/lib/templates.data.ts` (generated from `templates/` by `scripts/embed-templates.ts`) +3. Pre-built bare git repos for bundled apps → `dist/repos/*.git` (built by `scripts/build-repos.sh`) +4. Cross-compiled CLI binary → `dist/toes` (linux-arm64, built by `scripts/build.ts`) +5. Everything staged and packed into `dist/toes-.tar.gz` + +The Pi installer (`install/install.sh`) downloads this tarball, runs `bun install` for the server and all bundled apps (in parallel), copies pre-built repos and CLI binary into place, and starts the systemd service. No git commands, no compilation on the Pi. + +### Key scripts + +- `scripts/build.sh` -- Builds client JS bundle only (used during dev) +- `scripts/build-repos.sh` -- Pre-builds bare git repos for bundled apps (excludes node_modules, snapshots, logs) +- `scripts/release.sh` -- Full release pipeline: client + templates + repos + CLI → tarball +- `scripts/build.ts` -- CLI binary compiler (current platform or `--all` for cross-compile) +- `scripts/embed-templates.ts` -- Generates `src/lib/templates.data.ts` from `templates/` +- `install/install.sh` -- Pi installer, downloads release tarball and sets everything up +- `scripts/remote-install.sh` -- Runs the installer on a remote Pi over SSH + ## Project Structure ``` @@ -235,6 +259,21 @@ function start(app: App): void { } ``` +## Install & Deployment + +The install script (`install/install.sh`) is designed to run on a fresh Pi or as an updater: + +1. Installs system packages (git, fish, avahi-utils, etc.) via apt +2. Installs Bun and grants `cap_net_bind_service` +3. Downloads and extracts the release tarball into `~/toes` +4. Runs `bun install` for the server +5. Copies bundled apps to `~/apps/` and runs `bun install` for each (in parallel) +6. Copies pre-built bare repos to `~/data/repos/` (for git-based versioning) +7. Installs the pre-built CLI binary to `/usr/local/bin/toes` +8. Sets up SSH access and the systemd service + +The release tarball URL is configured as `RELEASE_URL` at the top of `install/install.sh`. + ## Writing Apps and Tools See `docs/GUIDE.md` for the guide to writing toes apps and tools. diff --git a/README.md b/README.md index 0903d27..31f4f03 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,69 @@ # 🐾 Toes -Toes is a personal web appliance you run on your home network. +Personal web appliance you run on your home network. Plug it in, turn it on, and forget about the cloud. -## setup +## Development -Toes runs on a Raspberry Pi. You'll need: +```bash +bun run dev # Hot reload (rebuilds client bundle on change) +bun run start # Production mode +bun run check # Type check +bun run test # Tests +bun run build # Build client JS bundle +bun run release # Build a release tarball for the Pi +``` -- A Raspberry Pi 5 running the latest Raspberry Pi OS -- A `toes` user with passwordless sudo +### Releasing -SSH into your Pi as the `toes` user and run: +`bun run release` builds everything the Pi needs into a single tarball: + +1. Client JS bundle (`pub/client/index.js`) +2. Embedded templates (`src/lib/templates.data.ts`) +3. Pre-built bare git repos for bundled apps (`dist/repos/`) +4. Cross-compiled CLI binary for linux-arm64 (`dist/toes`) + +Output: `dist/toes-.tar.gz` + +The Pi does zero building — it untars, runs `bun install`, and starts. Upload the tarball to wherever `RELEASE_URL` in `install/install.sh` points (currently `https://toes.dev/release/latest.tar.gz`). + +### Scripts + +| Script | What it does | +|--------|-------------| +| `scripts/build.sh` | Builds the client JS bundle into `pub/client/index.js` | +| `scripts/build-repos.sh` | Pre-builds bare git repos for bundled apps in `dist/repos/` | +| `scripts/release.sh` | Full release: client + templates + repos + CLI → tarball | +| `scripts/build.ts` | Builds the CLI binary (current platform or cross-compile) | +| `scripts/embed-templates.ts` | Generates `src/lib/templates.data.ts` from `templates/` | +| `scripts/setup-ssh.sh` | Configures SSH access for the `cli` user on the Pi | +| `scripts/remote-install.sh` | Runs the installer on a remote Pi over SSH | + +## Setup + +Toes runs on a Raspberry Pi 5 with a `toes` user and passwordless sudo. ```bash curl -fsSL https://toes.dev/install | bash ``` -This will: +The installer downloads the release tarball, installs bun and system packages, runs `bun install` for the server and all bundled apps (in parallel), copies the pre-built CLI and git repos into place, and starts the systemd service. -1. Install system dependencies (git, fish shell, networking tools) -2. Install Bun and grant it network binding capabilities -3. Clone and build the toes server -4. Set up bundled apps and tools (clock, code, cron, env, stats) -5. Install and enable a systemd service for auto-start +Dashboard: `http://.local` -Once complete, visit `http://.local` on your local network. +## Features -## features +- Hosts Bun/Hype webapps (SSR and SPA) +- `git push` Heroku-style deploys +- Web dashboard with real-time status, logs, and tools +- `toes` CLI (local install or SSH) +- Per-app environment variables, cron jobs, metrics +- Public sharing via tunnels -- Effortlessly hosts bun/hype webapps - both SSR and SPA. -- `git push`, Heroku-style deploys -- https://toes.local web UI for managing your projects. -- `toes` CLI for managing your projects. +## SSH CLI -## ssh cli - -You can manage your toes server from any machine on your network over SSH — no install required. +Manage your server from any machine on the network — no install required. ```bash ssh cli@toes.local # interactive shell with tab completion @@ -44,24 +71,12 @@ ssh cli@toes.local list # run a single command ssh cli@toes.local logs fog # stream logs for an app ``` -The `cli` user's login shell is the `toes` binary itself. No password is needed. With no arguments, you get an interactive REPL. With arguments, it runs the command and exits. +## CLI Configuration -## cli configuration - -by default, the CLI connects to `localhost:3000` in dev and `toes.local:80` in production. +By default, the CLI connects to `localhost:3000` in dev and `toes.local:80` in production. ```bash toes config # show current host TOES_URL=http://192.168.1.50:3000 toes list # connect to IP TOES_URL=http://mypi.local toes list # connect to hostname ``` - -## fun stuff - -- textOS (TODO, more?) -- Claude that knows about all your toes APIS and your projects. -- non-webapps - -## february goal - -- [ ] Corey and Chris are running Toes servers on their home networks, hosting personal projects and games. diff --git a/docs/WEBHOOKS.md b/docs/WEBHOOKS.md new file mode 100644 index 0000000..a20531e --- /dev/null +++ b/docs/WEBHOOKS.md @@ -0,0 +1,81 @@ +# Rev Webhooks for Toes + +Deploy Toes apps by saving to rev.host — no manual deploy step, no rsync scripts. + +## How It Works + +``` +rev save "fix combat" → rev.host → relay (sneaker.toes.space) → toes.local pulls + deploys +``` + +toes.local can't receive inbound connections (home NAT), so it maintains an outbound connection to a relay — same tunnel infrastructure used by `toes share`. + +## Setup Flow + +1. In the Toes dashboard (or `toes` CLI via SSH), enable rev webhooks for an app +2. Toes connects to the relay and gets a stable webhook URL +3. Add that URL in rev.host project settings as a webhook endpoint +4. Done — `rev save` and `rev merge` now trigger deploys + +## What Toes Does on Webhook + +1. Receives event from relay (repo, ref, timestamp) +2. Pulls latest from rev.host (needs a rev auth token stored in Toes env) +3. Runs `scripts.predeploy` if defined in package.json (type-check, build, etc.) +4. Runs `bun install` +5. Restarts the app + +## CLI + +```bash +# Enable/disable rev webhooks +toes webhook enable [name] # Shows the relay URL to paste into rev.host +toes webhook disable [name] + +# Manual trigger (pull latest and deploy now) +toes deploy [name] + +# Check webhook status +toes webhook status [name] +``` + +## Settings UI + +App settings page gets a "Rev Webhooks" section: +- Toggle to enable/disable +- Displays the relay URL (copy button) +- Field for rev.host auth token +- Last deploy timestamp + status +- "Deploy Now" button (manual trigger) + +## Auth + +Toes needs read access to pull from rev.host. Store a rev API token per-app (or globally): + +```bash +toes env set -g REV_TOKEN rt_abc123 +# or per-app +toes env set my-app REV_TOKEN rt_abc123 +``` + +## Predeploy Scripts + +Project-specific build steps go in package.json: + +```json +{ + "scripts": { + "toes": "bun run --watch index.tsx", + "predeploy": "bunx tsc --noEmit && bun build client/main.tsx --outdir dist --minify" + } +} +``` + +Toes runs `predeploy` after pulling but before restarting. If it exits non-zero, the deploy is aborted and the previous version stays running. + +## Open Questions + +- Should the relay URL be per-app or per-Toes-instance? (Per-instance with app routing via path seems simpler: `https://sneaker.toes.space/hooks//`) +- Webhook secret/signature verification — rev.host should sign payloads so the relay can't be spoofed +- Should `toes deploy` work without webhooks enabled? (Just pull from rev.host on demand — useful as a migration path from deploy.sh) +- Rollback: `toes rollback [name]` to revert to previous rev version? diff --git a/install/install.sh b/install/install.sh index 11e72c1..dfd2410 100755 --- a/install/install.sh +++ b/install/install.sh @@ -8,7 +8,7 @@ set -euo pipefail # Installs or updates toes on a Raspberry Pi. # Must be run as the 'toes' user with passwordless sudo. -ARCHIVE="https://git.nose.space/defunkt/toes/archive/main.tar.gz" +RELEASE_URL="https://toes.dev/release/latest.tar.gz" DEST=~/toes APPS_DIR=~/apps DATA_DIR=~/data @@ -63,23 +63,19 @@ sudo setcap 'cap_net_bind_service=+ep' "$BUN" info "Downloading toes" mkdir -p "$DEST" -curl -fsSL "$ARCHIVE" | tar xz --strip-components=1 -C "$DEST" +curl -fsSL "$RELEASE_URL" | tar xz --strip-components=1 -C "$DEST" # ── Directories ────────────────────────────────────────── mkdir -p "$APPS_DIR" "$DATA_DIR" "$DATA_DIR/toes" -# ── Dependencies & build ───────────────────────────────── +# ── Dependencies ───────────────────────────────────────── cd "$DEST" info "Installing dependencies" quiet bun install -info "Building" -rm -rf "$DEST/dist" -quiet bun run build - # ── Bundled apps ───────────────────────────────────────── REPOS_DIR="$DATA_DIR/repos" @@ -94,19 +90,6 @@ for app_dir in "$DEST"/apps/*/; do ( cp -a "$app_dir" "$APPS_DIR/$app" quiet bun install --frozen-lockfile --cwd "$APPS_DIR/$app" || quiet bun install --cwd "$APPS_DIR/$app" - - # Seed bare repo for git-based versioning - bare="$REPOS_DIR/$app.git" - quiet git -C "$APPS_DIR/$app" init -b main - quiet git -C "$APPS_DIR/$app" add -A - quiet git -C "$APPS_DIR/$app" -c user.name=toes -c user.email=toes@localhost commit -m "install" - if [ -d "$bare" ]; then - quiet git -C "$APPS_DIR/$app" push --force "$bare" main - else - quiet git clone --bare "$APPS_DIR/$app" "$bare" - quiet git -C "$bare" config http.receivepack true - fi - rm -rf "$APPS_DIR/$app/.git" ) & pids+=("$!") done @@ -115,15 +98,16 @@ for pid in "${pids[@]}"; do wait "$pid" || fail "A bundled app failed to install." done +# Copy pre-built bare repos for git-based versioning +cp -a "$DEST"/dist/repos/*.git "$REPOS_DIR/" + # ── CLI + SSH ──────────────────────────────────────────── info "Setting up SSH access" sudo bash "$DEST/scripts/setup-ssh.sh" info "Installing CLI" -sudo rm -f /usr/local/bin/toes -bun run cli:build -sudo cp dist/toes /usr/local/bin/toes +sudo install -m 755 "$DEST/dist/toes" /usr/local/bin/toes # ── Systemd ────────────────────────────────────────────── diff --git a/package.json b/package.json index 890c743..80d1542 100644 --- a/package.json +++ b/package.json @@ -17,6 +17,7 @@ "scripts": { "check": "bun run templates && bunx tsc --noEmit", "build": "./scripts/build.sh", + "release": "./scripts/release.sh", "cli:build": "bun run scripts/build.ts", "cli:build:all": "bun run scripts/build.ts --all", "cli:install": "bun cli:build && sudo cp dist/toes /usr/local/bin", diff --git a/scripts/build-repos.sh b/scripts/build-repos.sh new file mode 100755 index 0000000..6eb6b53 --- /dev/null +++ b/scripts/build-repos.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# Pre-builds bare git repos for bundled apps so the install script +# doesn't need to run any git commands. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +ROOT="$SCRIPT_DIR/.." +APPS_DIR="$ROOT/apps" +OUT_DIR="$ROOT/dist/repos" + +rm -rf "$OUT_DIR" +mkdir -p "$OUT_DIR" + +for app_dir in "$APPS_DIR"/*/; do + app=$(basename "$app_dir") + [ -f "$app_dir/package.json" ] || continue + + tmp=$(mktemp -d) + tar -C "$app_dir" \ + --exclude='node_modules' \ + --exclude='logs' \ + --exclude='current' \ + --exclude='[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9]' \ + -cf - . | tar -C "$tmp" -xf - + + git -C "$tmp" init -b main -q + git -C "$tmp" add -A + git -C "$tmp" -c user.name=toes -c user.email=toes@localhost commit -q -m "install" + git clone --bare -q "$tmp" "$OUT_DIR/$app.git" + git -C "$OUT_DIR/$app.git" config http.receivepack true + + rm -rf "$tmp" + echo " $app" +done + +echo ">> Bare repos built in dist/repos/" diff --git a/scripts/build.sh b/scripts/build.sh index e06df82..364fc01 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -16,3 +16,4 @@ bun build src/client/index.tsx \ echo ">> Client bundle created at pub/client/index.js" ls -lh pub/client/index.js + diff --git a/scripts/release.sh b/scripts/release.sh new file mode 100755 index 0000000..ab7c532 --- /dev/null +++ b/scripts/release.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# Builds a release tarball with all artifacts pre-built. +# The Pi just untars, runs bun install, and starts. +# +# Usage: bun run release +# Output: dist/toes-.tar.gz +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$ROOT" + +VERSION=$(grep '"version"' package.json | head -1 | sed 's/.*"version": *"\(.*\)".*/\1/') +STAGING="$ROOT/dist/toes-$VERSION" + +echo ">> Building toes $VERSION release" + +# ── Clean ──────────────────────────────────────────────── + +rm -rf dist +mkdir -p dist + +# ── Client bundle ──────────────────────────────────────── + +echo ">> Building client bundle" +rm -rf pub/client +mkdir -p pub/client +bun build src/client/index.tsx \ + --outfile pub/client/index.js \ + --target browser \ + --minify + +# ── Embedded templates ─────────────────────────────────── + +echo ">> Embedding templates" +bun run scripts/embed-templates.ts + +# ── Bare repos ─────────────────────────────────────────── + +echo ">> Building bundled app repos" +bash scripts/build-repos.sh + +# ── CLI binary (linux-arm64 for Pi) ────────────────────── + +echo ">> Building CLI for linux-arm64" +bun build src/cli/index.ts \ + --compile \ + --target bun-linux-arm64 \ + --minify \ + --sourcemap=external \ + --define="__GIT_SHA__=\"$VERSION\"" \ + --outfile dist/toes-linux-arm64 + +# ── Stage release ──────────────────────────────────────── + +echo ">> Staging release" +mkdir -p "$STAGING" + +# Source (excluding dev artifacts) +tar -C "$ROOT" \ + --exclude='node_modules' \ + --exclude='.git' \ + --exclude='dist' \ + --exclude='apps/*/node_modules' \ + --exclude='apps/*/logs' \ + --exclude='apps/*/current' \ + --exclude='apps/*/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9]' \ + -cf - . | tar -C "$STAGING" -xf - + +# Pre-built artifacts +cp -a pub/client "$STAGING/pub/client" +cp -a dist/repos "$STAGING/dist/repos" +mkdir -p "$STAGING/dist" +cp dist/toes-linux-arm64 "$STAGING/dist/toes" + +# Generated templates (so the server doesn't need to generate them) +cp src/lib/templates.data.ts "$STAGING/src/lib/templates.data.ts" + +# ── Tarball ────────────────────────────────────────────── + +echo ">> Creating tarball" +tar -C dist -czf "dist/toes-$VERSION.tar.gz" "toes-$VERSION" +rm -rf "$STAGING" + +SIZE=$(du -h "dist/toes-$VERSION.tar.gz" | cut -f1) +echo ">> dist/toes-$VERSION.tar.gz ($SIZE)"