241 lines
11 KiB
Markdown
241 lines
11 KiB
Markdown
# Toes
|
|
|
|
Personal web appliance that auto-discovers and runs multiple web apps on your home network.
|
|
|
|
"Plug it in, turn it on, and forget about the cloud."
|
|
|
|
## How It Works
|
|
|
|
1. Server scans `APPS_DIR` for directories with a `package.json` containing a `scripts.toes` entry
|
|
2. Each app is spawned as a child process with a unique port (3001-3100)
|
|
3. Dashboard UI shows all apps with status, logs, and links via SSE
|
|
4. CLI communicates with the server over HTTP
|
|
|
|
## Tech Stack
|
|
|
|
- **Bun** runtime (not Node)
|
|
- **Hype** (custom HTTP framework wrapping Hono) from `@because/hype`
|
|
- **Forge** (typed CSS-in-JS) from `@because/forge`
|
|
- **Commander** + **ansis** for CLI
|
|
- TypeScript + Hono JSX
|
|
- Client renders with `hono/jsx/dom` (no build step, served directly)
|
|
|
|
## Running
|
|
|
|
```bash
|
|
bun run dev # Hot reload (deletes pub/client/index.js first)
|
|
bun run start # Production
|
|
bun run check # Type check
|
|
bun run test # Tests
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
src/
|
|
server/ # HTTP server and process management ($)
|
|
client/ # Browser-side dashboard
|
|
shared/ # Types shared between server and client (@)
|
|
lib/ # Code shared between CLI and server (%)
|
|
cli/ # CLI tool
|
|
tools/ # @because/toes package exports
|
|
pages/ # Hype page routes
|
|
```
|
|
|
|
Path aliases: `$` = server, `@` = shared, `%` = lib (defined in tsconfig.json).
|
|
|
|
### Server (`src/server/`)
|
|
|
|
- `apps.ts` -- **The heart**: app discovery, process spawning, health checks, auto-restart, port allocation, log management, graceful shutdown. Exports `APPS_DIR`, `TOES_DIR`, `TOES_URL`, and the `App` type (extends shared `App` with process/timer fields).
|
|
- `api/apps.ts` -- REST API + SSE stream. Routes: `GET /` (list), `GET /stream` (SSE), `POST /:name/start|stop|restart`, `GET /:name/logs`, `POST /` (create via git), `POST /:name/rename`, `POST /:name/icon`, env var CRUD, tunnel management.
|
|
- `api/events.ts` -- SSE stream for discrete lifecycle events (`/stream`). Used by app processes (not the dashboard). Includes a 60s heartbeat ping.
|
|
- `api/sync.ts` -- File sync API: manifest endpoint, file read/write, app delete, app reload (triggered by git tool after deploy), file watch SSE.
|
|
- `api/system.ts` -- System info, metrics (CPU/RAM/disk per-app via `ps`/`du`), unified log aggregation, perf timing toggle, update check/apply, server restart.
|
|
- `index.tsx` -- Entry point. Mounts API routers, tool URL redirects (`/tool/:tool`), tool API proxy (`/api/tools/:tool/*`), on-demand CLI binary builds (`/dist/:file`), install script endpoint (`/install`), SPA catch-all routes, subdomain proxy (including WebSocket). Initializes apps.
|
|
- `mdns.ts` -- mDNS publishing via `avahi-publish` on Linux production. Publishes `<app>.hostname.local` A records pointing to local IP. Auto-republishes on unexpected exit with exponential backoff.
|
|
- `proxy.ts` -- Subdomain routing: extracts subdomain from `*.localhost` or `*.X.local`, proxies HTTP requests and WebSocket connections to the app's port. Sets `x-app-url` header. Optional perf timing.
|
|
- `shell.tsx` -- Minimal HTML shell for the SPA.
|
|
- `sync.ts` -- Re-exports `computeHash` and `generateManifest` from `%sync` (lib).
|
|
- `tui.ts` -- Terminal UI for the server process (renders app status table when TTY, plain logs otherwise). Debounced rendering at 50ms.
|
|
- `tunnels.ts` -- Public tunnel management via `@because/sneaker`. Persists tunnel config in `TOES_DIR/tunnels.json`. Auto-reconnects on drop with exponential backoff (max 10 attempts). Manages share/unshare lifecycle.
|
|
|
|
### Client (`src/client/`)
|
|
|
|
Client-side SPA rendered with `hono/jsx/dom`. No build step -- Bun serves `.tsx` files directly.
|
|
|
|
- `index.tsx` -- Entry point. Initializes rendering, SSE connection, theme, tool iframes.
|
|
- `state.ts` -- Mutable module-level state (`apps`, `selectedApp`, `sidebarCollapsed`, etc.) with localStorage persistence. Components import state directly.
|
|
- `api.ts` -- Fetch wrappers for server API calls.
|
|
- `tool-iframes.ts` -- Manages tool iframe lifecycle (caching, visibility, height communication).
|
|
- `update.tsx` -- SSE connection to `/api/apps/stream` for real-time state updates.
|
|
- `router.ts` -- Client-side router. Intercepts link clicks, handles popstate, maps URL paths to state (`/app/:name/:tab`, `/settings`, dashboard tabs).
|
|
- `ansi.ts` -- ANSI escape code handling for log rendering.
|
|
- `components/` -- Dashboard, Sidebar, AppDetail, Nav, AppSelector, LogsSection, DashboardLanding, SettingsPage, Vitals, UnifiedLogs, Urls, emoji-picker, modal.
|
|
- `modals/` -- NewApp, RenameApp, DeleteApp dialogs.
|
|
- `styles/` -- Forge CSS-in-JS (themes, buttons, forms, layout, logs, misc).
|
|
- `themes/` -- Light/dark theme token definitions.
|
|
|
|
### CLI (`src/cli/`)
|
|
|
|
- `index.ts` -- Entry point (`#!/usr/bin/env bun`).
|
|
- `setup.ts` -- Commander program definition with all commands.
|
|
- `commands/` -- Command implementations.
|
|
- `http.ts` -- HTTP client for talking to the toes server.
|
|
- `name.ts` -- App name resolution (argument or current directory).
|
|
- `prompts.ts` -- Interactive prompts.
|
|
- `pager.ts` -- Pipe output through system pager.
|
|
|
|
CLI commands:
|
|
- **Apps**: `status` (list or info), `new`, `get`, `open`, `rename`, `rm`
|
|
- **Lifecycle**: `start`, `stop`, `restart`, `share`, `unshare`, `logs`, `metrics`, `cron` (list/log/status/run)
|
|
- **Config**: `env` (list/set/rm, per-app or `--global`), `perf` (toggle request timing)
|
|
- **Hidden**: `list`, `info`, `log`, `version`, `shell`
|
|
|
|
Some commands (`shell`, `get`, `open`) are disabled when running over SSH (`USER=cli`).
|
|
|
|
### Shared (`src/shared/`)
|
|
|
|
Types shared between browser and server. **Cannot use Node/filesystem APIs** (runs in browser).
|
|
|
|
- `types.ts` -- `App`, `AppState`, `LogLine`, `Manifest`, `FileInfo`, `DEFAULT_EMOJI`, `VALID_NAME`
|
|
- `events.ts` -- `ToesEvent`, `ToesEventType`, `ToesEventInput` type definitions
|
|
- `urls.ts` -- `toSubdomain()` and `buildAppUrl()` for subdomain URL construction
|
|
- `gitignore.ts` -- `.gitignore` pattern matching (used by sync API and file watchers)
|
|
|
|
### Lib (`src/lib/`)
|
|
|
|
Server-side code shared between CLI and server. Can use Node APIs.
|
|
|
|
- `config.ts` -- `HOSTNAME` and `LOCAL_HOST` (`hostname.local`) constants
|
|
- `templates.ts` -- Template generation for `toes new` (bare, ssr, spa), reads embedded templates from `templates.data.ts`
|
|
- `templates.data.ts` -- Generated file containing embedded template contents (built by `scripts/embed-templates.ts`)
|
|
- `sync.ts` -- Manifest generation, hash computation (used by sync API for file diffing in tools)
|
|
|
|
### Tools Package (`src/tools/`)
|
|
|
|
The `@because/toes` package that apps/tools import. Published exports:
|
|
|
|
- `@because/toes` -- re-exports `computeHash`, `generateManifest`, `FileInfo`, `Manifest`, `VALID_NAME` (`src/index.ts` -> `src/tools/index.ts`)
|
|
- `@because/toes/tools` -- `baseStyles`, `ToolScript`, `theme`, `loadAppEnv`, `on` (event subscription), `appUrl`, `VALID_NAME`
|
|
|
|
### Pages (`src/pages/`)
|
|
|
|
Hype page routes. `index.tsx` renders the Shell.
|
|
|
|
## Key Concepts
|
|
|
|
### App Lifecycle
|
|
|
|
States: `invalid` | `error` | `stopped` <-> `starting` -> `running` -> `stopping` -> `stopped`
|
|
|
|
- Discovery: scan `APPS_DIR`, read each `package.json` for `scripts.toes`
|
|
- Spawn: `Bun.spawn(['bun', 'run', 'toes'])` with `PORT`, `APP_URL`, `APPS_DIR`, `DATA_DIR`, `DATA_ROOT`, `TOES_URL`, `TOES_DIR`, `NO_AUTOPORT`, plus per-app env vars from `loadAppEnv()`
|
|
- Startup: runs `bun install` first if `node_modules/` missing, then polls `/ok` every 500ms (30s timeout)
|
|
- Health checks: every 30s to `/ok` (5s timeout), 3 consecutive failures trigger restart
|
|
- Auto-restart: exponential backoff (1s, 2s, 4s, 8s, 16s, 32s), max 5 attempts, resets after 60s stable run. State becomes `error` after max attempts.
|
|
- Graceful shutdown: SIGTERM with 10s timeout before SIGKILL
|
|
- On startup, kills stale processes on ports 3001-3100 and orphaned `bun run toes` processes
|
|
|
|
### Subdomain Proxy
|
|
|
|
Every app gets a subdomain: `<app>.localhost` (dev) or `<app>.hostname.local` (prod). The server's fetch handler (`index.tsx`) checks for subdomains first and proxies to the app's port. WebSocket connections are also proxied via Bun's `server.upgrade()` with upstream bridging. The `x-app-url` header is set so apps know their public URL.
|
|
|
|
### Tools vs Apps
|
|
|
|
Tools are apps with `"toes": { "tool": true }` in package.json. From the server's perspective they're identical processes. The dashboard renders tools as iframe tabs instead of sidebar entries. Tool URLs redirect through the server via subdomain: `/tool/:tool?app=foo` -> `http://<tool>.host/?app=foo`. Tool API calls can also be proxied: `/api/tools/:tool/*` -> `http://localhost:<toolPort>/*`.
|
|
|
|
The `apps` field in package.json controls whether a tool shows on app detail pages (`false` to hide). The `dashboard` field controls whether a tool shows on the dashboard landing page.
|
|
|
|
### Versioning
|
|
|
|
Apps use git for version control. Each app has a bare git repo at `DATA_DIR/repos/<name>.git`. Deploying is a `git push` to the server's git tool, which extracts HEAD into `APPS_DIR/<name>/` and reloads the app. History, diffing, and rollback use standard git commands.
|
|
|
|
### Environment Variables
|
|
|
|
Per-app env files in `TOES_DIR/env/`:
|
|
- `_global.env` -- shared by all apps
|
|
- `<appname>.env` -- per-app overrides
|
|
|
|
The server sets these on each app process: `PORT`, `APP_URL`, `APPS_DIR`, `DATA_DIR` (per-app at `TOES_DIR/<name>`), `DATA_ROOT`, `TOES_DIR`, `TOES_URL`, `NO_AUTOPORT`.
|
|
|
|
### SSE Streaming
|
|
|
|
Two SSE endpoints serve different consumers:
|
|
|
|
- `/api/apps/stream` -- Full app state snapshots on every change. Used by the dashboard UI. Driven by `onChange()` in `apps.ts`.
|
|
- `/api/events/stream` -- Discrete lifecycle events (`app:start`, `app:stop`, `app:reload`, `app:create`, `app:delete`). Used by app processes to react to other apps' lifecycle changes. Driven by `emit()`/`onEvent()` in `apps.ts`. Apps subscribe via `on()` from `@because/toes/tools`.
|
|
|
|
## Coding Guidelines
|
|
|
|
TS files should be organized in the following way:
|
|
|
|
- imports
|
|
- re-exports
|
|
- const/lets
|
|
- enums
|
|
- interfaces
|
|
- types
|
|
- classes
|
|
- functions
|
|
- module init (top level function calls)
|
|
|
|
In each section, put the `export`s first, in alphabetical order.
|
|
|
|
Then, after the `export`s (if there were any), put everything else,
|
|
also in alphabetical order.
|
|
|
|
For single-line functions, use `const fn = () => {}` and put them in the
|
|
"functions" section of the file.
|
|
|
|
All other functions use the `function blah(){}` format.
|
|
|
|
Example:
|
|
|
|
```ts
|
|
import { code } from "coders"
|
|
import { something } from "somewhere"
|
|
|
|
export type { SomeType }
|
|
|
|
const RETRY_TIMES = 5
|
|
const WIDTH = 480
|
|
|
|
enum State {
|
|
Stopped,
|
|
Starting,
|
|
Running,
|
|
}
|
|
|
|
interface Config {
|
|
name: string
|
|
port: number
|
|
}
|
|
|
|
type Handler = (req: Request) => Response
|
|
|
|
class App {
|
|
config: Config
|
|
|
|
constructor(config: Config) {
|
|
this.config = config
|
|
}
|
|
}
|
|
|
|
const isApp = (name: string) =>
|
|
apps.has(name)
|
|
|
|
function createApp(name: string): App {
|
|
const app = new App({ name, port: 3000 })
|
|
apps.set(name, app)
|
|
return app
|
|
}
|
|
|
|
function start(app: App): void {
|
|
console.log(`Starting ${app.config.name}`)
|
|
}
|
|
```
|
|
|
|
## Writing Apps and Tools
|
|
|
|
See `docs/GUIDE.md` for the guide to writing toes apps and tools.
|