works better

This commit is contained in:
Corey Johnson 2025-10-08 17:30:30 -07:00
parent 7f52e5e7e3
commit 0a80f6d13d
26 changed files with 514 additions and 464 deletions

2
.gitattributes vendored
View File

@ -1,2 +0,0 @@
# Auto detect text files and perform LF normalization
* text=auto

2
.gitignore vendored
View File

@ -32,3 +32,5 @@ report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json
# Finder (MacOS) folder config
.DS_Store
/tmp

306
CLAUDE.md
View File

@ -1,107 +1,261 @@
I am using the lezer grammar [System Guide](https://lezer.codemirror.net/docs/guide/) [api](https://lezer.codemirror.net/docs/ref/).
# CLAUDE.md
Default to using Bun instead of Node.js.
This file provides guidance to Claude Code (claude.ai/code) when working with the Shrimp programming language.
- Use `bun <file>` instead of `node <file>` or `ts-node <file>`
- Use `bun test` instead of `jest` or `vitest`
- Use `bun build <file.html|file.ts|file.css>` instead of `webpack` or `esbuild`
- Use `bun install` instead of `npm install` or `yarn install` or `pnpm install`
- Use `bun run <script>` instead of `npm run <script>` or `yarn run <script>` or `pnpm run <script>`
- Bun automatically loads .env, so don't use dotenv.
## Pair Programming Approach
## APIs
Act as a pair programming partner and teacher, not an autonomous code writer:
- `Bun.serve()` supports WebSockets, HTTPS, and routes. Don't use `express`.
- `bun:sqlite` for SQLite. Don't use `better-sqlite3`.
- `Bun.redis` for Redis. Don't use `ioredis`.
- `Bun.sql` for Postgres. Don't use `pg` or `postgres.js`.
- `WebSocket` is built-in. Don't use `ws`.
- Prefer `Bun.file` over `node:fs`'s readFile/writeFile
- Bun.$`ls` instead of execa.
**Research and guide, don't implement**:
## Testing
- Focus on research, analysis, and finding solutions
- Explain concepts, trade-offs, and best practices
- Guide the human through changes rather than making them directly
- Help them learn the codebase deeply by maintaining ownership
Use `bun test` to run tests.
**Use tmp/ directory for experimentation**:
```ts#index.test.ts
import { test, expect } from "bun:test";
- Create temporary files in `tmp/` to test ideas out experiments you want to run.
- Example: `tmp/eof-test.grammar`, `tmp/pattern-experiments.ts`
- Clean up tmp files when done
- Show multiple approaches so the human can choose
test("hello world", () => {
expect(1).toBe(1);
});
**Teaching moments**:
- Explain the "why" behind solutions
- Point out potential pitfalls and edge cases
- Share relevant documentation and examples
- Help build understanding, not just solve problems
## Project Overview
Shrimp is a shell-like scripting language that combines command-line simplicity with functional programming. The architecture flows: Shrimp source → parser (CST) → compiler (bytecode) → ReefVM (execution).
**Essential reading**: Before making changes, read README.md to understand the language design philosophy and parser architecture.
Key references: [Lezer System Guide](https://lezer.codemirror.net/docs/guide/) | [Lezer API](https://lezer.codemirror.net/docs/ref/)
## Development Commands
### Running Files
```bash
bun <file> # Run TypeScript files directly
bun src/server/server.tsx # Start development server
bun dev # Start development server (alias)
```
## Frontend
### Testing
Use HTML imports with `Bun.serve()`. Don't use `vite`. HTML imports fully support React, CSS, Tailwind.
```bash
bun test # Run all tests
bun test src/parser/parser.test.ts # Run parser tests specifically
bun test --watch # Watch mode
```
Server:
### Parser Development
```ts#index.ts
import index from "./index.html"
```bash
bun generate-parser # Regenerate parser from grammar
bun test src/parser/parser.test.ts # Test grammar changes
```
Bun.serve({
routes: {
"/": index,
"/api/users/:id": {
GET: (req) => {
return new Response(JSON.stringify({ id: req.params.id }));
},
},
},
// optional websocket support
websocket: {
open: (ws) => {
ws.send("Hello, world!");
},
message: (ws, message) => {
ws.send(message);
},
close: (ws) => {
// handle close
### Server
```bash
bun dev # Start playground at http://localhost:3000
```
### Building
No build step required - Bun runs TypeScript directly. Parser auto-regenerates during tests.
## Code Style Preferences
**Early returns over deep nesting**:
```typescript
// ✅ Good
const processToken = (token: Token) => {
if (!token) return null
if (token.type !== 'identifier') return null
return processIdentifier(token)
}
},
development: {
hmr: true,
console: true,
// ❌ Avoid
const processToken = (token: Token) => {
if (token) {
if (token.type === 'identifier') {
return processIdentifier(token)
}
}
return null
}
```
**Arrow functions over function keyword**:
```typescript
// ✅ Good
const parseExpression = (input: string) => {
// implementation
}
// ❌ Avoid
function parseExpression(input: string) {
// implementation
}
```
**Code readability over cleverness**:
- Use descriptive variable names
- Write code that explains itself
- Prefer explicit over implicit
- Two simple functions beat one complex function
## Architecture
### Core Components
**parser/** (Lezer-based parsing):
- **shrimp.grammar**: Lezer grammar definition with tokens and rules
- **shrimp.ts**: Auto-generated parser (don't edit directly)
- **tokenizer.ts**: Custom tokenizer for identifier vs word distinction
- **parser.test.ts**: Comprehensive grammar tests using `toMatchTree`
**editor/** (CodeMirror integration):
- Syntax highlighting for Shrimp language
- Language support and autocomplete
- Integration with the parser for real-time feedback
**compiler/** (CST to bytecode):
- Transforms concrete syntax trees into ReefVM bytecode
- Handles function definitions, expressions, and control flow
### Critical Design Decisions
**Whitespace-sensitive parsing**: Spaces distinguish operators from identifiers (`x-1` vs `x - 1`). This enables natural shell-like syntax.
**Identifier vs Word tokenization**: Custom tokenizer determines if a token is an assignable identifier (lowercase/emoji start) or a non-assignable word (paths, URLs). This allows `./file.txt` without quotes.
**Ambiguous identifier resolution**: Bare identifiers like `myVar` could be function calls or variable references. The parser creates `FunctionCallOrIdentifier` nodes, resolved at runtime.
**Expression-oriented design**: Everything returns a value - commands, assignments, functions. This enables composition and functional patterns.
**EOF handling**: The grammar uses `(statement | newlineOrSemicolon)+ eof?` to handle empty lines and end-of-file without infinite loops.
## Grammar Development
### Grammar Structure
The grammar follows this hierarchy:
```
Program → statement*
statement → line newlineOrSemicolon | line eof
line → FunctionCall | FunctionCallOrIdentifier | FunctionDef | Assign | expression
```
Key tokens:
- `newlineOrSemicolon`: `"\n" | ";"`
- `eof`: `@eof`
- `Identifier`: Lowercase/emoji start, assignable variables
- `Word`: Everything else (paths, URLs, etc.)
### Adding Grammar Rules
When modifying the grammar:
1. **Update `src/parser/shrimp.grammar`** with your changes
2. **Run tests** - the parser auto-regenerates during test runs
3. **Add test cases** in `src/parser/parser.test.ts` using `toMatchTree`
4. **Test empty line handling** - ensure EOF works properly
### Test Format
Grammar tests use this pattern:
```typescript
test('function call with args', () => {
expect('echo hello world').toMatchTree(`
FunctionCall
Identifier echo
PositionalArg
Word hello
PositionalArg
Word world
`)
})
```
HTML files can import .tsx, .jsx or .js files directly and Bun's bundler will transpile & bundle automatically. `<link>` tags can point to stylesheets and Bun's CSS bundler will bundle.
The `toMatchTree` helper compares parser output with expected CST structure.
```html#index.html
<html>
<body>
<h1>Hello, world!</h1>
<script type="module" src="./frontend.tsx"></script>
</body>
</html>
### Common Grammar Gotchas
**EOF infinite loops**: Using `@eof` in repeating patterns can match EOF multiple times. Current approach uses explicit statement/newline alternatives.
**Token precedence**: Use `@precedence` to resolve conflicts between similar tokens.
**External tokenizers**: Custom logic in `tokenizers.ts` handles complex cases like identifier vs word distinction.
**Empty line parsing**: The grammar structure `(statement | newlineOrSemicolon)+ eof?` allows proper empty line and EOF handling.
## Testing Strategy
### Parser Tests (`src/parser/parser.test.ts`)
- **Token types**: Identifier vs Word distinction
- **Function calls**: With and without arguments
- **Expressions**: Binary operations, parentheses, precedence
- **Functions**: Single-line and multiline definitions
- **Whitespace**: Empty lines, mixed delimiters
- **Edge cases**: Ambiguous parsing, incomplete input
Test structure:
```typescript
describe('feature area', () => {
test('specific case', () => {
expect(input).toMatchTree(expectedCST)
})
})
```
With the following `frontend.tsx`:
When adding language features:
```tsx#frontend.tsx
import React from "react";
1. Write grammar tests first showing expected CST structure
2. Update grammar rules to make tests pass
3. Add integration tests showing real usage
4. Test edge cases and error conditions
// import .css files directly and it works
import './index.css';
## Bun Usage
import { createRoot } from "react-dom/client";
Default to Bun over Node.js/npm:
const root = createRoot(document.body);
- Use `bun <file>` instead of `node <file>` or `ts-node <file>`
- Use `bun test` instead of `jest` or `vitest`
- Use `bun install` instead of `npm install`
- Use `bun run <script>` instead of `npm run <script>`
- Bun automatically loads .env, so don't use dotenv
export default function Frontend() {
return <h1>Hello, world!</h1>;
}
### Bun APIs
root.render(<Frontend />);
```
- Prefer `Bun.file` over `node:fs`'s readFile/writeFile
- Use `Bun.$` for shell commands instead of execa
Then, run index.ts
## Common Patterns
```sh
bun --hot ./index.ts
```
### Grammar Debugging
For more information, read the Bun API docs in `node_modules/bun-types/docs/**.md`.
When grammar isn't parsing correctly:
1. **Check token precedence** - ensure tokens are recognized correctly
2. **Test simpler cases first** - build up from basic to complex
3. **Use `toMatchTree` output** - see what the parser actually produces
4. **Check external tokenizer** - identifier vs word logic in `tokenizers.ts`

View File

@ -4,28 +4,38 @@
Shrimp is a shell-like scripting language that combines the simplicity of command-line interfaces with functional programming concepts. Built using Lezer (CodeMirror's parser system) with TypeScript.
## Use it
Go to http://localhost:3000 to try out the playground.
echo "Hello, world!"
tail log.txt lines=50
name = "Shrimp"
greet = fn person: echo "Hello" person
result = tail log.txt lines=10
## Language Design Philosophy
- **Everything is an expression** - Commands, assignments, and functions all return values
- **Whitespace matters** - Spaces distinguish operators from identifiers (e.g., `x-1` is an identifier, `x - 1` is subtraction)
- **Shell-like command syntax** - `echo hello world` works naturally
- **Named arguments without quotes** - `tail file.txt lines=30`
- **Everything is an expression** - Commands, assignments, and functions all return values
- **Whitespace matters in binary operations** - Spaces distinguish operators from identifiers (e.g., `x-1` is an identifier, `x - 1` is subtraction)
- **Unbound symbols become strings** - `echo hello` treats `hello` as a string if not defined
- **Simplicity over cleverness** - Each feature should work one way, consistently. Two simple features that are easy to explain beat one complex feature that requires lots of explanation
### Parser Features
- ✅ Distinguishes between identifiers (assignable) and words e(non-assignable)
- ✅ Distinguishes identifiers from words to enable shell-like syntax - paths like `./file.txt` work without quotes
- ✅ Smart tokenization for named args (`lines=30` splits, but `./path=value` stays together)
- ✅ Handles ambiguous cases (bare identifier could be function call or variable reference)
## Grammar Architecture
## Architecture
See `src/parser/example.shrimp` for language examples and `src/parser/shrimp.grammar` for the full grammar.
**parser/** - Lezer grammar and tokenizers that parse Shrimp code into syntax trees
**editor/** - CodeMirror integration with syntax highlighting and language support
**compiler/** - Transforms syntax trees into ReefVM bytecode for execution
### Key Token Types
The flow: Shrimp source → parser (CST) → compiler (bytecode) → ReefVM (execution)
- **Identifier** - Lowercase/emoji start, can contain dashes/numbers (assignable)
- **Word** - Any non-whitespace that isn't a valid identifier (paths, URLs, etc.)
- **FunctionCall** - Identifier followed by arguments
- **FunctionCallOrIdentifier** - Ambiguous case resolved at runtime
See `example.shrimp` for language examples and `src/parser/shrimp.grammar` for the full grammar.

149
build.ts
View File

@ -1,149 +0,0 @@
#!/usr/bin/env bun
import plugin from "bun-plugin-tailwind";
import { existsSync } from "fs";
import { rm } from "fs/promises";
import path from "path";
if (process.argv.includes("--help") || process.argv.includes("-h")) {
console.log(`
🏗 Bun Build Script
Usage: bun run build.ts [options]
Common Options:
--outdir <path> Output directory (default: "dist")
--minify Enable minification (or --minify.whitespace, --minify.syntax, etc)
--sourcemap <type> Sourcemap type: none|linked|inline|external
--target <target> Build target: browser|bun|node
--format <format> Output format: esm|cjs|iife
--splitting Enable code splitting
--packages <type> Package handling: bundle|external
--public-path <path> Public path for assets
--env <mode> Environment handling: inline|disable|prefix*
--conditions <list> Package.json export conditions (comma separated)
--external <list> External packages (comma separated)
--banner <text> Add banner text to output
--footer <text> Add footer text to output
--define <obj> Define global constants (e.g. --define.VERSION=1.0.0)
--help, -h Show this help message
Example:
bun run build.ts --outdir=dist --minify --sourcemap=linked --external=react,react-dom
`);
process.exit(0);
}
const toCamelCase = (str: string): string => str.replace(/-([a-z])/g, g => g[1].toUpperCase());
const parseValue = (value: string): any => {
if (value === "true") return true;
if (value === "false") return false;
if (/^\d+$/.test(value)) return parseInt(value, 10);
if (/^\d*\.\d+$/.test(value)) return parseFloat(value);
if (value.includes(",")) return value.split(",").map(v => v.trim());
return value;
};
function parseArgs(): Partial<Bun.BuildConfig> {
const config: Partial<Bun.BuildConfig> = {};
const args = process.argv.slice(2);
for (let i = 0; i < args.length; i++) {
const arg = args[i];
if (arg === undefined) continue;
if (!arg.startsWith("--")) continue;
if (arg.startsWith("--no-")) {
const key = toCamelCase(arg.slice(5));
config[key] = false;
continue;
}
if (!arg.includes("=") && (i === args.length - 1 || args[i + 1]?.startsWith("--"))) {
const key = toCamelCase(arg.slice(2));
config[key] = true;
continue;
}
let key: string;
let value: string;
if (arg.includes("=")) {
[key, value] = arg.slice(2).split("=", 2) as [string, string];
} else {
key = arg.slice(2);
value = args[++i] ?? "";
}
key = toCamelCase(key);
if (key.includes(".")) {
const [parentKey, childKey] = key.split(".");
config[parentKey] = config[parentKey] || {};
config[parentKey][childKey] = parseValue(value);
} else {
config[key] = parseValue(value);
}
}
return config;
}
const formatFileSize = (bytes: number): string => {
const units = ["B", "KB", "MB", "GB"];
let size = bytes;
let unitIndex = 0;
while (size >= 1024 && unitIndex < units.length - 1) {
size /= 1024;
unitIndex++;
}
return `${size.toFixed(2)} ${units[unitIndex]}`;
};
console.log("\n🚀 Starting build process...\n");
const cliConfig = parseArgs();
const outdir = cliConfig.outdir || path.join(process.cwd(), "dist");
if (existsSync(outdir)) {
console.log(`🗑️ Cleaning previous build at ${outdir}`);
await rm(outdir, { recursive: true, force: true });
}
const start = performance.now();
const entrypoints = [...new Bun.Glob("**.html").scanSync("src")]
.map(a => path.resolve("src", a))
.filter(dir => !dir.includes("node_modules"));
console.log(`📄 Found ${entrypoints.length} HTML ${entrypoints.length === 1 ? "file" : "files"} to process\n`);
const result = await Bun.build({
entrypoints,
outdir,
plugins: [plugin],
minify: true,
target: "browser",
sourcemap: "linked",
define: {
"process.env.NODE_ENV": JSON.stringify("production"),
},
...cliConfig,
});
const end = performance.now();
const outputTable = result.outputs.map(output => ({
File: path.relative(process.cwd(), output.path),
Type: output.kind,
Size: formatFileSize(output.size),
}));
console.table(outputTable);
const buildTime = (end - start).toFixed(2);
console.log(`\n✅ Build completed in ${buildTime}ms\n`);

17
bun-env.d.ts vendored
View File

@ -1,17 +0,0 @@
// Generated by `bun init`
declare module "*.svg" {
/**
* A path to the SVG file
*/
const path: `${string}.svg`;
export = path;
}
declare module "*.module.css" {
/**
* A record of class names to their corresponding CSS module classes
*/
const classes: { readonly [key: string]: string };
export = classes;
}

View File

@ -7,8 +7,7 @@
"packages/*"
],
"scripts": {
"pretest": "bun generate-parser",
"serve": "bun --hot src/server/server.tsx",
"dev": "bun generate-parser && bun --hot src/server/server.tsx",
"generate-parser": "lezer-generator src/parser/shrimp.grammar --typeScript -o src/parser/shrimp.ts"
},
"dependencies": {

View File

@ -1,35 +0,0 @@
#! /usr/bin/env bun
import { parser } from '../parser/shrimp.js'
import { evaluate } from '../interpreter/evaluator.js'
const log = (...args: any[]) => console.log(...args)
log.error = (...args: any[]) => console.error(...args)
const repl = async () => {
log()
log('\033[38;5;219m»……¬¯\033[0m Shrimp REPL \033[38;5;219m¯¬……«\033[0m')
log()
process.stdout.write('> ')
const context = new Map<string, any>()
for await (const chunk of Bun.stdin.stream()) {
const input = new TextDecoder().decode(chunk).trim()
try {
const tree = parser.parse(input)
const output = evaluate(input, tree, context)
log(output)
} catch (error) {
log.error(`${error.message}`)
}
process.stdout.write('> ')
}
}
try {
repl()
} catch (error) {
log.error(`Fatal Error: ${error.message}`)
}

View File

@ -1,31 +0,0 @@
import { type SyntaxNodeRef } from '@lezer/common'
export const nodeToString = (nodeRef: SyntaxNodeRef, input: string, maxDepth = 10) => {
const lines: string[] = []
function addNode(currentNodeRef: SyntaxNodeRef, depth = 0) {
if (depth > maxDepth) {
lines.push(' '.repeat(depth) + '...')
return
}
const indent = ' '.repeat(depth)
const text = input.slice(currentNodeRef.from, currentNodeRef.to)
let child = currentNodeRef.node.firstChild
if (child) {
lines.push(`${indent}${currentNodeRef.name}`)
while (child) {
addNode(child, depth + 1)
child = child.nextSibling
}
} else {
const cleanText = currentNodeRef.name === 'String' ? text.slice(1, -1) : text
lines.push(`${indent}${currentNodeRef.name} ${cleanText}`)
}
}
addNode(nodeRef)
return lines.join('\n')
}

57
src/editor/editor.css Normal file
View File

@ -0,0 +1,57 @@
#output {
flex: 1;
background: #40318D;
color: #7C70DA;
padding: 20px;
overflow-y: auto;
white-space: pre-wrap;
font-family: 'Pixeloid Mono', 'Courier New', monospace;
font-size: 18px;
}
#output.error {
color: #FF6E6E;
}
#status-bar {
height: 30px;
background: #1E2A4A;
color: #B3A9FF55;
display: flex;
align-items: center;
padding: 0 10px;
font-size: 14px;
border-top: 3px solid #0E1A3A;
border-bottom: 3px solid #0E1A3A;
display: flex;
justify-content: space-between;
}
#status-bar .left,
#status-bar .right {
display: flex;
justify-content: center;
gap: 2rem;
}
#status-bar .multiline {
display: flex;
.dot {
padding-top: 1px;
margin-right: 4px;
}
.active {
color: #C3E88D;
}
.inactive {
color: inherit;
}
}
.syntax-error {
text-decoration: underline dotted #FF6E6E;
}

View File

@ -4,26 +4,14 @@ import { shrimpTheme } from '#editor/plugins/theme'
import { shrimpLanguage } from '#/editor/plugins/shrimpLanguage'
import { shrimpHighlighting } from '#editor/plugins/theme'
import { shrimpKeymap } from '#editor/plugins/keymap'
import { log } from '#utils/utils'
import { log, toElement } from '#utils/utils'
import { Signal } from '#utils/signal'
import { shrimpErrors } from '#editor/plugins/errors'
import { ViewPlugin, ViewUpdate } from '@codemirror/view'
import { debugTags } from '#editor/plugins/debugTags'
import { getContent, persistencePlugin } from '#editor/plugins/persistence'
export const outputSignal = new Signal<{ output: string } | { error: string }>()
outputSignal.connect((output) => {
const outputEl = document.querySelector('#output')
if (!outputEl) {
log.error('Output element not found')
return
}
if ('error' in output) {
outputEl.innerHTML = `<div class="error">${output.error}</div>`
} else {
outputEl.textContent = output.output
}
})
import '#editor/editor.css'
import type { HtmlEscapedString } from 'hono/utils/html'
export const Editor = () => {
return (
@ -41,44 +29,78 @@ export const Editor = () => {
shrimpLanguage,
shrimpHighlighting,
shrimpErrors,
debugTags,
persistencePlugin,
debugTags,
],
})
requestAnimationFrame(() => view.focus())
}}
/>
<div id="status-bar"></div>
<div id="status-bar">
<div className="left"></div>
<div className="right"></div>
</div>
<div id="output"></div>
<div id="error"></div>
</>
)
}
const persistencePlugin = ViewPlugin.fromClass(
class {
saveTimeout?: ReturnType<typeof setTimeout>
export const outputSignal = new Signal<{ output: string } | { error: string }>()
update(update: ViewUpdate) {
if (update.docChanged) {
if (this.saveTimeout) clearTimeout(this.saveTimeout)
let outputTimeout: ReturnType<typeof setTimeout>
this.saveTimeout = setTimeout(() => {
setContent(update.state.doc.toString())
}, 1000)
}
outputSignal.connect((output) => {
const el = document.querySelector('#output')!
el.textContent = ''
let content
if ('error' in output) {
el.classList.add('error')
content = output.error
} else {
el.classList.remove('error')
content = output.output
}
destroy() {
if (this.saveTimeout) clearTimeout(this.saveTimeout)
}
clearInterval(outputTimeout)
const totalTime = 100
const speed = totalTime / content.length
let i = 0
outputTimeout = setInterval(() => {
el.textContent += content[i]
i++
if (i >= content.length) clearInterval(outputTimeout)
}, speed)
})
type StatusBarMessage = {
side: 'left' | 'right'
message: string | Promise<HtmlEscapedString>
className: string
order?: number
}
export const statusBarSignal = new Signal<StatusBarMessage>()
statusBarSignal.connect(async ({ side, message, className, order }) => {
document.querySelector(`#status-bar .${className}`)?.remove()
const sideEl = document.querySelector(`#status-bar .${side}`)!
const messageEl = (
<div data-order={order ?? 0} className={className}>
{await message}
</div>
)
const getContent = () => {
return localStorage.getItem('shrimp-editor-content') || ''
}
// Now go through the nodes and put it in the right spot based on order. Higher number means further right
const nodes = Array.from(sideEl.childNodes)
const index = nodes.findIndex((node) => {
if (!(node instanceof HTMLElement)) return false
return Number(node.dataset.order) > (order ?? 0)
})
const setContent = (data: string) => {
localStorage.setItem('shrimp-editor-content', data)
if (index === -1) {
sideEl.appendChild(toElement(messageEl))
} else {
sideEl.insertBefore(toElement(messageEl), nodes[index]!)
}
})

View File

@ -1,5 +1,6 @@
import { EditorView, ViewPlugin, ViewUpdate } from '@codemirror/view'
import { syntaxTree } from '@codemirror/language'
import { statusBarSignal } from '#editor/editor'
export const debugTags = ViewPlugin.fromClass(
class {
@ -10,7 +11,7 @@ export const debugTags = ViewPlugin.fromClass(
}
updateStatusBar(view: EditorView) {
const pos = view.state.selection.main.head
const pos = view.state.selection.main.head + 1
const tree = syntaxTree(view.state)
let tags: string[] = []
@ -23,10 +24,12 @@ export const debugTags = ViewPlugin.fromClass(
}
const debugText = tags.length ? tags.reverse().slice(1).join(' > ') : 'No nodes'
const statusBar = document.querySelector('#status-bar')
if (statusBar) {
statusBar.textContent = debugText
}
statusBarSignal.emit({
side: 'right',
message: debugText,
className: 'debug-tags',
order: -1,
})
}
}
)

View File

@ -1,28 +0,0 @@
import { outputSignal } from '#editor/editor'
import { Compiler } from '#compiler/compiler'
import { errorMessage, log } from '#utils/utils'
import { keymap } from '@codemirror/view'
import { run, VM } from 'reefvm'
export const shrimpKeymap = keymap.of([
{
key: 'Cmd-Enter',
run: (view) => {
const input = view.state.doc.toString()
runInput(input)
return true
},
},
])
const runInput = async (input: string) => {
try {
const compiler = new Compiler(input)
const vm = new VM(compiler.bytecode)
const output = await vm.run()
outputSignal.emit({ output: String(output.value) })
} catch (error) {
log.error(error)
outputSignal.emit({ error: `${errorMessage(error)}` })
}
}

View File

@ -0,0 +1,89 @@
import { outputSignal, statusBarSignal } from '#editor/editor'
import { EditorState } from '@codemirror/state'
import { Compiler } from '#compiler/compiler'
import { errorMessage, log } from '#utils/utils'
import { keymap } from '@codemirror/view'
import { VM } from 'reefvm'
let multilineMode = false
const customKeymap = keymap.of([
{
key: 'Enter',
run: (view) => {
if (multilineMode) return false
const input = view.state.doc.toString()
run(input)
return true
},
},
{
key: 'Alt-Enter',
run: (view) => {
if (multilineMode) {
const input = view.state.doc.toString()
run(input)
return true
}
multilineMode = true
view.dispatch({
changes: { from: view.state.doc.length, insert: '\n' },
selection: { anchor: view.state.doc.length + 1 },
})
updateStatusMessage()
return true
},
},
])
const singleLineFilter = EditorState.transactionFilter.of((transaction) => {
if (multilineMode) return transaction // Allow everything in multiline mode
transaction.changes.iterChanges((fromA, toA, fromB, toB, inserted) => {
console.log(`🌭`, { string: inserted.toString(), newline: inserted.toString().includes('\n') })
if (inserted.toString().includes('\n')) {
multilineMode = true
updateStatusMessage()
return
}
})
return transaction
})
export const shrimpKeymap = [customKeymap, singleLineFilter]
const updateStatusMessage = () => {
statusBarSignal.emit({
side: 'left',
message: multilineMode ? 'Press Alt-Enter run' : 'Alt-Enter will enter multiline mode',
className: 'status',
})
statusBarSignal.emit({
side: 'right',
message: (
<div className="multiline">
<span className={multilineMode ? 'dot active' : 'dot inactive'}></span> multiline
</div>
),
className: 'multiline-status',
})
}
requestAnimationFrame(() => updateStatusMessage())
const run = async (input: string) => {
try {
const compiler = new Compiler(input)
const vm = new VM(compiler.bytecode)
const output = await vm.run()
outputSignal.emit({ output: String(output.value) })
} catch (error) {
log.error(error)
outputSignal.emit({ error: `${errorMessage(error)}` })
}
}

View File

@ -0,0 +1,29 @@
import { ViewPlugin, ViewUpdate } from '@codemirror/view'
export const persistencePlugin = ViewPlugin.fromClass(
class {
saveTimeout?: ReturnType<typeof setTimeout>
update(update: ViewUpdate) {
if (update.docChanged) {
if (this.saveTimeout) clearTimeout(this.saveTimeout)
this.saveTimeout = setTimeout(() => {
setContent(update.state.doc.toString())
}, 500)
}
}
destroy() {
if (this.saveTimeout) clearTimeout(this.saveTimeout)
}
}
)
export const getContent = () => {
return localStorage.getItem('shrimp-editor-content') || ''
}
const setContent = (data: string) => {
localStorage.setItem('shrimp-editor-content', data)
}

View File

@ -36,7 +36,6 @@ export const shrimpTheme = EditorView.theme(
fontFamily: '"Pixeloid Mono", "Courier New", monospace',
caretColor: '#80A4C2', // soft blue caret
padding: '0px',
minHeight: '100px',
},
'.cm-activeLine': {
backgroundColor: 'transparent',

View File

@ -1,26 +0,0 @@
import { ContextTracker } from '@lezer/lr'
import { Assignment } from '#parser/shrimp.terms'
interface ParserContext {
definedVariables: Set<string>
}
export const contextTracker = new ContextTracker<ParserContext>({
start: { definedVariables: new Set() },
reduce(context, term, stack, input) {
console.log(`🤏 REDUCE`, termToString(term))
if (term !== Assignment) return context
return context
},
shift(context, term, stack, input) {
console.log(` ⇧ SHIFT `, termToString(term))
return context
},
})
const termToString = (term: number) => {
return Object.entries(require('./shrimp.terms')).find(([k, v]) => v === term)?.[0] || term
}

View File

@ -349,7 +349,13 @@ describe('Assign', () => {
})
})
describe('whitespace', () => {
describe('multiline', () => {
test.only('parses multiline strings', () => {
expect(`'first'\n'second'`).toMatchTree(`
String first
String second`)
})
test('trims leading and trailing whitespace in expected tree', () => {
expect(`
3

View File

@ -1,4 +1,4 @@
@external propSource highlighting from "./highlight.js"
@external propSource highlighting from "./highlight"
@skip { space }
@ -15,7 +15,7 @@ statement {
NamedArgPrefix { $[a-z]+ "=" }
Number { "-"? $[0-9]+ ('.' $[0-9]+)? }
Boolean { "true" | "false" }
String { '\'' !["]* '\'' }
String { '\'' ![']* '\'' }
newlineOrSemicolon { "\n" | ";" }
eof { @eof }
space { " " | "\t" }
@ -31,7 +31,7 @@ statement {
"/"[@name=operator]
}
@external tokens tokenizer from "./tokenizers" { Identifier, Word }
@external tokens tokenizer from "./tokenizer" { Identifier, Word }
@precedence {
multiplicative @left,
@ -78,14 +78,14 @@ NamedArg {
}
FunctionDef {
singleLineFunctionDef | multiLineFunctionDef
singleLineFunctionDef | multilineFunctionDef
}
singleLineFunctionDef {
"fn" Params ":" expression
}
multiLineFunctionDef {
multilineFunctionDef {
"fn" Params ":" newlineOrSemicolon (expression newlineOrSemicolon)* "end"
}

View File

@ -1,7 +1,7 @@
// This file was generated by lezer-generator. You probably shouldn't edit it.
import {LRParser} from "@lezer/lr"
import {tokenizer} from "./tokenizers"
import {highlighting} from "./highlight.js"
import {tokenizer} from "./tokenizer"
import {highlighting} from "./highlight"
export const parser = LRParser.deserialize({
version: 14,
states: "'[OVQTOOOqQPO'#DTO!zQUO'#DTO#XQPOOOOQO'#DS'#DSO#xQTO'#CbOOQS'#DQ'#DQO$PQTO'#DVOOQO'#Cn'#CnOOQO'#C}'#C}O$XQPO'#C|OOQS'#Cu'#CuQ$aQTOOOOQS'#DP'#DPOOQS'#Ca'#CaO$hQTO'#ClOOQS'#DO'#DOOOQS'#Cv'#CvO$oQUO,58zO%SQTO,59_O%^QTO,58}O%^QTO,58}O%eQPO,58|O%vQUO'#DTO%}QPO,58|OOQS'#Cw'#CwO&SQTO'#CpO&[QPO,59qOOQS,59h,59hOOQS-E6s-E6sQOQPOOOOQS,59W,59WOOQS-E6t-E6tOOQO1G.y1G.yOOQO'#DT'#DTOOQO1G.i1G.iO&aQPO1G.iOOQS1G.h1G.hOOQS-E6u-E6uO&xQTO1G/]O'SQPO7+$wO'hQTO7+$xO'rQPO'#CxO(TQTO<<HdOOQO<<Hd<<HdOOQS,59d,59dOOQS-E6v-E6vOOQOAN>OAN>O",
@ -12,7 +12,7 @@ export const parser = LRParser.deserialize({
propSources: [highlighting],
skippedNodes: [0],
repeatNodeCount: 4,
tokenData: ")P~ReXY!dYZ!ipq!dwx!nxy#ryz#wz{#|{|$R}!O$W!P!Q$y!Q![$`![!]%O!]!^!i!_!`%T#T#X%Y#X#Y%h#Y#Z&c#Z#h%Y#h#i([#i#o%Y~~(z~!iOo~~!nO{~~!qUOr!nsw!nwx#Tx;'S!n;'S;=`#l<%lO!n~#YU]~Or!nsw!nwx#Tx;'S!n;'S;=`#l<%lO!n~#oP;=`<%l!n~#wOu~~#|Ox~~$ROW~~$WOY~~$]PZ~!Q![$`~$eQ^~!O!P$k!Q![$`~$nP!Q![$q~$vP^~!Q![$q~%OOX~~%TOe~~%YOh~Q%]Q!_!`%c#T#o%YQ%hOaQR%kS!_!`%c#T#b%Y#b#c%w#c#o%YR%zS!_!`%c#T#W%Y#W#X&W#X#o%YR&]QfP!_!`%c#T#o%Y~&fT!_!`%c#T#U&u#U#b%Y#b#c(P#c#o%Y~&xS!_!`%c#T#`%Y#`#a'U#a#o%Y~'XS!_!`%c#T#g%Y#g#h'e#h#o%Y~'hS!_!`%c#T#X%Y#X#Y't#Y#o%Y~'yQ_~!_!`%c#T#o%YR(UQcP!_!`%c#T#o%Y~(_S!_!`%c#T#f%Y#f#g(k#g#o%Y~(nS!_!`%c#T#i%Y#i#j'e#j#o%Y~)PO|~",
tokenData: "(j~ReXY!dYZ!ipq!dwx!nxy#]yz#bz{#g{|#l}!O#q!P!Q$d!Q![#y![!]$i!]!^!i!_!`$n#T#X$s#X#Y%R#Y#Z%|#Z#h$s#h#i'u#i#o$s~~(e~!iOo~~!nO{~~!qTOw!nwx#Qx;'S!n;'S;=`#V<%lO!n~#VO]~~#YP;=`<%l!n~#bOu~~#gOx~~#lOW~~#qOY~~#vPZ~!Q![#y~$OQ^~!O!P$U!Q![#y~$XP!Q![$[~$aP^~!Q![$[~$iOX~~$nOe~~$sOh~Q$vQ!_!`$|#T#o$sQ%ROaQR%US!_!`$|#T#b$s#b#c%b#c#o$sR%eS!_!`$|#T#W$s#W#X%q#X#o$sR%vQfP!_!`$|#T#o$s~&PT!_!`$|#T#U&`#U#b$s#b#c'j#c#o$s~&cS!_!`$|#T#`$s#`#a&o#a#o$s~&rS!_!`$|#T#g$s#g#h'O#h#o$s~'RS!_!`$|#T#X$s#X#Y'_#Y#o$s~'dQ_~!_!`$|#T#o$sR'oQcP!_!`$|#T#o$s~'xS!_!`$|#T#f$s#f#g(U#g#o$s~(XS!_!`$|#T#i$s#i#j'O#j#o$s~(jO|~",
tokenizers: [0, 1, tokenizer],
topRules: {"Program":[0,3]},
tokenPrec: 337

View File

@ -19,34 +19,3 @@ body {
display: flex;
flex-direction: column;
}
#output {
flex: 1;
background: #40318D;
color: #7C70DA;
padding: 20px;
overflow-y: auto;
white-space: pre-wrap;
font-family: 'Pixeloid Mono', 'Courier New', monospace;
font-size: 18px;
}
#output .error {
color: #FF6E6E;
}
#status-bar {
height: 30px;
background: #1E2A4A;
color: #B3A9FF;
display: flex;
align-items: center;
padding: 0 10px;
font-size: 14px;
border-top: 3px solid #0E1A3A;
border-bottom: 3px solid #0E1A3A;
}
.syntax-error {
text-decoration: underline dotted #FF6E6E;
}

View File

@ -10,7 +10,7 @@ const regenerateParser = async () => {
let generate = true
try {
const grammarStat = await Bun.file('./src/parser/shrimp.grammar').stat()
const tokenizerStat = await Bun.file('./src/parser/tokenizers.ts').stat()
const tokenizerStat = await Bun.file('./src/parser/tokenizer.ts').stat()
const parserStat = await Bun.file('./src/parser/shrimp.ts').stat()
if (grammarStat.mtime <= parserStat.mtime && tokenizerStat.mtime <= parserStat.mtime) {

View File

@ -22,7 +22,6 @@
"noUncheckedIndexedAccess": true,
"noImplicitOverride": true,
"baseUrl": ".",
"paths": {
"#*": ["./src/*"]
},