what have i done

scripts/csound-parser/README.md

@@ -0,0 +1,277 @@
+# Csound Manual Parser
+
+A robust parser that converts the Csound reference manual into TypeScript reference files for use in code editors with hover tooltips and searchable documentation.
+
+## Features
+
+- Downloads markdown files directly from GitHub
+- Parses 1000+ Csound opcodes
+- Extracts structured data (syntax, parameters, examples)
+- Generates organized TypeScript files by category
+- Creates a main aggregator file with lookup functions
+- Handles edge cases and provides detailed error reporting
+
+## Installation
+
+```bash
+cd scripts/csound-parser
+pnpm install
+```
+
+## Usage
+
+### Option 1: Download and Parse (Recommended)
+
+Download fresh markdown files from GitHub and parse them:
+
+```bash
+pnpm run download
+```
+
+This will:
+1. Download all opcode markdown files from the Csound manual repository
+2. Parse them into structured data
+3. Generate TypeScript reference files in `src/lib/csound-reference/`
+
+### Option 2: Parse Local Files
+
+If you already have the Csound manual cloned locally:
+
+```bash
+pnpm run parse -- --input=/path/to/csound-manual/docs/opcodes
+```
+
+### Custom Output Directory
+
+```bash
+pnpm run parse -- --output=/custom/output/path
+```
+
+## Output Structure
+
+The parser generates:
+
+```
+src/lib/csound-reference/
+├── types.ts                              # Type definitions
+├── csoundReference.ts                    # Main aggregator
+├── signal-generators-basic-oscillators.ts
+├── signal-modifiers-standard-filters.ts
+├── mathematical-operations-trigonometric-functions.ts
+└── ... (100+ category files)
+```
+
+Each category file contains an array of `CsoundReference` objects:
+
+```typescript
+export const signalGeneratorsBasicOscillators: CsoundReference[] = [
+  {
+    name: 'oscil',
+    type: 'opcode',
+    category: 'Signal Generators:Basic Oscillators',
+    description: 'A simple oscillator without any interpolation.',
+    syntax: 'ares = oscil(xamp, xcps [, ifn, iphs])\nkres = oscil(kamp, kcps [, ifn, iphs])',
+    rates: ['a-rate', 'k-rate', 'i-rate'],
+    parameters: [
+      {
+        name: 'ifn',
+        description: 'function table number. Requires a wrap-around guard point.',
+        type: 'initialization'
+      },
+      // ...
+    ],
+    seeAlso: ['oscili', 'poscil']
+  },
+  // ...
+]
+```
+
+## Integration
+
+Import in your application:
+
+```typescript
+import {
+  allCsoundReferences,
+  getCsoundReference,
+  getCsoundReferencesByCategory
+} from './lib/csound-reference/csoundReference'
+
+// Get a specific opcode
+const oscil = getCsoundReference('oscil')
+
+// Get all oscillators
+const oscillators = getCsoundReferencesByCategory('Signal Generators:Basic Oscillators')
+
+// Search all references
+const filtered = allCsoundReferences.filter(ref =>
+  ref.description.includes('filter')
+)
+```
+
+## Use Cases
+
+### 1. Hover Tooltips in CodeMirror
+
+Create a tooltip extension similar to the GLSL tooltips:
+
+```typescript
+import { hoverTooltip } from '@codemirror/view'
+import { getCsoundReference } from './lib/csound-reference/csoundReference'
+
+export const csoundTooltip = hoverTooltip((view, pos) => {
+  const word = getWordAt(view, pos)
+  if (!word) return null
+
+  const reference = getCsoundReference(word.text)
+  if (!reference) return null
+
+  return {
+    pos: word.from,
+    end: word.to,
+    above: true,
+    create() {
+      const dom = document.createElement('div')
+      dom.innerHTML = `
+        <div class="tooltip-header">
+          <strong>${reference.name}</strong>
+          <span>${reference.type}</span>
+        </div>
+        <div class="tooltip-description">${reference.description}</div>
+        ${reference.syntax ? `<pre>${reference.syntax}</pre>` : ''}
+      `
+      return { dom }
+    }
+  }
+})
+```
+
+### 2. Searchable Help Panel
+
+Create a reference panel with fuzzy search:
+
+```typescript
+import { allCsoundReferences } from './lib/csound-reference/csoundReference'
+import Fuse from 'fuse.js'
+
+const fuse = new Fuse(allCsoundReferences, {
+  keys: ['name', 'description', 'category'],
+  threshold: 0.4
+})
+
+const results = fuse.search('oscillator')
+```
+
+### 3. Autocomplete
+
+Use the reference data for intelligent autocomplete:
+
+```typescript
+import { allCsoundReferences } from './lib/csound-reference/csoundReference'
+
+const completions = allCsoundReferences.map(ref => ({
+  label: ref.name,
+  type: ref.type,
+  info: ref.description,
+  detail: ref.syntax
+}))
+```
+
+## Data Structure
+
+### CsoundReference Interface
+
+```typescript
+interface CsoundReference {
+  name: string           // Opcode name (e.g., "oscil")
+  type: 'opcode' | 'keyword' | 'header' | 'constant'
+  category: string       // Full category path
+  description: string    // Brief description
+  syntax?: string        // Syntax examples
+  example?: string       // Code example
+  rates?: string[]       // ['a-rate', 'k-rate', 'i-rate']
+  parameters?: {
+    name: string
+    description: string
+    type: 'initialization' | 'performance'
+  }[]
+  seeAlso?: string[]     // Related opcodes
+}
+```
+
+## Parser Architecture
+
+### 1. Downloader (`downloader.ts`)
+- Fetches markdown files from GitHub API
+- Handles rate limiting and retries
+- Downloads to `downloaded-opcodes/` directory
+
+### 2. Parser (`parser.ts`)
+- Parses markdown frontmatter (id, category)
+- Extracts sections using regex
+- Extracts parameters from initialization/performance sections
+- Handles modern and classic syntax variants
+- Detects rate types (a-rate, k-rate, i-rate)
+
+### 3. Generator (`generator.ts`)
+- Groups opcodes by category
+- Sanitizes category names for file names
+- Generates TypeScript files with proper escaping
+- Creates main aggregator with imports
+- Provides lookup functions
+
+## Troubleshooting
+
+### Download Fails
+
+If the GitHub download fails, manually clone the repository:
+
+```bash
+git clone https://github.com/csound/manual.git
+cd manual
+git checkout develop
+```
+
+Then run the parser with the local path:
+
+```bash
+pnpm run parse -- --input=../manual/docs/opcodes
+```
+
+### Missing Categories
+
+Some opcodes may not have categories defined. The parser will skip these and log warnings.
+
+### Parse Errors
+
+The parser is robust and will continue parsing even if individual files fail. Check the console output for warnings about skipped files.
+
+## Extending the Parser
+
+### Adding New Extraction
+
+To extract additional information from the markdown files, modify `parser.ts`:
+
+```typescript
+extractNewField(content: string): string {
+  const section = this.extractSection(content, 'NewSection')
+  // Parse the section content
+  return parsed
+}
+```
+
+Then add the field to the `CsoundReference` interface in `types.ts`.
+
+### Custom Categories
+
+To reorganize categories, modify the `category` field in the parser or create a mapping function in the generator.
+
+## Performance
+
+- Parsing 1000+ opcodes: ~2-5 seconds
+- Generating TypeScript files: ~1-2 seconds
+- Download from GitHub: ~30-60 seconds (network dependent)
+
+## License
+
+MIT