oldboy/scripts/csound-parser/README.md

# Csound Manual Parser

A robust parser that converts the Csound reference manual into TypeScript reference files for use in code editors with hover tooltips and searchable documentation.

## Features

- Downloads markdown files directly from GitHub
- Parses 1000+ Csound opcodes
- Extracts structured data (syntax, parameters, examples)
- Generates organized TypeScript files by category
- Creates a main aggregator file with lookup functions
- Handles edge cases and provides detailed error reporting

## Installation

```bash
cd scripts/csound-parser
pnpm install
```

## Usage

### Option 1: Download and Parse (Recommended)

Download fresh markdown files from GitHub and parse them:

```bash
pnpm run download
```

This will:
1. Download all opcode markdown files from the Csound manual repository
2. Parse them into structured data
3. Generate TypeScript reference files in `src/lib/csound-reference/`

### Option 2: Parse Local Files

If you already have the Csound manual cloned locally:

```bash
pnpm run parse -- --input=/path/to/csound-manual/docs/opcodes
```

### Custom Output Directory

```bash
pnpm run parse -- --output=/custom/output/path
```

## Output Structure

The parser generates:

```
src/lib/csound-reference/
├── types.ts                              # Type definitions
├── csoundReference.ts                    # Main aggregator
├── signal-generators-basic-oscillators.ts
├── signal-modifiers-standard-filters.ts
├── mathematical-operations-trigonometric-functions.ts
└── ... (100+ category files)
```

Each category file contains an array of `CsoundReference` objects:

```typescript
export const signalGeneratorsBasicOscillators: CsoundReference[] = [
  {
    name: 'oscil',
    type: 'opcode',
    category: 'Signal Generators:Basic Oscillators',
    description: 'A simple oscillator without any interpolation.',
    syntax: 'ares = oscil(xamp, xcps [, ifn, iphs])\nkres = oscil(kamp, kcps [, ifn, iphs])',
    rates: ['a-rate', 'k-rate', 'i-rate'],
    parameters: [
      {
        name: 'ifn',
        description: 'function table number. Requires a wrap-around guard point.',
        type: 'initialization'
      },
      // ...
    ],
    seeAlso: ['oscili', 'poscil']
  },
  // ...
]
```

## Integration

Import in your application:

```typescript
import {
  allCsoundReferences,
  getCsoundReference,
  getCsoundReferencesByCategory
} from './lib/csound-reference/csoundReference'

// Get a specific opcode
const oscil = getCsoundReference('oscil')

// Get all oscillators
const oscillators = getCsoundReferencesByCategory('Signal Generators:Basic Oscillators')

// Search all references
const filtered = allCsoundReferences.filter(ref =>
  ref.description.includes('filter')
)
```

## Use Cases

### 1. Hover Tooltips in CodeMirror

Create a tooltip extension similar to the GLSL tooltips:

```typescript
import { hoverTooltip } from '@codemirror/view'
import { getCsoundReference } from './lib/csound-reference/csoundReference'

export const csoundTooltip = hoverTooltip((view, pos) => {
  const word = getWordAt(view, pos)
  if (!word) return null

  const reference = getCsoundReference(word.text)
  if (!reference) return null

  return {
    pos: word.from,
    end: word.to,
    above: true,
    create() {
      const dom = document.createElement('div')
      dom.innerHTML = `
        <div class="tooltip-header">
          <strong>${reference.name}</strong>
          <span>${reference.type}</span>
        </div>
        <div class="tooltip-description">${reference.description}</div>
        ${reference.syntax ? `<pre>${reference.syntax}</pre>` : ''}
      `
      return { dom }
    }
  }
})
```

### 2. Searchable Help Panel

Create a reference panel with fuzzy search:

```typescript
import { allCsoundReferences } from './lib/csound-reference/csoundReference'
import Fuse from 'fuse.js'

const fuse = new Fuse(allCsoundReferences, {
  keys: ['name', 'description', 'category'],
  threshold: 0.4
})

const results = fuse.search('oscillator')
```

### 3. Autocomplete

Use the reference data for intelligent autocomplete:

```typescript
import { allCsoundReferences } from './lib/csound-reference/csoundReference'

const completions = allCsoundReferences.map(ref => ({
  label: ref.name,
  type: ref.type,
  info: ref.description,
  detail: ref.syntax
}))
```

## Data Structure

### CsoundReference Interface

```typescript
interface CsoundReference {
  name: string           // Opcode name (e.g., "oscil")
  type: 'opcode' | 'keyword' | 'header' | 'constant'
  category: string       // Full category path
  description: string    // Brief description
  syntax?: string        // Syntax examples
  example?: string       // Code example
  rates?: string[]       // ['a-rate', 'k-rate', 'i-rate']
  parameters?: {
    name: string
    description: string
    type: 'initialization' | 'performance'
  }[]
  seeAlso?: string[]     // Related opcodes
}
```

## Parser Architecture

### 1. Downloader (`downloader.ts`)
- Fetches markdown files from GitHub API
- Handles rate limiting and retries
- Downloads to `downloaded-opcodes/` directory

### 2. Parser (`parser.ts`)
- Parses markdown frontmatter (id, category)
- Extracts sections using regex
- Extracts parameters from initialization/performance sections
- Handles modern and classic syntax variants
- Detects rate types (a-rate, k-rate, i-rate)

### 3. Generator (`generator.ts`)
- Groups opcodes by category
- Sanitizes category names for file names
- Generates TypeScript files with proper escaping
- Creates main aggregator with imports
- Provides lookup functions

## Troubleshooting

### Download Fails

If the GitHub download fails, manually clone the repository:

```bash
git clone https://github.com/csound/manual.git
cd manual
git checkout develop
```

Then run the parser with the local path:

```bash
pnpm run parse -- --input=../manual/docs/opcodes
```

### Missing Categories

Some opcodes may not have categories defined. The parser will skip these and log warnings.

### Parse Errors

The parser is robust and will continue parsing even if individual files fail. Check the console output for warnings about skipped files.

## Extending the Parser

### Adding New Extraction

To extract additional information from the markdown files, modify `parser.ts`:

```typescript
extractNewField(content: string): string {
  const section = this.extractSection(content, 'NewSection')
  // Parse the section content
  return parsed
}
```

Then add the field to the `CsoundReference` interface in `types.ts`.

### Custom Categories

To reorganize categories, modify the `category` field in the parser or create a mapping function in the generator.

## Performance

- Parsing 1000+ opcodes: ~2-5 seconds
- Generating TypeScript files: ~1-2 seconds
- Download from GitHub: ~30-60 seconds (network dependent)

## License

MIT