# Spectral Synthesis Library

A library for synthesizing audio from images using additive synthesis and perceptual frequency mapping.

## Features

- **Image-to-Audio Synthesis**: Convert any image to audio by treating it as a spectrogram
- **Perceptual Accuracy**: Uses Mel-scale frequency mapping for better perceptual results
- **Spectral Peak Detection**: Only synthesizes significant frequency components
- **Temporal Smoothing**: Maintains coherent trajectories between time frames
- **Auto-Detection**: Automatically handles different image types (spectrograms vs diagrams)

## Quick Start

```typescript
import { synthesizeFromImage, downloadWAV } from './spectral-synthesis'

// Simple usage
const audioData = synthesizeFromImage(imageData)

// With custom parameters
const audioData = synthesizeFromImage(imageData, {
  duration: 10,
  minFreq: 100,
  maxFreq: 10000,
  maxPartials: 200
})

// Export as WAV
downloadWAV(audioData, 44100, 'my-audio.wav')
```

## API Reference

### Main Functions

#### `synthesizeFromImage(imageData, params?)`
- **imageData**: `ImageData` - Canvas image data
- **params**: `Partial<SynthesisParams>` - Optional parameters
- **Returns**: `Float32Array` - Audio samples

### Types

#### `SynthesisParams`
```typescript
interface SynthesisParams {
  duration: number           // Audio duration in seconds
  minFreq: number           // Minimum frequency in Hz
  maxFreq: number           // Maximum frequency in Hz
  sampleRate: number        // Sample rate in Hz
  frequencyResolution: number // Frequency bin downsampling
  timeResolution: number    // Time slice downsampling
  amplitudeThreshold: number // Minimum amplitude threshold
  maxPartials: number       // Maximum simultaneous partials
}
```

## Project Structure

```
spectral-synthesis/
├── core/
│   ├── types.ts       # Type definitions
│   ├── utils.ts       # Helper functions
│   └── synthesizer.ts # Main synthesis logic
├── audio/
│   └── export.ts      # Audio export utilities
└── index.ts           # Main exports
```

## Algorithm

1. **Image Analysis**: Auto-detect if colors should be inverted
2. **Frequency Mapping**: Convert image rows to Mel-scale frequencies
3. **Peak Detection**: Find significant spectral components
4. **Temporal Smoothing**: Apply continuity between time frames
5. **Perceptual Weighting**: Apply psychoacoustic amplitude scaling
6. **Additive Synthesis**: Generate and sum sine waves

## Usage Examples

### Basic Synthesis
```typescript
const canvas = document.createElement('canvas')
const ctx = canvas.getContext('2d')
// ... load image to canvas
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height)
const audio = synthesizeFromImage(imageData)
```

### Advanced Usage
```typescript
import { ImageToAudioSynthesizer } from './spectral-synthesis'

const synthesizer = new ImageToAudioSynthesizer({
  duration: 5,
  maxPartials: 150
})

const result = synthesizer.synthesize(imageData)
console.log(`Generated ${result.duration}s of audio`)
```