Elementary Audio I already explained. But it is the brainchild of Nick Thompson. He started this about 3 years ago. Prior, he had built some tooling to allow you to create the UI for music instruments and plugins via JUCE in React, which I thought was a pretty cool approach. He has a project using Elementary Audio and JUCE and his previous approach to using web technologies to build native AU and VST music plugins.

https://www.elementary.audio/https://github.com/elemaudio/srvb (project described abovespecific playground link If this sounds like jibberish, it is specialized software for creating instruments and effects for music production. Usually a very complicated c++ affair.

Update 26.6.25

Today I did some more diving into elementary audio and how I might get started.

my goal is 1) fast prototying with good DX locally, most likely in a webbrowser and 2) to build a full production auv3 app with this and a react based ui in a webview

So some key insights from a conversation with gpt jamie coach 4.0 are:

state is just a JS object.
describe() - not sure if this can be renamed, but it is passed to render() here you do all of your audio declaritive stuff and return el.out()
el. ⇐ just the elementary name spacing. sometimes seen as elem-audio ⇐ which is funny, almost reads like elm audio.
there is no actual ui. state is all on you to manage it. simple examples just have a directly mutated JS object.
there is no midi handling, you have to roll your own or inject that somehow into the state

side ideas about ui

use react-three-fiber and storybook to build the ui, create a clickable prototype
then wire this up for the web prototype.
then wire this up with some sort of state connector in a WKWebView
capacitor doesn’t really add any value as anything other than an auv3 is seen as a toy

Flow of the state

UI or Midi change state → passed to describe() function → el.render is called (updates state to the audio engine) ⇐ el.render(describe(state))

It is a bit functional lite, like elm vs. haskell.
not strictly pure let env = el.adsr(...
deterministic, composable, and side-effect managed

other thougths

not sure if you can access the input via webaudio renderer or if that is good, assuming you are building an effects unit. likely you would want a sample loop player that can be turned on via debug flag or special state feature
ui state and audio state are likely different things

next steps

start with either audio or ui bot noth both. try and get one micro win first.
build some knobs or audio sliders in storybook

more details in the gpt thread here: [ChatGPT thread removed]

update later in the day

downloaded and patched srbv example project to use juce 8 and fix the ui.

I have created a new branch feature/james-stone-starter-pack on my fork with a bunch of tooling upgrades and porting the existing code.

per cusor ide

TypeScript Migration Summary - 30k Foot View

Tooling Stack

TypeScript with strict mode configuration
Jotai for state management (replacing Zustand)
Storybook 9 for component development
ESLint + Prettier with automated formatting
Husky + lint-staged for git hooks
React Three Fiber for 3D UI components

Major Changes Made

Full TypeScript conversion with strict type checking (no any types)
Modern state management migration from Zustand to Jotai atoms
Component modernization with proper TypeScript interfaces and type guards
Development workflow enhancement with automated linting, formatting, and pre-commit hooks
Interactive prototyping setup with Storybook for rapid UI development
3D UI capabilities integrated for audio interface experimentation
Build process hardening with type checking enforced before builds
Error boundaries and proper error handling patterns

Key Philosophy

Type safety first - eliminated type assertions and escape hatches
Developer experience - fast feedback loops with proper tooling
Maintainability - strict linting and consistent code formatting
Modern React patterns - functional components with hooks and proper state management

The result is a fully modernized TypeScript development environment that maintains Elementary Audio compatibility while providing excellent DX for rapid prototyping and development.

side note, interesitng web audio 3d viz: https://tympanus.net/codrops/2025/06/18/coding-a-3d-audio-visualizer-with-three-js-gsap-web-audio-api/

TypeScript Migration Summary - 30k Foot View

Tooling Stack

TypeScript with strict mode configuration
Jotai for state management (replacing Zustand)
Storybook 9 for component development
ESLint + Prettier with automated formatting
Husky + lint-staged for git hooks
React Three Fiber for 3D UI components

Major Changes Made

Full TypeScript conversion with strict type checking (no any types)
Modern state management migration from Zustand to Jotai atoms
Component modernization with proper TypeScript interfaces and type guards
Development workflow enhancement with automated linting, formatting, and pre-commit hooks
Interactive prototyping setup with Storybook for rapid UI development
3D UI capabilities integrated for audio interface experimentation
Build process hardening with type checking enforced before builds
Error boundaries and proper error handling patterns

Key Philosophy

Type safety first - eliminated type assertions and escape hatches
Developer experience - fast feedback loops with proper tooling
Maintainability - strict linting and consistent code formatting
Modern React patterns - functional components with hooks and proper state management

🎵 ADDENDUM: Enhanced Audio-Visual Development Stack

🆕 Additional Tooling Stack

Animation & Visual Effects

GSAP + @gsap/react - Professional-grade animations for smooth audio UI interactions
Three.js + @types/three - 3D audio visualizations and reactive interfaces
Tone.js - Web Audio API abstraction for audio analysis and synthesis

Testing Infrastructure

Vitest - Fast, Vite-native testing framework with jsdom environment
@testing-library suite - React component testing with accessibility focus
Web Audio API mocks - Testing environment for audio-dependent components

UI & Form Management

Radix UI primitives - Headless, accessible component library
React Hook Form - Performant forms with minimal re-renders for parameter controls

🎨 Enhanced Development Capabilities

Audio-Reactive Development

Real-time audio visualization prototyping with Tone.js integration
Parameter-driven animations using GSAP for smooth knob/fader movements
3D audio interfaces for immersive plugin experiences
Web Audio API integration for development-time audio feedback

Advanced Testing Setup

Component isolation testing with mocked audio environments
Audio parameter validation testing
Visual regression capabilities through Storybook integration
Accessibility testing with React Testing Library

Professional UI Components

Accessible form controls with Radix UI primitives
Smooth micro-interactions powered by GSAP
Responsive audio visualizations with Canvas and WebGL support

🔧 Configuration Enhancements

Vite Configuration

// Added testing environment with jsdom
test: {
  environment: 'jsdom',
  globals: true,
  setupFiles: ['./src/test-setup.ts'],
}

Package.json Scripts

{
  "test": "vitest",
  "test:ui": "vitest --ui", 
  "test:coverage": "vitest --coverage"
}

Audio Development Mocks

Web Audio API simulation for testing environments
Elementary Audio global constants mocking
Cross-platform audio testing infrastructure

🎯 Audio Plugin Development Workflow

Prototyping Phase

Storybook development with live audio analysis (Tone.js)
GSAP animation testing for parameter responsiveness
3D interface experimentation with React Three Fiber

Production Phase

Parameter-driven visuals that work in native DSP mode
Accessibility-compliant controls with Radix UI
Comprehensive testing with Vitest coverage

Deployment Phase

Type-safe builds with strict TypeScript checking
Automated quality gates via git hooks
Visual documentation through Storybook builds

🚀 Result: Professional Audio Plugin Development Environment

The enhanced stack provides a complete audio plugin development ecosystem combining:

Rapid visual prototyping with real-time audio feedback
Production-ready UI components with accessibility built-in
Smooth animations that enhance the user experience
Comprehensive testing for reliable plugin behavior
Modern development workflow with instant feedback loops

This creates an unparalleled development experience for creating professional audio plugins while maintaining Elementary Audio’s performance advantages.

main.js (Elementary Audio DSP) ↕️ bi-directional JavaScript calls QuickJS Engine (embedded via CHOC) ↕️ bi-directional C++ ↔ JS bridge JUCE Plugin (native audio processing) ↕️ bi-directional plugin host protocol Host DAW (Logic, Pro Tools, etc.)

Yes, the Parameter System Has Tight Coupling Issues

Looking at the current architecture, you’ve identified a real problem:

🔗 Current Tight Coupling Problem

State Shape Defined in 4 Places:

manifest.json - Parameter definitions

json

Apply to manifest.jso…

{

“parameters”: [

{“paramId”: “size”, “name”: “Size”, “min”: 0, “max”: 1, “defaultValue”: 0.5}

]

}

Native C++ - Parameter handling

cpp

Apply to manifest.jso…

state.insert_or_assign(paramId, elem::js::Number(pr.value));

React UI - Interface components

typescript

Apply to manifest.jso…

props.requestParamValueUpdate?.(“size”, newValue)

DSP JavaScript - Parameter consumption

javascript

Apply to manifest.jso…

size: refs.getOrCreate(‘size’, ‘const’, {value: state.size}, [])

Any parameter changes require updates in all 4 places - very error-prone!

🎯 TypeScript on DSP Side: Pros & Cons

✅ Advantages:

1. Shared Type Definitions

typescript

Apply to manifest.jso…

// shared/types.ts

interface PluginState {

size: number;

decay: number;

mod: number;

mix: number;

sampleRate: number;

}

// Both UI and DSP use the same interface

2. Compile-Time Safety

typescript

Apply to manifest.jso…

// DSP side - TypeScript catches errors

globalThis.receiveStateChange = (serializedState: string) ⇒ {

const state: PluginState = JSON.parse(serializedState);

// TypeScript ensures state.size exists and is number

};

3. Refactoring Safety

Rename a parameter → TypeScript catches all places that need updating

❌ Disadvantages:

1. Build Complexity

Need TypeScript compilation step for DSP
QuickJS doesn’t natively understand TypeScript
Build pipeline becomes more complex

2. Debugging Complexity

text

Apply to manifest.jso…

TypeScript → JavaScript → QuickJS → Native debugging

Stack traces become harder to follow

3. Performance Considerations

Additional compilation step
Larger bundle size for DSP engine
QuickJS optimizations may be affected

🚀 Better Architecture Suggestions

Option 1: Code Generation from Schema

typescript

Apply to manifest.jso…

// plugin-schema.ts - Single source of truth

const PLUGIN_SCHEMA = {

parameters: {

size: { min: 0, max: 1, default: 0.5 },

decay: { min: 0, max: 1, default: 0.3 }

}

} as const;

// Generate:

// - manifest.json

// - TypeScript interfaces

// - React component props

// - DSP parameter handling

Option 2: Runtime Type Validation

javascript

Apply to manifest.jso…

// DSP side - runtime validation without TypeScript

import { validateState } from ’../shared/validation.js’;

globalThis.receiveStateChange = (serializedState) ⇒ {

const state = validateState(JSON.parse(serializedState));

// Runtime error if state shape is wrong

};

Option 3: Hybrid Approach

UI side: Full TypeScript with shared interfaces
DSP side: JavaScript with runtime validation
Shared: JSON schema for validation + type generation

🎯 My Recommendation

Don’t use TypeScript on DSP side yet - the complexity isn’t worth it for this use case.

Instead:

Extract parameter schema to a single source of truth
Generate interfaces for UI TypeScript
Generate validation for DSP JavaScript
Keep DSP simple - audio thread performance is critical

This gives you type safety where it matters (UI development) without the complexity/performance cost in the real-time audio engine.

Review changes

Edited 1 file

+35-148

Reject ⇧⌘⌫

Accept ⌘⏎

main.js

important files

/Users/james/workspace-local/srvb/public/manifest.json - parameter names from daw
/Users/james/workspace-local/srvb/native/PluginProcessor.cpp - the magic sauce between the UI and the dsp
dsp/main.js - dsp insertion point
public/dsp.main.js - likely built version goes here
src/ - the ui stuff
src/main - magic wrapper stuff and the usual
src/app - error boundry, some pass through
src/Interface - this is the big file of the UI

basically at the end of the day

massive diff

can’t check in because of es-lint, should be looser. there is a bunch of stuff I think we don’t need. store/atoms, types/jsx-modules, etc.

late at night looking at discord, these are probably worth a look / review

https://github.com/chromatone/elements/tree/main/elements

https://jake.cx/synth.html

https://github.com/Mozoloa/elemHPF

Codepen starting points

https://codepen.io/nick-thompson/pen/OJzPQZP

https://codepen.io/nick-thompson/pen/ExvWgJN?anon=true&view=pen

I am thinking with the above to create some basic controls with leva or ui-kit

Echoes: Notes & Signals by James StoneEchos

Explorer

Elementary Audio

So some key insights from a conversation with gpt jamie coach 4.0 are:

side ideas about ui

Flow of the state

other thougths

next steps

TypeScript Migration Summary - 30k Foot View

Tooling Stack

Major Changes Made

Key Philosophy

TypeScript Migration Summary - 30k Foot View

Tooling Stack

Major Changes Made

Key Philosophy

🎵 ADDENDUM: Enhanced Audio-Visual Development Stack

🆕 Additional Tooling Stack

Animation & Visual Effects

Testing Infrastructure

UI & Form Management

🎨 Enhanced Development Capabilities

Audio-Reactive Development

Advanced Testing Setup

Professional UI Components

🔧 Configuration Enhancements

Vite Configuration

Package.json Scripts

Audio Development Mocks

🎯 Audio Plugin Development Workflow

Prototyping Phase

Production Phase

Deployment Phase

🚀 Result: Professional Audio Plugin Development Environment

Yes, the Parameter System Has Tight Coupling Issues

🔗 Current Tight Coupling Problem

State Shape Defined in 4 Places:

🎯 TypeScript on DSP Side: Pros & Cons

✅ Advantages:

1. Shared Type Definitions

2. Compile-Time Safety

3. Refactoring Safety

❌ Disadvantages:

1. Build Complexity

2. Debugging Complexity

3. Performance Considerations

🚀 Better Architecture Suggestions

Option 1: Code Generation from Schema

Option 2: Runtime Type Validation

Option 3: Hybrid Approach

🎯 My Recommendation

important files

Codepen starting points

Table of Contents

Echoes: Notes & Signals by James Stone Echos