Audio Analysis
Raw audio → MFCC, beat detection, energy curves, structural segmentation, CLAP embeddings
The Vision
What if a lighting designer's years of experience could be encoded into a model — and used to generate a full light show from a single audio file?
MLI is a private independent non-profit research project exploring neural audio-to-light synthesis. The goal: train a model that understands the relationship between sound and light the way a professional designer does.
Phase one focuses on learning brightness, energy, and beat-synchronization from real concert footage. Phase two will introduce synthetic training data generated by an LLM trained on professional lighting design literature.
This is not a finished product. This is research, documented openly.
The System (for the Proof-of-Concept)
Feature Extraction
96 engineered features per frame, sampled at 30fps, synchronized with video ground truth
Sequence Modeling
BiLSTM encoder processes temporal audio context. LSTM decoder generates frame-by-frame light grid predictions.
Grid Output
12×8 RGB grid representing stage lighting state at every frame — brightness, color, spatial distribution
Synthetic Pipeline (In Development)
LLM-based lighting designer generates training data at scale. Stage layouts, fixture distributions, music-reactive cue lists — all synthetic, all labeled. Human training data remains essential for validation and real-world progression.
Current Status
Training Data
Active
96 real concert recordings processed. Synthetic pipeline in development. Target: 10,000+ training samples.
Model
In Progress
CLAP + BiLSTM architecture. 3.3M trainable parameters. Beat-synchronization: functional. Style transfer: research phase.
Product
Planned
Private independent non-profit research project moving toward a desktop application for lighting designers. Built to connect with every major professional lighting workflow and control environment. Target: late 2026 to mid 2027.
Get In Touch
Research inquiries, collaboration, data partnerships.
If you work in professional lighting design and are interested in this private independent non-profit research project — or if you have access to lighting show recordings and would like to contribute to the research — reach out.
[ LinkedIn → ]External link disclaimer: LinkedIn is operated by LinkedIn Ireland Unlimited Company. MLI is not responsible for external content, data processing, tracking, availability, or changes on LinkedIn after leaving this website.