Visuals & the Audio Engine

Visuals are driven by the same session as the music, but they never run on the audio thread. The audio has a deadline every few milliseconds; a frame has sixteen. Everything on this page follows from keeping those two apart while keeping them in time.

Time comes from the playhead

The built-in uniforms iTime, iCycle and iBeat are the playhead’s position, the same one the notes play from — not the GPU’s clock, and not wall time. iTime is the position in seconds, iCycle in cycles and iBeat in beats. A shader therefore moves in step with the audio it is shown with.

They do what the transport does: a pause holds them, a seek moves them, a stop rewinds them to 0. A tempo change bends iCycle and iBeat rather than making them jump. A signal bound to a uniform is read at the same position, and so is the span a sequenced screen lane shows.

Bound values are sampled once per frame

A uniform bound to a signal or a meter feed is read once for each frame the renderer draws, outside the audio thread. The audio engine does not know how many visuals read it, or how often. Binding a signal to a shader costs the audio nothing.

The feeds a shader reads are levels, and levels are spiky: a peak that lasts a few milliseconds jumps from one frame to the next. That is why the feeds have .smooth, applied on the visuals’ side of the fence — the same attack-and-release idea as a meter’s ballistics.

An analyzer runs only while it is used

A spectrum needs an FFT of the channel, and an FFT is real work. So an analyzer runs on a channel only while that channel’s .fft, or a band of it, is bound to a visual that reaches the screen. A channel no visual looks at costs nothing.

One channel has one analyzer, however it is read. Binding master.fft and master.fft.smooth(120) to two uniforms still runs one FFT; each different smoothing is its own texture, computed from that one analysis. .band reads the analysis before any smoothing, which is why a band is smoothed after it is taken.

A shader error can’t stop the music

A shader is compiled by the renderer, away from the audio. When it fails to compile, the audio plays on and the last shader that compiled keeps drawing. With a visuals window open, the GLSL error is sent back to the editor as a diagnostic, and it clears when the corrected shader compiles. With no window open nothing is drawn live, so nothing is compiled and no error is reported until one opens — or until render(screen) or screen.snap() draws the picture to a file.

The diagnostic is placed where you can fix it. A shader loaded from a .glsl file is marked in that file, on the line you wrote — the changes the engine makes to reach the GPU never shift it. A shader built from a string has no file of its own, so its error is marked on the >> screen line, with the line number inside the string in the message.

The Visual Graph — names, wiring and feedback
Bind Audio to a Shader — the feeds in practice
Signals & Time — the transport that drives both