Skip to content
RESONON

Bind Audio to a Shader

Every audio track and the master have three live feeds you can bind to a visual with <<: .rms (the average level), .peak, and .fft (the spectrum). The feeds and their methods are in the MeterFeed and SpectrumFeed reference.

Bind a level

Section titled “Bind a level”

Declare a float uniform, and bind a level to it:

v.parameter("kick")  << drums.rms;     // one track
v.parameter("level") << master.peak;   // the whole mix

A level goes from 0 to 1. .range(lo, hi) maps it onto what the shader expects:

v.parameter("kick") << drums.rms.range(0.2, 3.0);

Stop it from flickering

Section titled “Stop it from flickering”

A raw level jumps on every transient, so a shader driven by it strobes. .smooth(ms) glides it; bigger is slower — about 100–200 ms feels punchy, 300–500 ms feels like a swell:

v.parameter("kick") << drums.rms.smooth(280);

To catch a hit at once and let it die away, give an attack and a release. 0 rises instantly:

v.parameter("kick") << drums.rms.smooth(20, 300);   // up in 20 ms, down over 300 ms
v.parameter("kick") << drums.rms.smooth(0, 400);    // a peak follower

Then drive something that can move smoothly — hue, glow, a radius — rather than multiplying the whole picture’s brightness by a transient.

Draw the spectrum

Section titled “Draw the spectrum”

Bound as a whole, .fft arrives as a sampler2D: a one-row texture, lowest frequency at x = 0, highest at x = 1. Read the bin under each column:

shaders/analyzer.glsl
#version 300 es
precision highp float;
uniform sampler2D spectrum;
uniform vec2 iResolution;
out vec4 fragColor;
void main() {
  vec2 uv = gl_FragCoord.xy / iResolution;
  float mag = texture(spectrum, vec2(uv.x, 0.5)).r;
  fragColor = vec4(vec3(step(uv.y, mag * 4.0)), 1.0);
}
use "std/instruments" { Sampler, Kit };
use "std/visuals" as gfx;


let v = gfx.load_shader("analyzer", "shaders/analyzer.glsl");


let drums = AudioTrack("drums");
drums.set_instrument(Sampler(Kit("CR-78")));
drums << [bd [hh sd hh] bd sd];


v.parameter("spectrum") << master.fft.smooth(120);
v >> screen;
PLAY;

.smooth on a spectrum glides every bin on its own, with one time or with an attack and a release (.smooth(20, 300)). Without it the bars flicker.

Each .fft binding is its own texture, so different tracks can drive different samplers:

v.parameter("drumSpec") << drums.fft.smooth(80);
v.parameter("bassSpec") << bass.fft.smooth(200);

React to a frequency band

Section titled “React to a frequency band”

To react to “the bass” without averaging bins in GLSL, reduce a range in Hz to one float with .band(lo, hi). The result is a level, so .smooth and .range chain on:

shaders/bands.glsl
#version 300 es
precision highp float;
uniform vec2 iResolution;
uniform float bass;
uniform float treble;
out vec4 fragColor;
void main() {
  vec2 uv = (gl_FragCoord.xy * 2.0 - iResolution) / min(iResolution.x, iResolution.y);
  float r = length(uv);
  float disc = smoothstep(0.6 + bass, 0.55 + bass, r);
  float ring = treble * smoothstep(0.02, 0.0, abs(r - 0.7));
  fragColor = vec4(vec3(disc) + ring * vec3(1.0, 0.7, 0.3), 1.0);
}
use "std/instruments" { Sampler, Kit };
use "std/visuals" as gfx;


let v = gfx.load_shader("bands", "shaders/bands.glsl");


let drums = AudioTrack("drums");
drums.set_instrument(Sampler(Kit("CR-78")));
drums << [bd [hh sd hh] bd sd];


v.parameter("bass")   << master.fft.band(20, 200).smooth(150).range(0, 0.4);
v.parameter("treble") << master.fft.band(4000, 12000).smooth(80).range(0, 4);
v >> screen;
PLAY;

.band reads the unsmoothed spectrum, so smooth the band — .band(…).smooth(ms) — not the spectrum before it.

Section titled “Related”