The original video can be seen here:
Sound-based video manipulationVisit the project
With P5.js its possible to get sound values from mp3 files. We split the mp4 to mp3 and video, got the sound values and implemented it in a shader.
WebGL made it possible to dynamically update video images based on sound.
This is where the magic happens.
Message us about anything.