Audio Painting Pt4

After a long while of abandonment, my Audio Painter project has finally reached a first version of completion. By mixing the second prototype and the logic of the Sound Tracer, I’ve managed to create an audio painter with a deterministic behavior. Before this version the direction of the paint cursor movement was dictated by a Random Walker which didn’t quite reflect the nature of the song. This version has some very flexible settings to control the aspect of the image rendition of the audio. The result is exactly as i first envisioned it… 2 years ago. I think I’m gonna theme the images differently (maybe depending on the song, I guess), so that they can become like different views of little universes created by the sound properties of songs.

Drawback: It’s much more interesting to see the way the song is painted than it is to see the finished image, so this will probably work better as a multimedia installation or a live performance thing than as just little painterly images. So, a couple of videos will be coming soon.

Pj Harvey- White Chalk

At the Drive In-Rolodex Propaganda

Omar Rodriguez-Lopez - Boiling Death Request A Body To Rest Its Head On

So, the idea is that different music looks radically different… well not that radical, but it does look different. When working with live audio, it also looks very different for the voices of different people, so it’s fun so just sit in front on it and make it draw things (video for that also coming soon). More at my Flickr Page.

Sound tracer

Going back to the most basic expression of painting with sound.
This is meant to be a solution for the randomness of movement of the painting cursor in my other sound painting programs.

I was remembering that we used to do “vibration paintings” in school, holding a pen over a paper while on the bus, trying to transmit the vibrations of the road on the paper. I tried to reproduce this kind of effect by using the music and tracing a point.

A point leaded with audio driven forces. The direction is based on the frequency analysis, the weight is based on the amplitude of the frequencies. The latter version uses one point for the left channel and other for the right channel, it can be viewed as an Anagliph.
The upper one is done using Harrowdown Hill by Thom Yorke The lower one is done using Only if you run by Julian Plenti.

Audio Painting pt. 3

Another version of the audio Painter I’ve made, uses concentric colored circles (choosing colors much the same way as the other audio painters) and lets the user paint with mouse input the direction of the resulting image.

me whistling the beggining of "scars of time"

With this software, i wanted to make a visualization of different phonemes with the sounds they make. I speak Spanish natively (it shows, doesn’t it?) which thankfully does not have as many phonemes as English, so every letter was painted with… audio input of me pronouncing the main pronunciation of every letter (some have two… and h is mute but… i pronounced it as an aspired ‘j’)… so this visualization ends up being pretty interesting since the letters associate looks with each other, and those which are alike (which I pronounce the same) can easily be seen.

The Spanish alphabeth

This visualization is affected by human error (namely me). but… i find it interesting anyway :P

Audio Painting Pt2

In continuity with my interest in Audio Painings, i kept thinking on ways to create different tools to draw based on audio information. After the colored waveform, my next idea was to keep using frecuency information as source for color hue, but i was looking for a more organic result.

As a first try, I used filled circles around a point, assigning different frequencies to each and varying their size by the frequency magnitude at a given time. The position of this ‘brush’ changes by using a random walker driven by the sound’s level and the difference between the stereo channels.

organic?

After, I changed the circles for blended paint spots, inpired by Pollock and his randomness, and added support for capturing the sequence of painting with sync with the music.

As of now, this app has control over the step size of the brush, it’s size, and the spacing of the shapes around the center of the brush.

Aphex genious. Spectrograms

Spectrogram, Spectrogram, Spectrogram, that’s a classic audio visualization example. It’s not as uninformative as a waveform which just shows you the level vs. time (“just” shows you all the information the audio contains in a way not really easy to interpret). Spectrograms are prettier because they show you the spectrum of frequencies in the Y axis, the time in X axis and the current level of each frequency as a false color scale.

Fun fact, if you check the wikipedia page for spectrograms, It will show you the genious of Aphex Twin (Trent Reznor followed him later),  at hiding an image in a music track’s spectrogram by reverse-coding it, so that the image was used as source for the sound. the song is called: “math equation” from the Windowlicker album.

math equation

Even seeing the image of the hidden face in the wikipedia page, I wanted to code my own spectrogram maker and use it to check the song, in the last seconds of it, this was the resulting image:

Ow yeah

While coding this, the problem i noticed was that an spectrogram might be linear or logarithmic, consequently the image would look with long cartoon eyes if done linearly… funny enough it took me a almost a year to notice how to use a log scale (using, surprise, the log() function to map the Y values).  There is an applet for this, but openprocessing.com applets do not support audio inputs, so if you want to use it, you’re gonna have to download the code.

Flocking Music Painter test,

or when mixing things starts getting messy

Well, when i had several unfinished projects comes the time when I’m feeling tempted to mix them. So I took my Colored waveforms and mixed them with my flocking thingies… Completely ignoring audio levels and taking in account the tone frequencies converted in colors, turned the flock into a random number of bugs joined by lines that followed the mouse on click resulting in an audio colored flocking brush.

Cicatriz ESP chorus, sang by... me.

Also it turned into an interesting opportunity to try and write to se what kind of typographic artifacts would appear.

My Name, Also known as "i didn't sleep last night"

It’s pretty lovely to have a brush changing it’s color depending on input. Eventually I made a painting tool which could be controlled by singing or whistling. Eventually I will upload a video demo of how it works.

Audio Painting pt 1

Besides from Motion Graphics and Compositing, one of my interests on learning processing was to be able to create illustrations by using audio as the data source for visualization, not moving visuals, but paintings that were made of audio. This has been a long term project for me which has evolved on many different ways to create audio paintings, which will be logged in this blog over time.

The first attempt at creating representations of audio started as a particular new type of waveform. It started by my first looks at Jared Tarbell’s Sand Strokes. In which I thought they were stylish waveforms coming from audio sources… after reading, I noticed they were not, and started my own proyect for creating different waveforms which were more informative and more pleasant to the eye than the traditional ones.

The Mars Volta's "Frances the mute" album, Illustrated

So, thanks to minim i managed to easily read both input audio and audio from files, take their fft and interpret the audio’s frequency to create a false-color to add to the classic waveform. At that time i was drawing by using every audio sample with it’s frequency information, which probed itself not very useful when rendering high audio levels.

The nice thing about this version was that different genres of music yielded completely different results and the variety in the use of percussion and certain instruments was clearly visible… and white noise looked… white :D

Nice about this version also, was that different people’s voices returned different colors because of their different tones and everyone could expend a while staring at my computer’s screen making weird noises.