Colorsynth - play music by touching colors
Ever wanted to touch a color with the tip of your finger and make it sound? Well that was our initial point of departure! By combining color sensor with a gyroscope and connecting it to an arduino we were aiming to make it happen! (or at least get a first prototype to test with).
So we set up the arduino. Via serial protocol we communicated the data to processing, which was in a loop with wekinator in order to recognize shapes. Maybe was not the most necessary gadget being able to switch from playing single note to chords by drawing a circle in the air, but hey we did it! Following data was send via a midi protocol to a virtual midi port which could then communicate with any virtual instrument.
So let's get to the interessting part, showcase videos! First video shows how I play notes when touching the colors. When puting up the device and drawing the circle in the air, it changes from one note to chords to even more chords and then back to one note. The gesture was trained using a neural network, which worked in the end in 80 % of the cases.
Second video shows a training mode we created. The screens dicates colors that should be played and the learned is playing those colors. The program detects if the right color was hit and gives according to that points and a score at the end.