MiXem

MiXem in action!

2007-12-16T01:00:00.000-08:00

Jose Olivares' notes (MIDI/code/Ableton)

2007-12-15T22:38:00.000-08:00

So in this post I'll talk a bit about the project in general, my introduction to the world of MIDI via coding and using Ableton live in ways that I've never tried up until this point. In my blog you'll find pictures, videos and coding examples.

The MiXem started out as continuation to our physical midterm project. For that project I worked with Matt Young and HeeJin Joo. Basically the midterm was called Sonic Cubes and consisted of acrylic cubes that would trigger sound and light up inside. We used three cubes (due to time constraints) and spent a lot of time on the design, specifically trying to achieve a very clean and simple interface.

After that midterm was done we all decided to keep working on it and this time make some new improvements to them such as having them send wireless data, visual feedback, expressive sound controls and more cubes overall. So I was very happy to learn that Tymm Twillmann wanted to join our team, and then later on he invited Dave Spector to join the team so that meant we had another musician in the group.

As Matt and Tymm already stated, we met for the first couple of weeks to try to decide what we actually wanted to do and how to do it. Conflicts of schedule, and the fact that we were so many in the group made these meetings hard to organize, but I think that considering the number of times we got together we came out with a very good idea. Ever since Matt, HeeJin and I started to work together we really wanted to do something that had to do with my music and real-time controls. The original purpose of the MiXem was to make it a device that I could take on the road and play with my band. I was very honored that my team wanted to do this, as it meant creating a custom-built instrument that I can use.

And it was this process of trying to define the project where we spent a lot of time, and we all had to talk extensively about it. Was this project going to be a musical instrument or a MIDI interface? That was the question that took weeks to answer. Since the set-up of my band deals with electronic and acoustic instruments, and I spend a lot of time working with my laptop during the live performances, I was more inclined towards the MIDI interface approach since it's something that is very flexible and can serve different purposes depending on how you set it up. When working live with my laptop I do a lot of sound processing and manipulation, and that's something that I wanted to incorporate in the project that could be used in my live shows or as part of a sound installation where the visitor can interact with the sounds being played back. Dave also proposed the idea of doing a sequencer-type device where the user can build his/her own song on the fly and have the control to select the instruments he/she wants for that song. I also liked the idea, but with the design that we had at that moment (using cubes on a grid) it seemed like something that could be kind of confusing and that it wouldn't serve effectively the sound processing aspect, but we didn't discard that functionality at all. I also liked the idea of having controls to change the sound because that was something that we also wanted to link with the visual aspect.

So after a few weeks of trying to define the device, we got to a point where we had to say exactly what the project was going to be because at that point we've been going back and forth between instrument, sequencer, audio-visual installation, or MIDI controller. At the beginning I didn't understand very well Dave's proposal of how he thought the machine could work, and I had a hard time trying to fit that with the rest of our ideas, specifically sound and visual manipulation . Tymm brought the possibility of using sensors similar to iPod scrollwheel where we could trigger loops and also manipulate sound. I thought that our problem was solved there, but then began the logistics nightmare of making that interface talk with Processing to handle the a/v side. From my previous attempt at working with sound in Processing I knew that the sound libraries where not that good for what we wanted to do. So I suggested using MIDI and bypassing Processing in favor of working with Ableton Live which is the software that use when I play live and that I most comfortable with. With this decision also came the final stage f defining the instrument when I decided to work with Dave's idea of a sequencer since this is something that I can also use and it meant that I would have to try a new performance/composition approach that I wanted to do for a while. Now theinterface design consisted of 8 sensors that would handle 56 loops, spread across 8 instruments.

Very good. The next step was then figuring out how to send data from the Arduino to Ableton live. I was aware that Tom Igoe had a MIDI out lab in his website but I only thought that would only work with external synth modules and not software. One day looking through the ITP blogs I saw that someone sent MIDI note messages to Garageband. That was very helpful, so Matt and I decided to do the MIDI lab. And by using a MIDISport 2 X2 box we sent MIDI data to Garageband, and it worked. The next step was to learn then how to send continuous controller messages because we where going to work with event handling and not note-ons/note-offs. With a lot of help from Tymm we set-up a code in the Arduino that would identify three pots as CCs and they would send sensor data via MIDI (Channel 0) to Ableton using. They worked very good, but we had some issues with the sensor values not being stable and the would fluctuate a bit when we hooked them up to parameters controls such reverb decay, resonance amouunt, etc. But since the project consisted of only triggering sounds, we knew that switches would work very good and they would be stable.

At this point, I then started working with Dave on setting up a series of switches that would trigger loops depending on the state they where. This was by far the hardest part of the project. Being myself not very fluent in programming I had a hard time trying to grasp the logic of "if statements", "for loops" and states. I'm very used to directly hooking up things and just make them work directly on the interface without having to resort to any code. We started out with 4 switches that would, in theory, send 16 different CC messages such as instrument selection, loop selection, and loop muting. The way that the switches had to work meant that we had to state what was the first switch pressed and then the second switch pressed so that we can trigger a loop, and that if let go of the the both of them they would go back to a state where the program resets and you can do this all over again and trigger the loop that you want in another instrument. Another nightmare. We had a LOT of issues with this mainly because we didn't know how to tell the program to handle this. And on top of that we also had to figure out how to define each event with a particular continuous controller number that we could use to define in Ableton what loop we wanted to trigger. We spent weeks and weeks working on this mainly because the code needed more things that we didn't know. Also there where two things that made the integration with Ableton live very difficult and that was that Dave wanted a switch to send more than one CC message. For those not very familiar with Ableton Live and the way it handles the MIDI mapping is by going into a mode where to map each controller you just select the Edit Map option, select your parameter and then move your knob (or anything else) in your controller. It's done one by one and you can't manually input the CC number to a parameter, you have to send it from your controller. Because the logic behind our switches wasn't working very well we had to make the code work so that we can then do the MIDI mapping. Also Dave wanted to include an option where the user can go from one song to another. This meant then that we needed to add two more or switches or make the switches handle more than one thing. Again, another nightmare. This time because a trigger would have to select an instrument, trigger a loop, mute the loop and select another song and then control those other loops. We are talking about 4 different MIDI messages coming from the same trigger.

The next phase was when we started to cut out features and make the device easier to understand and to program. I thought that 56 loops for one song was too much, and that we only should have 6 circles. I also thought that it was better to have separate sensors to do the song selection, and to add a silent loop to each instrument and that could serve as our mute button. This made a bit easier the code, but still it was a hard task. Dave thought of making states so that we can define what was happening when a person would hold one switch and then another switch at the same time. We had to consider several factors such as the order in which the switches where pressed, what would happen if a person triggered the same switch twice, how to make these switches go from controlling one song to another and how to incorporate a mute button per instrument. After working a lot with Matt Parker he suggested the idea of using a 6X6 two-dimensional array so that for each song the switches would have a particular CC number if pressed in a particular order, so that they don't repeat. That was good because we had a problem where if you press switch 1 and then 4, it would be the same as pressing 4 and then 1. The way it ended working is that the program would constantly look for the input of the first and the second switch pressed and then depending on that it would look up a cc value in the 2-d array and then if this combination of switches corresponded to a particular state then it would send a message as a specific CC number. If you then switched to other song it would do the same but the program would add 36 to the CC number so that it's not confused with the first song. And by switching from one song to another we programmed these sensors that handle this to send a CC number to global loop stop command.

Arduino code:

#define CHANNEL 0x00

// Variables:

int state = 1;
int loopActive = -1;

boolean daveSong = false;

int firstPressed = -1;
int secondPressed = -1;
int musical_Array [6][6]={
{
1,7, 8, 9,10,11 }
,
{
12,2,13,14,15,16 }
,
{
17,18,3,19,20,21 }
,
{
22,23,24,4,25,26 }
,
{
27,28,29,30,5,31 }
,
{
32,33,34,35,36,6 }

};

int DigitalValue1 = 0;
int DigitalValue2 = 0; // value from thedigital input

int ledPin = 13;

void setup() {

pinMode(13, INPUT);
pinMode(12, INPUT);
tlc5940_init();
Serial.begin(31250);
// set the states of the I/O pins:
// pinMode(switchPin, INPUT);
//Serial.begin(9600);
// pinMode(ledPin, OUTPUT);
// Set MIDI baud rate:
// blink(3);
}

void sendController(char controller_num, char controller_val)
{
Serial.print(0xB0 | CHANNEL, BYTE); // MIDI Channel 0
Serial.print(controller_num, BYTE); // Midi controller #
Serial.print(controller_val, BYTE); // Value of the MIDI controller
}

// Blinks an LED 3 times
//void blink(int howManyTimes) {
// int i;
// for (i=0; i< howManyTimes; i++) {
// digitalWrite(ledPin, HIGH);
// delay(100);
// digitalWrite(ledPin, LOW);
// delay(100);
// }
//}

void loop(){

if (state == 1) { //state 1 waits for you to press a button
loopActive = -1;
// Serial.println("you are in beatiful state 1");
for (int i = 5; i < 11; i++){ //keep checking our pins to see if anything has been pressed
DigitalValue1 = digitalRead(i); //read the number we are on (pins 5 to 12)

if (DigitalValue1 == HIGH){ //if you push a switch
if (i == 9) {
state = 7;
}
else if (i == 10) {
state = 8;
}
else {
firstPressed = (i-5); //assign the value of i as first pressed (we subtract 5 to match the values of our array(for example 8-5 = value 3))
// Serial.println("The first button is:");
// Serial.println(firstPressed);
state = 2; //you pressed a button/chose an instrument
// delay(400);
}
}
}
}

else if (state == 2) { //state two waits for you to press a second button
// Serial.println("you are in lovely state 2");

for (int i = 5; i < 9; i++){ //keep checking to see if anything has been pressed
DigitalValue2 = digitalRead(i);
DigitalValue1 = digitalRead(firstPressed+5);
if ((DigitalValue2 == HIGH) && (i != (firstPressed + 5))){
secondPressed = (i-5);
// Serial.println("The second button is:");
// Serial.println(secondPressed);
//delay(400);
state = 3; //you pressed a second button (chose a loop for that instrument)
}
if (DigitalValue1 == LOW) {
if (loopActive== -1) {
state = 6;
}
if (loopActive== 1) {
state=1;
}
}
}
}

else if (state == 3) { //state 3 checks the array and sends the MIDI value to Ableton
//Serial.println("you are in magnificent state 3");
//Serial.println("THE VALUE IS: ");

if (!daveSong) { //if not daveSong
//Serial.println(musical_Array[secondPressed][firstPressed], DEC);
sendController(musical_Array[secondPressed][firstPressed], 127); //set the Continuous Controller # to the value from the array
loopActive = 1;
//delay(1000);
state = 4; //the MIDI signal was sent to Ableton--now wait for buttons to be released
}

else { //else if dave song == true
//Serial.println(musical_Array[secondPressed][firstPressed] + 36, DEC);
sendController(musical_Array[secondPressed][firstPressed] + 36, 127); //set the Continuous Controller # to the value from the array
loopActive = 1;
//delay(1000);
state = 4; //the MIDI signal was sent to Ableton--now wait for buttons to be released
}
}

else if (state == 4) { //state 4 waits for you to release a button
//Serial.println("you are in delightful state 4");
DigitalValue1 = digitalRead(firstPressed+5);
DigitalValue2 = digitalRead(secondPressed+5);
if (DigitalValue1 == HIGH) {
if (DigitalValue2 == LOW) {
secondPressed = -1;
state = 2;
}
}
if ((DigitalValue1 == LOW) && (DigitalValue2 == HIGH)) {
firstPressed = -1;
state = 5; //you let go of your instrument but still have the loop button pressed
}

}

else if (state == 5) { //state 5 occurs if you released button 1 before button 2--it stops reading until you release button 2
// Serial.println("it's state 5. like it or get out");
DigitalValue2 = digitalRead(secondPressed+5);
if (DigitalValue2 == LOW) {
secondPressed = -1;
state = 1; //you let go of both buttons----start checking again
}
}
else if (state == 6) { //state 6 mutes the instrument
//Serial.println("how did you get to state 6");
// sendController(musical_Array[firstPressed][firstPressed], 127);
//***tell the instrument to go to the muted loop******
firstPressed = -1;
loopActive = -1;
//delay(800);
state = 1; //start over
}

else if (state == 7) {
//Serial.println("this is state 7");
daveSong = false;
//Serial.println("this is jose's song");
sendController(73, 127);
//delay(100);
for (int i = 9; i < 11; i++){ //keep checking our pins to see if anything has been pressed
DigitalValue1 = digitalRead(i); //read the number we are on (pins 5 to 12)

if (DigitalValue1 == LOW) {
state = 1;
}
}
}
else if (state == 8) {
//Serial.println("this is state 8");
daveSong = true;
//Serial.println("this is dave's song");
sendController(73, 127);
//delay(100);
for (int i = 9; i < 11; i++){ //keep checking our pins to see if anything has been pressed
DigitalValue1 = digitalRead(i); //read the number we are on (pins 5 to 12)

if (DigitalValue1 == LOW) {
state = 1;
}
}
}
}

Now the fun part finally arrived! Dave and I decided to write original music for the project so each one of us made loops. Now I can finally test the MIDI controller! When I set out to compose my loops it was a first for me. I usually write music in a linear and additive way because I work with software step sequencers, so by fragmenting a song in sections and in loops is something that I do not really do. It was very fun to write musical phrases that would sound good when combined with any of the other instruments. I decided to compose my song by writing themes and variations in just one key (C major). I laid out my song in different layers: bass, beats, arpeggios, ambient pads, lead melodies, and a heavily processed melody harp that would act as a sound texture. I wrote 4 different phrases for each layer and made them to be 6 seconds in duration. With Ableton's live syncing features we have so that each loop can be triggered but they only start at the beginning of the next phrase (in our case every 2 bars @ 160 bpm) so that the phrases are together in sync and in harmony. Needless to say I learned a lot by doing this song, and now I think I can write more music using this approach that then I can trigger live and improvise more on the fly.

This project was very hard to do but I think it came out with a lot of the things that we wanted form the beginning. The deisgn is very beautiful, the lights work perfectly and add a lot, the sound reactive visuals look very good and the user experience is very gratifying. I learned a lot of programming and how to work in a group with something that I hold very close to my heart which is music. Is not very easy for me to work musically with other people since I have been playing with my current band mates for 7 years now and we have developed a musical understanding that is not easy to achieve. So for me to explain myself musically and explain how Ableton live works was a very difficult thing for me to do. Dave and I come from very different musical background and we perceive music composition and performance in very different ways, and to find a common ground between the two of us wasn't easy but we got there! I'm glad that even though the group had a lot of members, each one of us had a lot to do in the project and it shows. So, hopefully next year my band will get a few shows here in NYC and we can take out the MiXem!

Tymm Twillman - Notes on the Build

2007-12-15T20:09:00.000-08:00

I have some notes & code on my personal blog but especially since Matt has put so much work into documentation here I'd like to add a bit to this collection too, and talk a little about the process and some of the things we ran into along the way.

Matt, HeeJin and Jose had decided to continue their project from midterms, the block-based sequencer. I decided to join the team, as it seemed like they had some really good ideas that mainly needed a bit of rounding out and engineering work, and I needed a really good idea to persue.

A little later we found that Dave wasn't going to be working with the team he'd expected, and gladly welcomed him into the project.

We quickly started coming up with features and ideas and new things to add. There were several things that complicated issues; a number of us had pretty big deliverables in other classes, meeting times were difficult to work out with so many people, and backgrounds were diverse enough that it was hard to find common ground without leaving out things that were really important to team members. When we agreed on implementation details, they were generally technically advanced and (as it turned out) would require greater degrees of coordination or had to be serialized. We met week after week, trying to eliminate things which were superfluous or too difficult to implement and trying to find ideas which would bring the team together and allow us to focus on an actual build, and every week we had a new design which would end up knocking us back to... well, if not square one then square two. We started with a design based on a grid of magnetic sensors which would allow both triggering and adjustment of parameters based on rotation of blocks (deemed not quite the right interface), then moved to RFID and modular scanning arrays (decided that we needed an immersive visual element) then moved to a circular setup with a display in the middle (the RFID element turned out to be too complicated, plus as it turns out the technology wasn't there to do many of the things we wanted to do reasonably cheaply). We switched to iPod-style scroll wheels (required team to wait for scroll wheels to be done to develop most of interface, which it was decided was not a good idea), and finally ended up with the current proximity sensor design, as this made the system easy to prototype & test with normal switches, and the whole team could be working on different aspects of the design concurrently.

While the roles were mostly not completely vertical, we mostly ended up in the following:

Jose and Dave decided to tackle Midi code and the state machine, and work on sounds for the final piece.

HeeJin took up the visual development, including designs for the pads and Processing code to give visual feedback to users & for performance.

Matt took on fabrication & testing.

I took on building out the hardware & interface code.

It ended up being an intense time for all of us. I'll let the others speak for their parts, but I'll give a rundown of the hardware build.

I had a board I'd put together years ago using qprox qt140 touch sensors, and I had a few random qt140's lying about. When we decided to move away from the touch wheel interface, I decided to pull them out as a simple-to-interface alternative. They have some configuration to set up, but for the most part they have 4 touch inputs & 4 digital outputs. Easy.

I brought the board to Matt & we played with it a bit with copper clad circles we were planning to use for touch sensors; while it seemed to work somewhat, the sensitivity was way too high. We decided that using switches for prototyping would be the best for Jose and Dave until we had this all a bit more worked out. I knew the range could be altered by adjusting capacitors on the board, and was able to test this to some degree with the initial board (enough to tell that we should be able to reach the range we wanted, but also enough to know that mounting was so important to the sensitivity that we shouldn't depend too much on results from the test board).

I spent a few hours & was able to create a PCB layout (using Eagle) that would work as a shield, holding two qt140's (with a careful groundplane layout) and support circuitry. Since the case wasn't quite ready I kept this board to work on code, and began to work on the LED control circuitry. TLC5490's had been talked about on some of the ITP lists before, so I decided to give some a shot (having played with, and been disappointed by, some Maxim i2c-based LED controllers in the past -- though that was likely largely due to my selection of LEDs at the time). They turned out to be a total pain in the ass, though in the end I think worth it.

I was getting low on time, so I decided to skip doing a full PCB layout for the 5940's (besides I had DIPs and for PCB layouts I generally prefer to do SMD). I just used wirewrap wire, sockets and pin headers and soldered a board to plug into the top of the qt140 board. It took a while, and soldering the LED's to little boards & attaching wires to plug into the 5940 board took another while.

I ended up needing to steal pins from the Arduino analog port, since the touch controllers took 8 pins for switches + 2 pins for control; the serial connection took another 2, and with only 2 remaining digital pins I couldn't handle all the 5940 connections necessary. So the TLC5940 board plugs into the Analog 0-4 (PORTC 0-4) pins, with two lines going to pins 10 and 11 (luckily I'd managed to leave two pins which can be timer-controlled available when doing the earlier layout).

Code was the biggest pain. There's an example on the Arduino site, but it didn't have quite the right interface and the clocking options weren't in the range I was looking for, so I ended up putting together my own code. I used some code I'd had lying around for a while (evolved from my first AVR-based design, years ago!) to do a color fade test, and when I got my first color cycled output I knew it was all worthwhile.

The team pulled together for a big build night; lots of cutting (Matt did a great job at putting together a hatbox-cutting jig & getting our case trimmed down to size and cutting holes for the LED's to shine through), hot gluing, tweaking and testing. I think we were all amazed, after everything we'd been through on building miXem, at just how great it looked in operation. The code required only minor tweaking to get the LED bits working along with the Midi code (initially the two parts weren't integrated; the LED code did its own sensing of what buttons were pushed, etc... but this turned out to be a pretty lousy interface). A few more changes of just a couple of lines brought us to the interface we showed in class.

Our biggest problem since the build has been occasional glitches; they seem to be due to the touch controllers getting confused (especially when items are placed on the table next to miXem, or when the case is opened & then re-closed). For the ITP show, we opened it and cleaned up the wiring a bit, and hot glued the Arduino in place in the center. This seems to have fixed most of these problems (though opening the case sometimes still causes it to become confused for a few moments). I learned here to pay close attention with the prox sensors to how they scan; it's covered in a short section in the datasheet, but especially when using multiple prox sensors in conjunction -- it's important to synchronize the chips (qt140's have a sync line; sync lines from all of them should be connected together) and to make sure that multiple pads aren't being scanned at the same time in close proximity to each other (which is a wiring layout issue). We were seeing (almost) rhythmic self-triggering in some of our initial tests where care wasn't given to wiring layout.

In the end, I think the project turned out well. I think most of the problems we ran into we were aware of and actively working to minimize (mainly with regards to trying to come to agreement on what & how to build, plus running into parts of the build that couldn't be parallelized), though I still hope to learn more about How to Quickly Develop Good Ideas With a Group.

Matt Young -- notes on QProx sensors and fabrication

2007-12-15T13:09:00.000-08:00

Well, here's what looks to be my final entry on the MiXem blog. It's been an awesome experience to work with this team and create something that we're all totally proud of, not to mention something that actually looks good and works and is fun to play. That said, I'm writing this entry in a way that will be useful to come back to in a year for any reference purposes. I'm going to detail my role in developing the touch sensors and the fabrication of MiXem's housing, so if you're reading this and don't care to know about those things, please stop here. Otherwise, let's get started:

For the sensors, we used Quantum Qprox QT140 proximity sensors attached to copper clad pc boarding. Creating the actual sensors was as easy as taking two-sided copper clad pc boarding and cutting it into 3” diameter discs. I also drilled holes in the middle (5/8”) to allow room for the LEDs that would live in the center.

However, it was attaching these sensors to the copper clad that proved a little tricky. The first time I tried this, I crudely laid a stripped wire on top of the copper clad and tried to solder it down. It took a good 20 minutes to get even a little bit of solder to stick to the copper and the wire, and even then the connection was not that trustworthy, not to mention ugly. Tymm then showed me that the right way to do this is to drill a tiny hole through each pc board, insert the end of a wire, solder that end, then file it down to be flush with the sensor. You get nice, tight connections this way with a mound of filed down solder connecting your wire to your sensors. He also showed me that a drop or two of resin (basically pine tar) on the copper will react when it's heated up by the iron to make the solder bond to the copper plate much better. And it smells good when you burn it as an added bonus.

I should mention here that it's important not to file the solder down to the point where you can see the copper wire through the silver-colored solder. If this is the case, the wire can easily become dislodged and your connection will suffer. In general, it's a good idea so solder both sides of the wire to the copper (which worked because we have two sided copper clad) just to make sure the connection stays strong. Oh, and another thing, adding that resin to the copper can badly burn the copper connection, and if this happens your solder will simply refuse to attach. There's nothing you can do with burnt copper once it happens, so it's a good idea to merely heat up the resin rather than char it.

Anyways, now that we had our sensors, it was time to test them out. Tymm had already done some work on a QProx chip and made a board with inputs/outputs and capacitors, so we used those to test. I wrote the following code to trigger LEDs to see how sensitive they were.

/*
*
**** PCompFinalSwitchTest2serial****
Basic switch test. Really just to see if we can get 4 switches to turn on LEDs.
The idea is then replace the LEDs with .Ess files. Comments by Matt Young
*
11/17/07
*
*/

// intialize your LEDs on their appropriate Digital pins
int ledPin7 = 7;
int ledPin8 = 8;
int ledPin12 = 12;
int ledPin13 = 13;

// same goes for the switches. Each switch according to its input on the Arduino
int switchPin2 = 2;
int switchPin3 = 3;
int switchPin4 = 4;
int switchPin5 = 5;

// initialize the state of each switch. Start them in the OFF position.
int switchState2 = 0;
int switchState3 = 0;
int switchState4 = 0;
int switchState5 = 0;

int inByte = 0;

void setup()
{

Serial.begin(9600);

//set each switch as an INPUT, using the pinMode function
pinMode (switchPin2, INPUT);
pinMode (switchPin3, INPUT);
pinMode (switchPin4, INPUT);
pinMode (switchPin5, INPUT);

//set each LED as an OUTPUT, using the pinMode function
pinMode (ledPin7, OUTPUT);
pinMode (ledPin8, OUTPUT);
pinMode (ledPin12, OUTPUT);
pinMode (ledPin13, OUTPUT);
}

void loop ()
{
// the state of your switch means READING the switch. Use digitalRead command
switchState2 = digitalRead(switchPin2);
switchState3 = digitalRead(switchPin3);
switchState4 = digitalRead(switchPin4);
switchState5 = digitalRead(switchPin5);

// if your switch state reads “on“, then...
if (switchState2 == 1) {
digitalWrite(ledPin7, HIGH); // turn on the LED
//Serial.print(switchState2, BYTE);
} else {
digitalWrite(ledPin7, LOW);
//Serial.print(switchState2, BYTE);
}

if (switchState3 == 1) {
digitalWrite(ledPin8, HIGH);
//Serial.print(switchState3, BYTE);
} else {
digitalWrite(ledPin8, LOW);
//Serial.print(switchState3, BYTE);
}

if (switchState4 == 1) {
digitalWrite(ledPin12, HIGH);
//Serial.print(switchState4, BYTE);
} else {
digitalWrite(ledPin12, LOW);
//Serial.print(switchState4, BYTE);
}

if (switchState5 == 1) {
digitalWrite(ledPin13, HIGH);
//Serial.print(switchState5, BYTE);
} else {
digitalWrite(ledPin13, LOW);
//Serial.print(switchState5, BYTE);
}

if (Serial.available() > 0) { // serial.available basically asks, “do we detect any incoming bytes?“. This says
// yes, if an incoming byte is detected, then execute these tasks:
inByte = Serial.read(); // GET the incoming byte.
Serial.print(switchState2, BYTE);
Serial.print(switchState3, BYTE);
Serial.print(switchState4, BYTE);
Serial.print(switchState5, BYTE);

}
} // that's it! We're done

This particular version of the code you'll notice contains information for Serial output. At the time, we were working towards building our loops in Processing, but we later abandoned this in favor of a MIDI protocol. Either way, this code works for making LEDs turn on when they're triggered, so that's primarily what we were concerned with.

One thing we noticed with the prox sensors is that they were super sensitive. Your hand could be, I dunno, 5 inches above the sensor and it would trigger. And because it was so sensitive, it acted in a buggy nature -- the LED was always blinking at irregular moments, due to either accidentally getting too close to it or from other electrical interference. Even buffering the sensor pads with acrylic and cardboard didn't affect their buggy-ness too much. To correctly temper their sensitivity, we changed capacitors from .1µF (microfarad) to .01µF. This made the sensors much less sensitive and buggy feeling, so we kept those for the final product. And just like that, our sensors were done. Here's a pic of the wiring... note the bright blue capacitors:

Fabrication of MiXem was another component of the project that I had a hand in. Building the housing was a little tricky, but I'm glad I got the opportunity to focus more on this part of the project. If I learned one thing with our midterm, it's to not discount the amount of time or energy that goes into actually building the prototype. Pretty things work better, right?

Anyways, MiXem's housing is built using 3 materials: a cardboard hatbox, acrylic, and spray paint. The acrylic came from Canal plastics, and is just a 12”x12”x1/8” plexiglass sheet cut and laser etched (by the fantastic AMS) into 12 2.5” circles. That's as easy as dropping off and picking up from AMS, and the finished product looks fantastic.

MiXem's housing is nothing more than a very sexy leopard skin hatbox cut down to size. We chose a hatbox due mainly to it's size and cost, and not so much for the leopard print. Working with the hatbox was the tricky part though -- being a large and round object meant that it didn't neatly fit underneath any of the power tools we have in the shop, so any operating on it had to be done by hand.

I started off needing to cut the box down to a manageable size. Knowing that we were intending to use the lid of the box as our touch surface for the acrylic, I aimed to cut the very bottom of the box down to a size that would allow enough room for MiXem's guts but still keep a low profile. I settled on 2” vertical.

Measuring this neatly around a curved surface was not quick. Making a clean line at an even 2” from the bottom of the box is not easily achieved with a (flat) ruler and scratch marks, and I only had one hat box, so I knew I had to get this right on the first shot. My solution to this was to make a jig that would score the hatbox evenly around it's edge. Spending a little time in the woodshop, I made something that resembled a c-clamp out of wood that was precisely the height of the box, and then making a thin cut at 2” into the jig to insert a blade. Here's a few pictures:

Anyways, the extra work to making the jig proved VERY handy, as I was able to score a nice, tight line around the box.

From here I had to cut the box, which was also a bit tricky. Like I said, because of the size of the hatbox, I couldn't exactly fit it underneath the head of the scroll saw. Besides, I had to be careful which tool to use, as any faster cuts could fray the edges of the cardboard and perhaps ruin it. My first instinct was to use a hacksaw, thinking that the fine teeth would cut a clean line. I experimented with a couple different ones I found in the lab, but none of them felt right. Then, by absolute chance, I stumbled across a regular serrated edge kitchen knife, and that turned out to work absolutely perfectly. It was small enough to give me good control but long enough to hold a straight line, and the edge was tight enough to not split the ends of the cardboard. Perfect. I made one pass around my line to score the box a bit deeper, and then made my cut. After this, I sanded the edge, put the lid on, and just like that we had our basic housing.

A quick word on this process though: one thing it taught me is the value of proper planning. Actually taking time to spec out the making of a tool like my scoring jig, and taking time to test how different tools react with your material like I did with my hacksaws, helped out tremendously in terms of how the final product looks. Word to the wise: build in plenty of time for this because it's worth it in the end. No doubt, it felt a little weird to take an hour or so to build a tool that I only used for 30 seconds, but it looked professional by the time I was done with it.

That being said, I now had my box and my lid. From here, however, I had to find a way to cut clean holes into the lid to allow room for our LEDs. I started this process by tracing out the footprint of the lid on white paper and basically making a blueprint of where our touch pads were going to go. Finding the center of the blueprint with a square and some measurements, I measured the lip of each touchpad to be 5” from the center after testing a few other distances. 5” felt about right for the instrument - Dave and I mimicked playing the instrument on the blueprint, and we both decided that 5” was the way to go. I then measured out at 60,120,180,240,300, and 360 degrees and scored where I'd drill my holes for the LEDs.

Drilling the holes for the LEDs proved to be problematic though. Using some scrap hatbox from earlier, I tested the drill press with the 5/8” bit and it simply tore it up. The cardboard frayed like crazy, so I knew drilling wouldn't work. I even tried taping both sides with painters tape to see if that would help suppress the fraying (on a great suggestion by Christian Cerrito, by the way), but that didn't help much. This cardboard was proving to be very delicate, so I had to find some other way. I'm definitely going to keep this taping trick in mind for the future though; even though cardboard isn't the right material for it, tape would be a good solution to this problem in many cases.

Eventually, I decided on, again, making a jig to help me out. This time, I simply drilled a hole into a piece of scrap acrylic 5/8” wide and used that as a template with which I could trace the 6 LED ports on the lid. From there, I cut each hole with an exacto knife by hand. This was an incredibly tedious process, but none of the cardboard got frayed. I also filed a couple of the rough edges down with the circular file, just to make it look a little prettier. The end result looked great.

Once I had my holes, it was just a matter of painting and hooking the sensors up inside. For paint, I went with a white spray enamel that I picked up from Kmart for $1.79 per canister. The enamel is a little more tricky to work with, but it does leave a nice glossy finish on the outside, sort of like an iPod. Sort of. I took this home to paint outside and, after letting this dry for about 5 or 6 hours, came back and we were ready to attach the sensors.

2007-12-12T12:25:00.000-08:00

Real life photos of the MiXem!

2007-12-11T16:46:00.000-08:00

The rumors were true.

Design

2007-12-02T18:15:00.000-08:00

This is Heejin's design for the acrylic switches. The idea here was to come up with something that we could send off to the Advanced Media Studio to be laser etched. We're pretty happy with the results so far. Check it:

Sensors test

2007-12-02T11:42:00.001-08:00

Well, we got the prox sensors working. We've decided to switch to something more like straight-up switches for the functionality of the sequencer. Originally, we wanted to do little iPod-scroll-wheel-like sensors, but the functionality of this would have been very complicated to both code and to use, so we dropped that idea in favor of something a bit more simple. Simple is good.

Anyways, below you have the video of how this worked. I wrote a very simple code to trigger different colored LEDs. The green and ultrabright blue LEDs are hooked up through a .1 uF capacitor that makes them especially sensitive. The yellow LED is wired to a .01 uF capacitor that makes for a bit better functionality I think: the switches don't accidentally trigger, which will be important as the circular shape of the sequencer will require the user to reach across certain switches to access others. So, lower capacitance means less buggy feeling.

Here's the video. Note the yellow LED and how it works consistently, and how the blue and green trigger erratically. Simple LED code below.

int ledPin1 = 8;
int ledPin2 = 12;
int ledPin3 = 13;

int switchPin4 = 4;
int switchPin3 = 3;
int switchPin2 = 2;

int switchState2 = 0;
int switchState3 = 0;
int switchState4 = 0;

void setup() {
pinMode (switchPin2, INPUT);
pinMode (switchPin3, INPUT);
pinMode (switchPin4, INPUT);

pinMode (ledPin1, OUTPUT);
pinMode (ledPin2, OUTPUT);
pinMode (ledPin3, OUTPUT);
}

void loop() {
switchState2 = digitalRead(switchPin2);
switchState3 = digitalRead(switchPin3);
switchState4 = digitalRead(switchPin4);

if (switchState2 == 1) {
digitalWrite(ledPin1, HIGH);
} else {
digitalWrite(ledPin1, LOW);
}

if (switchState3 == 1) {
digitalWrite(ledPin2, HIGH);
} else {
digitalWrite(ledPin2, LOW);
}

if (switchState4 == 1) {
digitalWrite(ledPin3, HIGH);
} else {
digitalWrite(ledPin3, LOW);
}
}

even better MIDI

2007-11-21T12:14:00.000-08:00

sweet. 3 potentiometers, tied to two types of pitch and frequency -- pay attention to the two circular dials at the bottom and the graph thing. all done in Ableton Live.

code to come.

MIDI anyone?

2007-11-20T10:26:00.000-08:00

Wait. Rather than muck up our coding in Processing, why not generate sound through MIDI? It makes better sound AND we can play it through Ableton Live AND it was invented in Kazakhstan in 1981. I think. And it means no gross coding in Processing.

Hi 5!

Actual pics & vid of Jose demonstrating:

some basic coding

2007-11-19T18:36:00.001-08:00

here comes the boring part of the project: the ugly birth of arduino/processing code. Basically, we started off just trying to get some switches to play sound. Seemed easy enough. Below is the video of how we're doing so far, followed by the code for Arduino/Processing for reference sake.

}

ARDUINO:

/*
*
**** PCompFinalSwitchTest2serial****
Basic switch test. Really just to see if we can get 4 switches to turn on LEDs.
The idea is then replace the LEDs with .Ess files, then replace the switches with
analog resistors. Comments by Matt Young
*
11/17/07
*
*/

// intialize your LEDs on their appropriate Digital pins
int ledPin7 = 7;
int ledPin8 = 8;
int ledPin12 = 12;
int ledPin13 = 13;

// same goes for the switches. Each switch according to its input on the Arduino
int switchPin2 = 2;
int switchPin3 = 3;
int switchPin4 = 4;
int switchPin5 = 5;

// initialize the state of each switch. Start them in the OFF position.
int switchState2 = 0;
int switchState3 = 0;
int switchState4 = 0;
int switchState5 = 0;

int inByte = 0;

void setup()
{

Serial.begin(9600);

//set each switch as an INPUT, using the pinMode function
pinMode (switchPin2, INPUT);
pinMode (switchPin3, INPUT);
pinMode (switchPin4, INPUT);
pinMode (switchPin5, INPUT);

//set each LED as an OUTPUT, using the pinMode function
pinMode (ledPin7, OUTPUT);
pinMode (ledPin8, OUTPUT);
pinMode (ledPin12, OUTPUT);
pinMode (ledPin13, OUTPUT);
}

void loop ()
{
// the state of your switch means READING the switch. Use digitalRead command
switchState2 = digitalRead(switchPin2);
switchState3 = digitalRead(switchPin3);
switchState4 = digitalRead(switchPin4);
switchState5 = digitalRead(switchPin5);

// if your switch state reads "on", then...
if (switchState2 == 1) {
digitalWrite(ledPin7, HIGH); // turn on the LED
} else {
digitalWrite(ledPin7, LOW);
}

if (switchState3 == 1) {
digitalWrite(ledPin8, HIGH);
} else {
digitalWrite(ledPin8, LOW);
}

if (switchState4 == 1) {
digitalWrite(ledPin12, HIGH);
} else {
digitalWrite(ledPin12, LOW);
}

if (switchState5 == 1) {
digitalWrite(ledPin13, HIGH);
} else {
digitalWrite(ledPin13, LOW);
}

if (Serial.available() > 0) { // serial.available basically asks, "do we detect any incoming bytes?". This says
// yes, if an incoming byte is detected, then execute these tasks:
inByte = Serial.read(); // GET the incoming byte.
Serial.print(switchState2, BYTE);
Serial.print(switchState3, BYTE);
Serial.print(switchState4, BYTE);
Serial.print(switchState5, BYTE);

}
} // that's it! We're done

PROCESSING:

import processing.serial.*;

Serial myPort;
int[] switchArray = new int [4];
int serialCount = 0;

boolean firstContact = false;

import krister.Ess.*;

AudioChannel mySample1;
AudioChannel mySample2;
AudioChannel mySample3;
AudioChannel mySample4;

void setup()
{
//intitializing serial ports
println(Serial.list()); // start off with printing the available serial ports
myPort = new Serial (this,Serial.list()[0],9600); // selecting this serial port, at #0

println("hello");
//println(myPort.available());
myPort.write(65);

//start the Ess library
Ess.start(this);

mySample1=new AudioChannel(); //set up audio channel
mySample1.loadSound("bass.aif"); // load the .wav file in it. Have your assets in a data folder

mySample2=new AudioChannel();
mySample2.loadSound("adore.wav");

mySample3=new AudioChannel();
mySample3.loadSound("guitar.aif");

mySample4=new AudioChannel();
mySample4.loadSound("synth.aif");

}

void draw()
{
if (firstContact == false) {
delay(300);
myPort.write(65);
println("I just sent some more data");
}
}

public void stop() { // note how "public void stop" is on it's own, outside of "draw" and "serialEvent"
Ess.stop();
super.stop();
}

void serialEvent(Serial myPort) {

if (firstContact == false) {
firstContact = true;
println("Data Received yo");
}

switchArray[serialCount] = myPort.read(); // read what you get from your serial port. Define that as your sensor array
serialCount++; // scroll through your sensor count

int val0 = switchArray[0]; // equate your values to particular spots in the array
int val1 = switchArray[1];
int val2 = switchArray[2];
int val3 = switchArray[3];

if (serialCount > 3) { // if we detect more than two bytes being sent, execute the below code.
println(myPort.available());

if (val0 > 0) { // if we get anything from switch 1, play sample 1.
mySample1.play(1);
}

if (val1 > 0) {
mySample2.play(1);
}

if (val2 > 0) {
mySample3.play(1);
}

if (val3 > 0) {
mySample4.play(1);
}

println(val0 + "\t" + val1 + "\t" + val2 + "\t" + val3); // let's just print our data for right now.
myPort.write(65);
serialCount = 0; // ...and then reset the serial counter back to 0, or otherwise it jams up.
}

new stuff

2007-11-17T16:30:00.000-08:00

ok well, physical sequencer may or may not be the best name for this project at this point. After a few meetings and tons of ideas floating around, we've arrived at this:

Again, Heejin's interpretation is awesome. The idea is to assign each circle an instrument, but once an instrument is selected (with your hand), the other circles will be "soft" buttons that control different effects.

Where are the cubes you ask? Good question. After chewing this over for a few hours, we decided that the appropriate appendage to use is the old-fashioned human hand rather than a new-fangled acrylic blinking cube. This will be awesome though -- as you touch one pad, it will animate with a tri-colored LED embedded underneath and respond with the assigned instrument. Then, using ______ sensors [insert link], you'll be able to control the effects.

We're using etched acrylic for each pad, tri-colored LEDs underneath, and ______ sensors to perform the effects. These will be located underneath each 3" pad. In the middle is an LCD monitor that will animate further.

More pics/ideas to come.

sound cubes, redux

2007-11-05T14:15:00.001-08:00

Hi all -- and welcome to Round II of Heejin, Jose, and Matt's sound cubes project for their Physical Computing midterm. I'm Matt, and from this point forward I'll post in first-person only. I promise. For Round I notes on this project, check out our midterm blog, which is helplessly out of date.

If you like movies, and I know you do, check this out:

Anyways, big changes this time around. First and foremost, we're welcoming a fourth member into our group, Tymm (picture forthcoming). Tymm's got some great ideas, and when we all met today we decided on blowing out this project. Changes to come:

1) We're going to focus on making this a sequencer rather than an instrument. This is important, because it will allow us greater flexibility in incorporating both audio and video.

2) Yes, we're going to incorporate video, or some sort of visual interface (most likely Processing sketches for now, which Heejin is working on).

3) We're gonna be tinkering around with better magnets and their effects, along with RFID tagging, as long as neither becomes cumbersome to the functionality of the sequencer.

4) Everyone gets a free taco! (pending Taco Bell's sponsorship).

Anyways, that's where we've been and where we're planning on going. Check back often for pictures, updates, and celebrity gossip.