my website stylesheet -->

Arduinos, MIDI and real apps

Since it’s really been a while since I posted here, I’m going to publish my research on getting an Arduino based interface to talk directly to something else via MIDI. This project could easily be expanded to more complex solutions (i.e. not just two momentary switches), could easily use OSC, channel MIDI data across a network, have a more complex user interface. The point with this stage in the project was to get a framework to build new projects from.

When I first started working with Ruby and Cocoa, the RubyCocoa project was pretty elementary, but many leaps and bounds have been made. Thanks go out to bleything and and the RubyCocoa team without whose wonderful work this would not be possible.

Making an interface from scratch with the Arduino at its core

Equipment needed:

  • arduino (diecimila or equivalent)
  • wire + wire cutters
  • breadboard
  • selection of resistors (10k preferable)
  • switches (any sort, I used momentary)
  • computer – mac, linux or pc (mac preferable)
  • box of some sort – optional
  • soldering iron – only necessary for finishing the project

Inside the FootStompBox

# Wire a switch – one side to ground and the other to digital input 6. Use the breadboard.
# Put a resistor between the side of the switch connected to ground and +5v.
This is called a pullup resistor and makes the current default to ‘on’ or +5v when the switch is closed. When the switch is open, the voltage remains at 0v.

If you want additional switches, follow the two steps above making sure you use a new resistor each time.

Difficult bit: find a nice box and glue the whole lot into it and plug in a usb cable.

FootStompArduinoBox

Getting your computer prepared to talk to the Arduino

If you are comfortable with using the terminal, and want a one-step install for the various packages required for this tutorial, try:

MacPorts

To install: go to the MacPorts website and install using the installer there.
Check everything is working by firing up a terminal and typing

port version

which should return

Version: 1.710

Preparing Ruby

All the below are necessary before continuing

Installing (upgrading) Ruby
Ruby is already included in MacOSX, but for compatibility reasons, it’s best to upgrade. If you’re happy with your installation, just continue as is until you come up against any errors. Please note the Ruby included with 10.4 is very old, and possibly broken.
Fire up a terminal (if it isn’t already open from above) and type

sudo port upgrade ruby

Rubygems — required to use the below libraries
Fire up a terminal (if it isn’t already open from above) and type

sudo port install rb-rubygems

MIDIator — Ruby MIDI library
MIDIator allows us to use Ruby to easily create MIDI messages in a simple and comprehensible fashion
Fire up a terminal (if it isn’t already open from above) and type

sudo gem install midiator

Serialport — Ruby Serial Port library
This is installed by default as part of Ruby, but best to upgrade:

sudo gem upgrade ruby-serialport

Easy! Just sit back and watch incomprehensible lines of text scroll by.

Arduino code

The Arduino itself requires code to run. This tells the Arduino what to do with incoming and outgoing data. The language is called Wiring.

Make sure you have the Arduino programming environment installed (from http://arduino.cc)

Use the following code:
To explain, the first block initialises the pins we will use.
The setup() method initialises the pins and serial connection and tells us that they are sending data in to the Arduino.

The loop() method reads the pins every cycle, stores the data in variables and decides what to do with it. In this case, it sends a different value to the computer if the pin is up or down. The swi variable checks if it’s already been sent, and prevents doubling the ‘note on’ value. i.e. it can’t send a note off before it’s sent a note on and vice versa.

 /*
 sends serial data over usb to be converted by ruby to MIDI..
 */
int inPin1 = 6;   // choose the input pin (for a pushbutton)
 int inPin2 = 7;
 int val1 = 0;     // variable for reading the pin status
 int val2 = 0;
 boolean swi1 = false;
 boolean swi2 = false;
void setup() {
 Serial.begin(9600);
 pinMode(inPin1, INPUT);    // declare pushbutton as input
 }
void loop(){
 val1 = digitalRead(inPin1);  // read input val1ue
 val2 = digitalRead(inPin2);
if (val1 == HIGH && swi1 == false) {         // check if the input is HIGH (button released)
 swi1 = true;
 Serial.println(89, DEC);
 delay(100);
 }
if (val1 == LOW && swi1 == true){            // check if the input is low and it's just been high
 swi1 = false;
 Serial.println(90, DEC);
 delay(100);
 }
if (val2 == HIGH && swi2 == false) {         // check if the input is HIGH (button released)
 swi2 = true;
 Serial.println(91, DEC);
 delay(100);
 }
if (val2 == LOW && swi2 == true){            // check if the input is low and it's just been high
 swi2 = false;
 Serial.println(92, DEC);
 delay(100);
 }
}

Building a simple communication script in Ruby

Download the RubyMIDI.zip file and look in the lib folder at usbchecker.rb. This checks to see if the Arduino is present. In short, OSX creates a list of all the USB devices on your computer at /dev/. This script checks for one in the correct syntax for an Arduino, and passes it back to our main program.

Look in the main folder at BasicMIDI.rb. This contains all the information to communicate with the Arduino and change it to MIDI data.  The require functions at the top load in the libraries we installed earlier – rubygems, midiator, serialport and the file we’ve just looked at, usbchecker.rb. The first paragraph runs the usbchecker.rb and passes the identity of the Arduino back and initialises it (turns it on and starts communicating with it).

The next paragraph creates a MIDI device with Midiator and tells it to send all MIDI data to CoreMIDI on OSX, also known as the IAC Driver (Load up Audio MIDI setup and have a look).

The whole of the next section is a bit boring, and merely consists in grabbing data from the serial port and converting it into a meaningful format rather than the incomprehensible rubbish that comes across normally. sp.getc.chr grabs a character (getc) from the serial port (sp), and converts it into a real character (chr). The next loop puts it into a string.

Then, if it has something, it decides which one it’s received from the Arduino (see above that we’re sending different numbers according to what’s happening) and sends either a midi note on (midi.note_on(89, 1, 100)) or note off (midi.note_off(89, 1, 0)).

The other parts are failsafes for if it receives a line-end or newline character – the Arduino sends these after each transmission as part of the Serial.println() method.

Magic, that’s all the code explained

We can run this script by opening a terminal, typing ruby and dragging and dropping the BasicMIDI.rb file into the window, then hitting return. It will fail if there isn’t an Arduino present.

Integrating your Ruby script into a fully fledged native MacOSX application

First, go and download Xcode from http://developer.apple.com and install it. This will take a while.

You’ll notice there’s another folder called RubyCocoaUSBApp which contains lots of files and folders and a file ending .xcodeproj which has our Xcode project in it. Open this, and Xcode will open the project. You’ll notice the lib folder is still there, and that AppController.rb has the same code as BasicMIDI.rb above. It’s not necessary to know the structure of the rest of the folders, just that there’s lots of necessary stuff in there that it’s unwise to touch. Just modify the AppController.rb with your code and link it into the MainMenu.nib using Interface Builder – I won’t explain here because it’ll take ages, unless people need me to? Easy peasy.

Next step: Build and Run (Command-R).
This should launch our fully fledged application, which runs when we click on the button. It will display ‘Ready’ when connected and sending MIDI.

Simply connect any MIDI client to the built-in IAC Driver on OSX and away we go.

Oh, and before I forget, here’s the fully fledged application in case you’re having problems compiling using Xcode.

Tags: , ,

Humidity

Two days and our performance is in a building in the town square. I’m not altogether absolutely happy with the improvisation yet. I would like it to be a bit more impromptu, but it’s starting to move in the right direction. I have to learn how to use a new tool, come to terms with performance using it and create all the materials. In a week of working with it I think I’m starting to get the hang, but I need to think about the structuring of the materials in terms of shape. At the moment I’m controlling textures/atmospheres using the USB controller, but it would be nice to have something more manipulatable, like my coming controller/interface. A range of envelopes/gestures/morphologies would be nice, rather than variations on a slow fade. Also, there is very little I’m controlling in terms of timbre, at the moment it’s done my simply changing to a different sound and combining layers, a very flexible and fertile procedure, but limited in terms of the amount of time it takes to switch sounds. This could be done automatically, but it would be nice (and very possible) to do this in real-time.

I’m not sure we should ever move away from the sense of a computer being a servant, in fact it has been proven that it is very difficult for it to be any more. It is very effective as a servant, and has the possibility to extend our actions and interpret our behaviour, but not act reliably of its own volition. It is only as useful as the information we put into it.

Tags: , ,

Reflections

So. Just a few notes again from Sicily. Angela and I are working today towards our performance on Friday. The Mexicans arrived yesterday, and we’ve been exchanging a few dialogues. They are thinking of doing another festival with Cadmium next year in Mexico City, and would like to do a concert at Chop of alternative musics, as part of it. Personally I’d love to go to Mexico and perform, and I really like their style. It’s a kind of dancefloor mixup, made with traditional instruments – flutes, ocarinas, saxophones, etc.

I have finally got a good improvisation going with Live and some control routed through Max. Angela is happy with it too – there are some good sounds coming out, but I have the feeling that it sounds like ‘Live’. There’s a particular sound that the Session time stretcher has – both the granulator and the fft, which doesn’t make it ideal for live performance, but then Max is so convoluted that to do some of the most basic things that Live does would take months to program in Max, and even then it would be very unstable. I think a lot of people just end up rewriting some of the basic features of Live in Max without realising it, but then there is this element of direct control over sound that Max/MSP has over Live. If one could get all the interwoven signal processing features of Max, i.e. let every element be able to control another then it would be perfect. The problem is making a system which is not too difficult for the average user, yet powerful enough to satisfy the needs of the complex musician. There is a compromise between the two, that one can pass audio and MIDI between them, make plugins for DSP in Max and control what you will, but as always there are barriers to overcome.

One cannot always create all the sounds they would like to use in real time, a lot of preparation in terms of sound making must be done before the process starts. Preprocessed sounds are infinitely better to listen to that real-time processed ones, especially in the live instrument processing domain. As soon as you introduce a computer into the equation, the sound quality dramatically reduces.  Like João Pedro was saying, he has to create all his sounds in the studio before trying to do it live. He has not heard a solution for live processing that works, and I trust his opinion. However, in that domain one is comparing studio created work performed on stage to live work created on stage, and there is a fundamental difference there.

Evening:
It’s very loud and busy here at the moment. Kukultech (the Mexicans) are working on some material, practising and performing. Andres has discovered his camera, the one he uses for everything, is completely broken and is going to cost a lot of money to repair. It’s extremely hot and humid, difficult to work in circumstances like these, and my computer is complaining a lot – not overheating but struggling to keep cool.

I start to think that it would be easier to live somewhere that’s not England in order to do workshops, but then there are so many barriers in the way that it would make the difficulties in Britain seem trivial. But, saying that, Tullis is moving to Barcelona and leaving (almost) everything he has established in Britain behind.

Tags: , ,

Thoughts

So, the first of the entries from Sicily. We’re working on a new performance, Textures, and Angela is having difficulty working both on Ninnananna and this too. I am having problems getting SuperCollider to work with basic MIDI protocols, which seems crazy, but I’m going to start and see if there is a way of doing it with RubyCocoa instead. I think I’d be spending less time doing really basic things instead of wasting my time on such a convoluted and non-intuitive programming language.

Some thoughts: visual feedback (and tactile, I’ll get to that) seems such a basic thing that many of the programs for advanced musicians working with computers are incapable of supporting in a simple and stable way. SuperCollider has crashed a number of times when I’ve been here, doing simple things, it is buggy and prone to disaster. The problem is that ixiQuarks would be a great tool to use live with MIDI control, if only it worked. Live seems to be much better (more stable), but is too fixed around the quantised loop paradigm. If I could control external AUs from within, and liberate the loops from the ‘warp’ function, it would be perfect, especially by routing all the audio through SoundFlower, from ixi, Max etc. With Max/MSP again, there are problems with control, even though it is quicker to program than SC.

The solution at the moment is:

Route all audio through SoundFlower – Jack destabilises programs and cannot remember setups.
Send ixiQuarks audio through MaxMSP, in order to control levels etc using MIDI.
Use loops and sequencing in Live in order to maintain a structure in performance.

The solution in the future is:

Have a standalone control program – Max, SC, RubyCocoa – that sends data (MIDI/OSC) to the others.
Have multiple sound producing programs – Max, ixiQuarks, SC, MetaSynth – that are controlled by the above program and routed through soundflower.

That way we can have an external, controllable fx rack and keep everything as independent as possible, and finally integrate external interfaces such as the Arduino.

* OSC and Ruby – already sorted
* MIDI and Ruby
* Audio and Ruby

Tags: , ,