For this musical instrument controller project using MIDI, I decided to experiment with flex sensors and DIY my own “Nintendo Power Glove“ to send MIDI commands and playback notes on software synthesizer on my computer.
My hardware choices are 5 Spectra Symbol flex sensors, one for each finger control, a joystick module with x axis controlling pitch bend, and MKR Zero, as it can be plugged into computer without the need of MIDI jack to show up as USB MIDI device.
User flow:
User put on the glove on the right hand, bend any fingers to trigger/release notes and use the other hand to control joystick to bend pitch.
Process:
In week 2, I connected my MKRZero to computer using Audio MIDI setup on mac OS. Using MIDIUSB example code, I sent melody as MIDI command to make sound in SimpleSynth.
After I picked my sensors for the controller, I tested out the range and sensitivity of my flex sensors and joystick. Then, following this MIDI instrument guide, I treated the analog input of flex sensor as a digital input 1 and 0 since they only control two MIDI commands: note on and note off. I split the range of analog readings into two ends from the reading where the sensor is bent at right angle (~140 threshold). Therefore, 0-140 is 1 (bent) and 140-300 is 0(straight), and I used state change detection to send note on and note off command. Locating MIDI note values from MIDIUSB library’s pitchToNote, I had five sensors correspond to notes C, D, E, F, G on a C major scale. Following this MIDIUSB joystick example, I added pitch bend command to my code.
I was able to test my code in SimpleSynth:
Then I created a max patch receiving MIDI note messages from MKR Zero. Instead of using general MIDI channel instruments like I did in SimpleSynth, I decided to make my own sampler sampling any audio files of my choice as well as the option of sampling my own voice on the spot. By changing the playback speed of the samples, I was able to receive note messages from MKR Zero and playback different notes in Max:
Max patch for MIDI sampler
Code:
Github repo for all the resources used in the project: https://github.com/ada10086/Tangible-Interaction/tree/master/W2-3%20Musical%20Instrument%20Controller
Final Glove MIDI Sampler code: https://github.com/ada10086/Tangible-Interaction/tree/master/W2-3%20Musical%20Instrument%20Controller/Glove_MIDIPlayer_w_pitchBend_MIDIUSB
Fabrication:
To make sure all the flex sensor connections are secure during finger movement, I solder the legs of flex sensors to thin wires and wrapped the connections with shrink tubes, then soldered the wires to header pins. I cut a few holes on the back of each finger on the glove so that the flex sensor can criss cross through them and stay fixed in place.
Soldered flex sensors
Glove MIDI Sampler prototype
Challenge:
I tried to avoid soldering the flex sensors in the first place, and tried both female header pins and female to male jumper cables on the flex sensors and secured the connections with shrink tube, however either way resulted in very loose contacts and unstable readings.
My joystick reading changes gradually from the middle point reading 760 to 0 in one direction. However, when I tried to move to the other direction, it always snaps straight from 760 to 1023, without steps in between, therefore my pitch bend sounds a little off.
I was able to receive the msb and lsb values in Max from pitch bend command sent from MKR Zero, but I haven’t figured out how to change the pitches with those values in Max.