Gesture-Sensing for Musical Performance: A Percussive Interface
To create the next-generation Radio Drum. Note that ultimately it might
not be a "Radio" Drum. The technology might end up being different. It
must be a 3-dimensional controller that "doubles" as a drum. That is, it
senses the space above a surface, but it also can be played like a drum,
and identify "whacks" and different kinds of gestures, bounces, etc. It
is a controller, and does not generate sound.
The fist-generation Radio Drum was designed by Bob Boie and built at Bell
Labs in the mid-1980's as a "three-dimensional mouse." The Radio Drum
uses capactive sensing; you use a radio frequency to measure capacitance.
The two sticks are differentiated by using different frequencies for each
The Radio Drum itself could be called a "gesture sensor" that keeps
constant track of the 3-dimensional spatial location of mallets in the
following way: A small amount of wire is wrapped around the end of each
of two sticks and acts as a conducting surface electrically driven by a
radio frequency voltage source. The drum surface beneath the sticks
receives the signal, and the x and y positions of the sticks are derived
by determining the first moment of capacitance between the sticks and the
surface below. The z position (height) is given by the reciprocal of the
first moment. The greatest accuracy in z is in the region from about 5 cm
above the surface and closer. The response time is between 5 and 10
msecs, but this latency can be lowered to below 5 msec. (For technical
details see [Boie, et al 1989]).
WHAT WE HAVE
We have several copies of the original Boie drum (in fact we have all of
them still in existence), and we have a design for a "wireless" version in
progress. The new prototype wireless sticks were made last year by some
UVic engineering grad students, and I think we can finish at least a basic
prototype of the wireless drum by the end of 1999. Then we can test it
and see if it is good enough. If not, we might use another technology, or
possibly use a hybrid -- for example, one sensor method for "whack" and
one for "continuous" modes.
Temporal resolution: 1 millisecond latency
Spatial resolution: 1 millimeter resolution in x,y,z
1. Drum must be linear in x and y as much as possible. Or if not linear,
at least well-behaved (i.e. not random) so that software can compensate
for the nonlinearities.
2. Greater range of z if possible (more space above surface usable, up to
a meter or more would be great).
3. Need to have edge effects under control (or have a boundary so you
don't run into them).
4. There should be little or no "interference" between sticks (Max's drum
definitely has this problem -- when the sticks are close together, they
get less dependable).
5. ELIMINATE "DUFFING," that is, when a hard stroke or "accent" is
reported as a soft one. Boie thinks this might be an actual physical
phenomenon in which the material itself is distorting and sending
electrical signals. I'm 99% sure this is a hardware problem. It is one
of the things that makes the current drum frustrating to play.
1. Gesture detection (figuring out when you hit the drum) can be done by
detecting a change of direction of the stick, and can be much more elegant
than it is now. This is more like a real drum than the "threshold
crossing" that Max's drum uses. But velocity might be better measured
directly (the actual speed of the stick) rather than by reporting dz at
the moment of change of direction, as it is done now. This would free us
from constraints about what kind of foam we're using, which would make it
more versatile. Velocity could be reported even if you're using a hard
rubber surface that doesn't compress at all. Note that this can all be
done on the host, even using MSP or other DSP tools to analyze the data
stream, so again, we don't need to decide in advance. In fact, algorithms
for gesture-detection can be an on-going process.
2. Should be able to put different kinds of stuff on the surface of the
drum and then recalibrate the drum to work with that surface. We could
provide people with some options, but they could also use anything they
came up with too.
3. Would be nice to be able to see how the drum is configured right on
its own surface; since this is a playing surface, it can't be an LCD,
would probably have to use some kind of projection system.
CNMAT's work on gesture sensing and interfacing (i.e. ethernet and/or XLR
connections?). Is there any instrument made today that is ethernet yet?
MIT Media Lab -- Neil Gershenfeld (now working w/ the Flying Karamozov
Brothers on a new show, using accelerometers on the clubs). He's got all
kinds of tricks; if we find that the capacitance method isn't good enough,
we should meet w/ him to see a panorama of possibilities.
Tactex Controls -- "Smart fabric" technology for drum surface? (If using
a hybrid system). This would be good for hands or sticks.
www.tactex.com. This company is in Victoria and is very friendly.
Interface 1993: Intelligent Musical Instruments: The Future of Musical
Performance or the Demise of the Performer
CMJ 1994: The Computer-Extended Ensemble
ICMC1990/Glasgow proceedings: Recent Advances in the Coupling of the
Language Max with the Mathews/Boie Radio Drum
UCSD Symposium/1999: Improvising Across Borders in Cuba
ICMC 1994/Aarhus: A Virtual Piano Concerto -- The coupling of the
Mathews/Boie Radio Drum and Yamaha Disklavier Grand Piano in "The Seven
Wonders of the Ancient World."
Virtual Percussion: Drumming in the Ether. Invited lecture/concert with
Mark Goldstein, IBM Research Headquarters, Yorktown Heights, New York,
June 23, 1997.
Letter to Tatsuya Kitsuda, Yamaha Corporation, about building a new drum.
Pisa, Italy: Man-Machine Interaction in a Jazz Improvisation Context,
presented at First International Workshop on Man-Machine Interaction in
ICMC 1989/Ohio State proceedings: The Radio Drum as a Synthesizer
Controller. By Max Mathews and W. Andrew Schloss.
Strumenti Musicali (Italian music magazine, Torino Italy): Andy Schloss
e il Radio Drum (interview conducted by Giovanni Ramello).
VIDEOS THAT SHOW THE RADIO DRUM IN CONCERT:
Western Front: Suite from the Seven Wonders
ONADIME demo with live images generated from the drum along with the Suite
MS/NBC news show with live piano demo
Wildlife performance from Stanford's Centennial concert with DAJ
UCSD video of live performance of first four movements of Seven Wonders
Middle East/ MIT concert in Boston, May 1999 (not great audio quality)
written by Dr. A. Schloss, School of Music, UVic, email@example.com