Archive | Robots – music and videos

RSS feed for this section

Experiment in Augmentation 1

Spring 2017

In the work, a human performer, Cyther (a human-playable robotic zither) and modular percussion robots interact with each other. The interaction between these performers is enabled by both the physical design of Cyther and software written by the composer. The perceptual aspects of the system distinguish auditory events, create groupings and find patterns. In response to perceived information, the system can mimic, transform and generate material. It stores information about past events, and thus has memory, which shape the expressive choices that it makes. It is used in improvisatory contexts to illuminate unique gestures that are only possible through electromechanical actuation, which inspire a human performer to explore new expressive territory. The improvisations provide structure and freedom in order to both present the possibilities of this ensemble and allow for spontaneity. In particular, the work explores rhythms and timbres that are enabled by these machines.


We often think of an instrument and the agent that plays it as unified. That is, we talk about a flautist or a violinist as a single thing that requires both human and instrument working symbiotically together. In other ways, performer and instrument are meaningfully distinct, and the boundary between the two is inflexible. What if this boundary is made porous, allowing the human to play the role of pseudo-static sound shaper while the instrument becomes dynamic and expressive? By integrating robotic actuation into a human-playable instrument, agency becomes amorphous and distributed as performer and machine interact through a shared medium. A human performer and the machine are able to fluidly move between the roles of impulse and filter. The robot inspires the performer with expressions made possible by mechanical actuation while the performer transforms these gestures by physically manipulating the instrument. Reciprocally, the performer can affect how the robotic system both interprets and generates ideas. The results illuminate the expressive spaces that are human, that are mechanical, that are shared between the two, and that emerge as these worlds synthesize. The actions of both become parts of a symbiotic whole, rather than self-contained instances that are co-located, thus the system exemplifies cooperative interaction. The project builds on the lineage of technology that seeks augmentation through human-machine symbiosis. The possibilities offered by such human-playable robotic musical instruments have been little explored (the vast majority of musical robots function autonomously).



for flute, clarinet, piano, violin, cello, tom-tom, PAM (robotic string instrument), and robotic percussion

Fall 2014 – Spring 2015

commissioned by the Juventas New Music Ensemble

Machine expressivity is often thought of as involving precision, speed, rhythmic complexity, non-idiomatic (for human performers) pitch patterns and replication.  Human expressivity is often thought of involving groove, phrasing, affect, contour, variation, articulation, entrainment and communication.  While these attributes help shape our conceptions of what is human versus what is mechanical, they are not confined to one category or the other: humans can be precise and robots can groove.  Expressive identity is more analog than digital.  This does not preclude expressive spaces that are unique to humans and machines, rather, it suggests the areas between them are ambiguous and that the attributes that define them do not do so in a one-to-one fashion (instead, attribute-space relationships are a function of combination and context).  The music explores these areas of ambiguity and clarity.  Genre is treated in a similar way such that stylistic exemplars are presented authentically and in transformation.  The intersections in expressive identity and style illuminate what is exclusive and what is shared.

Rise of a City


for guitar and robotic ensemble (PAM, MADI and CADI)

produced and recorded by Scott Barton, mixed by Marc Urselli and Scott Barton at East Side Sound Studios, NYC

Rise of a City introduces a human performer to the robotic creations of EMMI (Expressive Machines Musical Instruments, for the first time. The piece features complementary string parts (one played by a human guitarist, one played by the robotic string instrument PAM) that are supported by a robotic percussion ensemble. The piece explores mechanical gestures, human expression, virtuosity and synchronicity by placing specific musical ideas in a variety of instrumental and temporal spaces. Material is soloed and shared between the parts, giving us a sense of the unique expressive characteristics of human versus robotic instrumentalists. From the perspective of narrative, the musical interactions between human and machine can be understood through the metaphor of how ideas develop. Sometimes multiple groups of people simultaneously cultivate ideas towards similar goals even though they don’t live in the same place. Sometimes the paths of this race are parallel. Sometimes they diverge. When they diverge, the separation can result in either an alternate route to the original goal or a new path(s) that clears the way to previously unimagined possibilities. This has become a familiar phenomenon to us via technological innovation, scientific discovery, stylistic innovation and the construction of physical communities (dwellings → cities). The construction of physical communities has particular metaphoric weight in the case of this piece. From small beginnings a city exudes reiterative processes in multiple directions. New neighborhoods spring up that incorporate and / or react to adjacent areas. The restatements are accumulative, so that the entirety of the city becomes perpetually more massive and complex. At the same time, the most recent individual additions, buildings in the case of a city, mirror the qualities of the whole in terms of grandeur and intricacy. This path is not purely linear of course, and the ability to start simply, small-ly, or differently is always preserved.

Human-Robot Improvisation

I have been developing software that allows humans to improvise with the robots built by EMMI and the Music, Perception and Robotics Lab at WPI.  The bots have interacted with some wonderful performers:

Performance at Clark 2o-21 with Matt Jaskot, Peter Sulski and CADI / modular percussion instruments, Oct 30, 2013

Performance excerpt 2 at the Urban Canyon with Chris Fisher-Lochhead and Jenna Lyle, July 2013

Performance excerpt 1 at the Urban Canyon with Chris Fisher-Lochhead and Jenna Lyle, July 2013

Rehearsal 3 with Chris Fisher-Lochhead, Matt Orenstein and Alex Temple at HCL Chicago, July 2013

Rehearsal 2 with Chris Fisher-Lochhead, Matt Orenstein and Alex Temple at HCL Chicago, July 2013

Rehearsal 1 with Ammie Brod and Matt Orenstein at HCL Chicago, July 2013

Drum Circle

composed with Steven Kemper, Winter-Spring 2010

for the robotic instruments MADI, CADI and assorted found percussion instruments

Drum Circle features the robotic instruments MADI and CADI playing a diverse percussion ensemble that includes beer bottles, woodblocks, metal bowls and traditional drums in the woods of Virginia. In some sense, the colocation of machine and nature strikes us as a juxtaposition of things that cannot coexist. Indeed, machine / nature interactions often result in dramatic transformations where nature is displaced to make way for some unlike object(s) of human will. This is not such a story: here, the robots tuck peacefully into the landscape. This contextualization allows us to see and hear robots not as imperialist amalgams of electromagnets and plastic, but rather as agents that are governed by the kinetic and acoustic characteristics of our physical world that can cooperatively interact and coexist with surrounding objects. The lines between nature and machine are made fuzzy. Compositionally, the piece integrates unpredictable physical systems, machine listening and algorithmic responses. Over the course of the work, musical ideas are stated, absorbed, re-interpreted and stated again to create a cyclic yet developing story.

A video of the work is featured on the eco sono DVD Agents Against Agency