Publications

Musical Machines: History, Theory, and Techniques for Makers and Musicians

Barton, S. (2026). Musical machines: History, theory, and techniques for makers and musicians. Oxford University Press.

Musical instruments that “play themselves” have a storied history reaching back thousands of years. These machines have transformed in the modern era, synthesizing acoustic, kinesthetic, and computational worlds, expanding creative possibilities for artists and musicians. Musical Machines’ purpose is to inspire the creativity of musicians, engineers, and makers by helping them imagine and realize such machines. It begins by asking why we build musical machines and what they mean as expressions of individual and cultural values. The book then details key concepts and a historical landscape, establishing an artistic and technical context to help explain what these machines are and how they have been used. Technical skills are built through a series of projects devoted to mechanisms, actuation, electrical circuits, programming, and communication, located on the Companion Website. A template for the design and realization process provides a basis for examples in the major instrument categories—strings, percussion, and aerophones—that are further enhanced by first-person perspectives from prominent builders in the field. Remembering that the point of these efforts is to make music, the final chapter describes concepts and techniques involved when composing with and for musical machines.

Context-specific Knowledge Is the “Key” to Salsa Music

Getz, L., Barton, S., & Perry, L. (2021). Auditory Perception & Cognition. https://doi.org/10.1080/25742442.2021.1964341

Introduction: Previous research has shown ways in which both formal training and informal exposure affect perceptual experience and the development of musical abilities. Here we asked what types of training and exposure are necessary to acquire the context-specific knowledge associated with expertise. We specifically focused on the perception of salsa music: a genre that is rich in rhythmic complexity, but has received relatively little attention in experimental settings.

Methods: We examined specific groups within the exposure and training populations: those with musical training in the production of salsa rhythms (Study 1) and “native listeners” who grew up listening to salsa music without formal training (Study 2). Using two clave patterns (3–2 and 2–3 son clave) and three constructed alternatives, we asked participants to choose the correct clave pattern for a variety of music excerpts.

Results: We found that informal listening exposure was not enough to detect the salsa–clave pairings. Instead, proficiency was only developed when training and exposure were both domain-specific.

Discussion: Our results show the importance of deliberate training and the degree to which expertise comes to fruition through context-specific focus, thus helping to illuminate the complex relationship between the local and the universal in musical-cultural experience.

Parthenope: A Robotic Musical Siren

Sidler, M., Bisson, M., Grotz, J., & Barton, S. (2020). Proceedings of the International Conference on New Interfaces for Musical Expression, 297–300.

Parthenope is a robotic musical siren developed to produce unique timbres and sonic gestures. Parthenope uses perforated spinning disks through which air is directed to produce sound. Computer-control of disk speed and air flow in conjunction with a variety of nozzles allow pitches to be precisely produced at different volumes. The instrument is controlled via Open Sound Control (OSC) messages sent over an ethernet connection and can interface with common DAWs and physical controllers. Parthenope is capable of microtonal tuning, portamenti, rapid and precise articulation (and thus complex rhythms), and distinct timbres that result from its aerophonic character. It occupies a unique place among robotic musical instruments.

Circularity in Rhythmic Representation and Composition

Barton, S. (2020). Proceedings of the International Conference on New Interfaces for Musical Expression, 505–508.

Cycle is a software tool for musical composition and improvisation that represents events along a circular timeline. In doing so, it breaks from the linear representational conventions of European Art music and modern Digital Audio Workstations. A user specifies time points on different layers, each of which corresponds to a particular sound. The layers are superimposed on a single circle, which allows a unique visual perspective on the relationships between musical voices given their geometric locations. Positions in-between quantizations are possible, which encourages experimentation with expressive timing and machine rhythms. User-selected transformations affect groups of notes, layers, and the pattern as a whole. Past and future states are also represented, synthesizing linear and cyclical notions of time. This paper will contemplate philosophical questions raised by circular rhythmic notation and will reflect on the ways in which the representational novelties and editing functions of Cycle have inspired creativity in musical composition.

Mechatronic Expression: Reconsidering Expressivity in Music for Robotic Instruments

Kemper, S., & Barton, S. (2018). Proceedings of the 18th International Conference on New Interfaces for Musical Expression.

Robotic instrument designers tend to focus on the number of sound control parameters and their resolution when trying to develop expressivity in their instruments. These parameters afford greater sonic nuance related to elements of music that are traditionally associated with expressive human performances including articulation, timbre, dynamics, and phrasing. Equating the capacity for sonic nuance and musical expression stems from the “transitive” perspective that musical expression is an act of emotional communication from performer to listener. However, this perspective is problematic in the case of robotic instruments since we do not typically consider machines to be capable of expressing emotion. Contemporary theories of musical expression focus on an “intransitive” perspective, where musical meaning is generated as an embodied experience. Understanding expressivity from this perspective allows listeners to interpret performances by robotic instruments as possessing their own expressive meaning, even though the performer is a machine. It also enables musicians working with robotic instruments to develop their own unique vocabulary of expressive gestures unique to mechanical instruments. This paper explores these issues of musical expression, introducing the concept of mechatronic expression as a compositional and design strategy that highlights the musical and performative capabilities unique to robotic instruments.

A Robotic Percussive Aerophone

Sundberg, K., Barton, S., Walter, A., Sane, T., Baker, L., & O’Brien, A. (2018). Proceedings of the 18th International Conference on New Interfaces for Musical Expression.

Percussive aerophones are configurable, modular, scalable, and can be constructed from readily-available materials. They can produce rich timbres, a wide range of pitches and complex polyphony. Their use by humans, famously by the Blue Man Group, inspired us to build an electromechanically-actuated version of the instrument in order to explore the expressive possibilities enabled by machines. The Music, Perception, and Robotics Lab at WPI has iteratively designed, built and composed for a robotic percussive aerophone since 2015, which has both taught lessons in actuation and revealed promising musical capabilities of the instrument.

Using Recurrent Neural Networks to Judge Fitness in Musical Genetic Algorithms

Mitrano, P., Lockman, A., Honicker, J., & Barton, S. (2017). Proceedings of the 5th International Workshop on Musical Metacreation (MUME), at the 8th International Conference on Computational Creativity (ICCC).

We used a recurrent neural network as a fitness function for a genetic algorithm to generate monophonic solos. The genetic algorithm is based on GenJam as described in Biles (1994). We conducted training sessions with human participants in order to compare and quantify some of the differences between human-feedback and RNN fitness functions. We found that the RNNs can effectively play the role of human fitness feedback, but still suffer in many areas. Our results suggest that certain types of recurrent neural networks can address the issues with human feedback, and thus should be explored in future research.

Cyther: A Human-Playable, Self-Tuning Robotic Zither

Barton, S., Prihar, E., & Carvalho, P. (2017). Proceedings of the 17th International Conference on New Interfaces for Musical Expression.

Human-robot musical interaction typically consists of independent, physically-separated agents. We developed Cyther — a human-playable, self-tuning robotic zither — to allow a human and a robot to interact cooperatively through the same physical medium to generate music. The resultant co-dependence creates new responsibilities, roles, and expressive possibilities for human musicians. We describe some of these possibilities in the context of both technical features and artistic implementations of the system.

Systematic Variation in Rhythm Production as Tempo Changes

Barton, S., Getz, L., & Kubovy, M. (2017). Music Perception: An Interdisciplinary Journal, 34(3), 303–312.

We investigated the effect of tempo on the production of the syncopated 3-2 son clave rhythm. We recorded eleven experienced percussionists performing the clave pattern at tempi ranging from 70 bpm to 210 bpm. As tempo increased, percussionists shortened the longest intervals and lengthened the shortest interval towards an intermediate interval that is located in the first and second positions in the pattern. This intermediate interval was stable across tempi. Contrary to prior studies, we found that the complexity of interval ratios had little effect on production accuracy or stability and the “short” interval in the pattern was not particularly stable. These results suggest that as tempo is varied, (1) experienced musicians systematically distort rhythmic intervals, (2) rhythmic configuration, and not just the complexity of interval ratios, affects the production of rhythmic intervals, and (3) the distinction between long and short intervals is context-dependent.

Creativity in the Generation of Machine Rhythms

Barton, S. (2016). Proceedings of the 1st Conference on Computer Simulation of Musical Creativity.

This paper explores musical, psychological and philosophical ideas about how humans and machines interact in creative processes. It argues that creativity is a function of both generator and receiver, and that these roles can be amorphous in the creation and experience of electronic music. It offers an approach to structuring temporal spaces for rhythmic composition, which leads to the idea of machine rhythms, which are proposed as a promising area for creative expression.

MARIE: Monochord-Aerophone Robotic Instrument Ensemble

Rogers, T., Kemper, S., & Barton, S. (2015). Proceedings of the 15th International Conference on New Interfaces for Musical Expression.

The Modular Electro-Acoustic Robotic Instrument System (MEARIS) represents a new type of hybrid electroacoustic-electromechanical instrument model. Monochord-Aerophone Robotic Instrument Ensemble (MARIE), the first realization of a MEARIS, is a set of interconnected monochord and cylindrical aerophone robotic musical instruments created by Expressive Machines Musical Instruments (EMMI). MARIE comprises one or more matched pairs of Automatic Monochord Instruments (AMI) and Cylindrical Aerophone Robotic Instruments (CARI). Each AMI and CARI is a self-contained, independently operable robotic instrument with an acoustic element, a control system that enables automated manipulation of this element, and an audio system that includes input and output transducers coupled to the acoustic element. Each AMI-CARI pair can also operate as an interconnected hybrid instrument, allowing for effects that have heretofore been the domain of physical modeling technologies, such as a “plucked air column” or “blown string.” Since its creation, MARIE has toured widely, performed with dozens of human instrumentalists, and has been utilized by nine composers in the realization of more than twenty new musical works.

Extended Abstract for Drum Circle

Barton, S., & Kemper, S. (2015). UTS ePress. (Presented at the International Conference on Social Robotics, 2014.)

Dysrhythmia of Timed Movements in Parkinson’s Disease and Freezing of Gait

Tolleson, C. M., Dobolyi, D. G., Roman, O. C., Kanoff, K., Barton, S., Wylie, S. A., et al. (2015). Brain Research, 1624, 222–231.

A well-established motor timing paradigm, the Synchronization-Continuation Task (SCT), quantifies how accurately participants can time finger tapping to a rhythmic auditory beat (synchronization phase) then maintain this rhythm after the external auditory cue is extinguished, where performance depends on an internal representation of the beat (continuation phase). In this study, we investigated the hypothesis that Parkinson’s disease (PD) patients with clinical symptoms of freezing of gait (FOG) exhibit exaggerated motor timing deficits. We predicted that dysrhythmia is exacerbated when finger tapping is stopped temporarily and then reinitiated under the guidance of an internal representation of the beat. Healthy controls and PD patients with and without FOG performed the SCT with and without the insertion of a 7-second cessation of motor tapping between synchronization and continuation phases. With no interruption between synchronization and continuation phases, PD patients, especially those with FOG, showed pronounced motor timing hastening at the slowest inter stimulus intervals during the continuation phase. The introduction of a gap prior to the continuation phase had a beneficial effect for healthy controls and PD patients without FOG, although patients with FOG continued to show pronounced and persistent motor timing hastening. Ratings of freezing of gait severity across the entire sample of PD tracked closely with the magnitude of hastening during the continuation phase. These results suggest that PD is accompanied by a unique dysrhythmia of measured movements, with FOG reflecting a particularly pronounced disruption to internal rhythmic timing.

The Specificity of Expertise: For Whom Is the Clave Pattern the “Key” to Salsa Music?

Getz, L., Barton, S., & Kubovy, M. (2014). Acta Psychologica, 152.

Each Latin salsa music style is associated with a characteristic clave pattern that constitutes an essential structure for performers. In this article we asked what types of expertise are needed to detect the correct salsa–clave pairing. Using two clave patterns (the 3–2 and 2–3 son clave) and three manipulated alternatives, we asked listeners to choose the correct clave pattern for a variety of bomba, calypso, mambo and merengue excerpts. The results of Studies 1 and 2 show that listeners unfamiliar with salsa were unable to detect the correct salsa–clave pairing. Listeners who had some music training or were familiar with salsa detected the need for syncopation but not the specific pairing. Only musicians well-acquainted with salsa correctly detected the salsa–clave pairing. Studies 3 and 4 showed that incorrect choices were not due to an inability to distinguish between the alternatives: both adults and five-year-olds could easily tell apart the various patterns we used. We conclude that the distinction between the 2–3 and 3–2 claves is not inherent in the music itself, but rather is a convention to be learned through exposure and training. We discuss the results using an analogy to language learning.

The Human, the Mechanical, and the Spaces in between: Explorations in Human-Robotic Musical Improvisation

Barton, S. (2013). Proceedings of the Ninth Artificial Intelligence and Interactive Digital Entertainment International Conference (AIIDE).

HARMI (Human and Robotic Musical Improvisation) is a software and hardware system that enables musical robots to improvise with human performers. The goal of the system is not to replicate human musicians, but rather to explore the novel kinds of musical expression that machines can produce. At the same time, the system seeks to create spaces where humans and robots can communicate with each other in a common language. To help achieve the former, ideas from contemporary compositional practice and music theory were used to shape the system’s expressive capabilities. In regard to the latter, research from the field of cognitive psychology was incorporated to enable communication, interaction, and understanding between human and robotic performers. The system was partly developed in conjunction with a residency at High Concept Laboratories in Chicago, IL, where a group of human improvisers performed with the robotic instruments. The system represents an approach to the question of how humans and robots can interact and improvise in musical contexts. This approach purports to highlight the unique expressive spaces of humans, the unique expressive spaces of machines, and the shared spaces between the two.

Understanding Musical (dis)continuity

Barton, S. (2012). [Doctoral dissertation]. ProQuest Dissertations Publishing.

While our experience of musical (dis)continuity is often powerful and clear, articulating the relations that inspire such percepts is not always easy. Part of the reason for this is that our experience of musical (dis)continuity is influenced by a number of physical, cognitive and perceptual factors, and thus is complex. I will therefore explore (dis)continuity’s character by applying ideas from psychology, music theory, algorithmic information theory, and statistics to a variety of musical compositions, including my own. These explorations will describe (dis)continuity as primarily dependent on five key elements: how we bring entities into relations, holistic associations, perceived (dis)similarity of intra- and inter-entity attributes and relational structures (which we can describe in terms of type and degree); hierarchical organization; and context. These various elements work in isolation and in combination depending on the particular musical situation. Because (dis)continuity is multi-faceted in this way, no singular approach will illuminate the full extent of its richness and complexity. Instead, we must be able to approach (dis)continuity from a number of different perspectives; the one that we choose depends on the specific musical scenario. I will therefore incorporate ideas from the aforementioned disciplines, including structure mapping, transformational distance, and parametric dimensions, to describe, represent and eventually quantify the nature of these characteristics and how they interact to produce (dis)continuity percepts. Such a varied set of tools, which can be used in isolation or in combination, will allow us to describe (dis)continuity in a wide variety of musical styles. This may prove useful not only in identifying commonalities between stylistically diverse musics but also in providing an analytic approach to musics that are resistant to traditional tools. Such flexible, yet rigorous, approach will allow us to illuminate the nature of (dis)continuous relations so that we can analyze and compose (dis)continuous music more thoughtfully.