Resilience in Color: Sonic Portrayals of Women’s Resistance is an audiovisual installation project that translates the visual iconography of women’s global sociopolitical movements into an immersive audiovisual experience. It leverages the concept of RGB color distance to recontextualize powerful images of women exhibiting extraordinary resilience and courage against police brutality, dictatorships, censorship, and human-rights violations, aiming to amplify these acts of defiance.

The process begins with these images of defiance. A target color, often emblematic of the protest itself or inspired by news titles and prominent visual themes, is pre-selected for each image. Custom software then scans each selected image pixel-by-pixel, from left to right and top to bottom. The Euclidean distance between each pixel’s RGB value and the target color is calculated; if this distance falls within a defined threshold, the pixel is considered a match. To provide ongoing visual feedback, a marker glides over the image, indicating the scanner’s live position. Simultaneously, a duplicative visual representation is generated beneath the original image. This is accomplished by selectively drawing colored points in a dedicated space which corresponds to pixels that meet the color criteria. Specifically, when the calculated distance between the current pixel’s color and the target color falls within the threshold.

The sonic core of the project is built upon formant synthesis, chosen to capture the spirit of vocal determination inherent in acts of resistance and to evoke the human voice, creating a soundscape that forms a chorus of resilience. The data derived from the scan, specifically the Euclidean distance of a pixel’s color to the target color, directly influences parameters such as the amplitude and formant change of specific voices within a multi-part vocal composition. To maintain the prominence of these voices and symbolize unyielding persistence, pixels that deviate significantly from the target color trigger a sequenced kick drum sound. This interplay between the presence of matching colors (represented by vocal sounds) and their perceived absence (represented by the rhythmic pulse) forms a narrative chorus. Furthermore, for certain images depicting isolated figures of resistance, such as “Women in Red” or “The Girl In The Blue Bra,” a delay effect is implemented, allowing the data from these distinct images to influence each other’s delay parameters, sonically weaving a web of solidarity.

Resilience in Color is adaptable in its presentation, designed to be experienced over varying durations that invite audiences to come and go freely, engaging as they wish. The adjustable nature of the scanning speed allows for creating temporal dynamics between the images, potentially condensing or expanding the experience from desired minutes to days.

It premiered as a 5-channel, 8-hour-long installation at ICAD (International Conference on Auditory Display) 2024 at the EMPAC (Experimental Media and Performing Arts Center) Studio 2, Rensselaer Polytechnic Institute, Troy, NY [6]. It was also installed as a stereo, 24-hour-long installation at SEAMUS (Society for Electro-Acoustic Music in the United States) 2025 at Yue-Kong Pao Hall, Acting Studio I, Purdue University, West Lafayette, IN. At ICMC (International Computer Music Conference) 2025, a 15-minute fixed media installation for the Bright Family Screening Room at Emerson College, Boston, MA, will be presented. In this shorter version, the scanning speed is adjusted, ranging from 10 milliseconds per pixel to 120, allowing for temporal dynamics that can condense or expand the experience. This variability creates interesting sonic textures, from subtle formant changes at slower speeds to faster transitions at higher speeds, offering a condensed yet immersive experience of the work’s essence.

This work features a curated selection of five images, each captures a moment of women actively resisting political oppression around the world. These images serve as the source material for the sonification. List of images used in the project from left to right:

• Women in Red (Turkey)
  Photo Credit: Osman Orsal/Reuters [2]
  The image, captured in Istanbul, Turkey during a moment of violence at the 2013 Gezi Park protests, shows a woman in a red dress protesting as a masked policeman discharges tear gas directly at her. Gezi Park Protests lasted between 28 May and 20 August 2013.
• The Girl In The Blue Bra (Egypt)
  Photo Credit: Reuters [5]
  The image, captured in Cairo’s Tahrir Square in Egypt during a moment of violence in 2011, shows a woman being dragged and beaten by the Egyptian military. On the cusp of a soldier’s kick, her starkly exposed torso is clad only in a bright-blue bra.
• Green Scarf Movement (Argentina)
  Photo Credit: Brandon Bell/Getty Images [4]
  The image, captured in Argentina during reproductive rights protests, shows a group of activists holding triangle- shaped green scarves above their heads as they stand in solidarity.
• Pink PussyHats (USA)
  Photo Credit: Carolyn Cole/Los Angeles Times/TNS [3]
  The image, captured in the USA during the 2018 Women’s March, shows activists wearing pink beanies, also known as Pink PussyHats. The protest focused on voter registration and electing women and progressive candidates.
• Yellow Scarves (India)
  Photo Credit: Danish Siddiqui/Reuters [1]
  The image, captured in India on International Women’s Day in 2021, shows women activists clad in bright yellow scarves, which are symbolic of mustard fields, joining mass sit-ins and hunger strikes on the outskirts of New Delhi. Their protest rallied against agricultural reforms that they feared would compromise their livelihoods.

ludus vocalis (2024), a 25-minute multimedia work for eight-channel audio and 4K video that explores and reimagines paralinguistic vocal sounds as musical objects—such as laughing, crying, screaming, gasping, and moaning. These sounds carry unique emotional and largely cross-cultural semiotic qualities, prompting me to explore how their acoustic and affective properties could be harnessed in a musical context. As such, ludus vocalis is divided into sixteen short vignettes or movements, each focusing on a specific type of nonverbal vocalization. The source material for ludus vocalis was drawn from a moderately sized collection of vocal samples, manually curated and gathered from various online platforms, including Epidemic Sound (https://www.epidemicsound.com/), Splice (https://splice.com/), and, to a lesser extent, FreeSound (https://freesound.org/). These samples—586 audio files in WAV format, with an approximate duration of 40 minutes—were organized into distinct paralinguistic categories, and smaller, more focused audio corpora were created. Each corpus employed different segmentation strategies and audio analyses to facilitate deeper exploration of the source material. The video component was developed using AI- generated video, primarily created via text-prompts with RunwayML (https://runwayml.com/), as raw material. Broadly speaking, all generated videos were intended to represent mouths enacting the different kinds of vocalizations that each movement was based on, and the “failure” of the generative model to adhere to certain prompts was often embraced.

The Right Not (To Be) features the voices of members of the Israeli anti-occupation activist organization Breaking the Silence, an anti-occupation Israeli activist organization whose work is focused on collecting and spreading testimonies of veterans from the Israel Defense Forces whose mandatory military service took place in the occupied territories, West Bank, and Gaza. In this work, I explore the complex relationship between belonging and historical narratives (or myths) that form a collective identity and highlight the difficulty in cultivating a political morality opposed to the status quo imposed by one’s state. The piece engages with questions about identity, the need to define good and bad, and the tendency to stop listening to others when their opinion or life experiences do not align perfectly with common and popular rhetoric and conduct. 

Sonically, my engagement with speech and oral sounds takes on multiple forms, ranging from the setting of full spoken sentences, through electronic manipulation that emphasizes, distorts, and amplifies the relation between vocalizing and wording, and the creation of rhythmic and timbral textures out of the artifacts of speech; sounds produced by the mouth that don’t amount to the production of legible language, but are nonetheless parts of speech’s meaning-making process. 

My conversation with Breaking the Silence began almost three years ago now, and I finally got to record some of their members in June of 2023. The conversation I had with them did not touch the military service directly, but the lived experience in Israel-Palestine, as it is perceived from Leftist Israeli eyes. 

Needless to say, the reality of the region has shifted between June 2023 and now, and as a result, the piece sounds in different tones as well. Although the piece does not address the ongoing war in Gaza, it would be reckless to assume its perception can be dissociated from the current context of the region. The war has captured the interest of the international community, and yet, there have been limited attempts to comment on or engage with it, as well as the lived realities of Jewish Israelis, Israeli Arabs, and Palestinians in the region through an artistic medium since October 7th, 2023. In this current saturated moment in time, The Right Not (To Be) offers invaluable insights into the possibility of peace and the lived experience in Israel-Palestine, as it is perceived from Leftist Israeli eyes. the members of Breaking the Silence with whom I talked and whose voices are heard in the piece share a commitment to a peaceful future and a bettered, equal society. I hear a rare sense of hope in their voices, which I was committed to doing justice to in my compositional process.

Given a database of over 1000 audio and video recordings of 93 cities from 43 countries worldwide – over 28 hours of video – agents communicate with one another and negotiate their way through the archive to arrive at a single clip. As with other “Coming Together” works, it is the negotiation that is of artistic interest: the movement from apparent chaos to unity. 

Five loudspeakers present the initially random audio recordings, granulated using a bespoke granular system, and heavily filtered through selected MFCC bands. Visually, a single screen displays five slices of the selected videos in greyscale, with separate hues highlighted, thereby analogous to the equalised audio recordings. Videos are drawn from parallel sources as the audio recordings. 

The work has three distinct phases: 

Part One: Convergence
The agents attempt to arrive at the same clips. They initially chose a random clip from the database, which is shuffled with each run. Each audio file has been analysed and segmented based on timbral novelty; a segment is initially chosen and is time-stretched and heavily filtered. Because audio is derived from the walks, the city sounds are stretched and amplified resulting in a noisy environment. Agents select prominent MFCC bands in the recordings, and “claim” these spectral regions – the number of bands an agent can claim depends upon how far along the clip negotiation has progressed. The frequency bands are initially quite wide (and thus noisy); as the agents converge, the bands narrow and the audio texture becomes more chordal. The partnered video agent similarly applies a selective colour filter based on hue to its video. The agent’s audio/video clip location is displayed, its vertical placement indicating its relative position in the database; during convergence, these graphics will begin to visually align horizontally. 

Part Two: Alignment
The agents attempt to align their starting points. To do so, they shorten their gestures. As the agents attempt to align their starting points, more silence & black screen results. The continuous beat is displayed, with the agent’s current beat selection displayed in red. Edge detection is added to the agent video as it fades, slowly filling the blackness 

Part Three: Extension
The agents have converged on their audio/video clips and their onsets; a frame from this portion of the video briefly covers the screen, only to fade away while being processed. The agents attempt to expand their segments again. Often, they get out of alignment during the process, as their lengths will differ, and play through the negotiated “downbeat”: this is not a bug, but an embraced feature. The cumulative audio of the agent’s multiple MFCC bands, slowly is replaced by a spectral freeze of this audio (mimicking the single video frame of the visuals) while the agent’s video deteriorates.

A Max4Live visualizer patch was used to generate audio peaks when being fed audio from a VCV Rack software modular rack session.

A Dialogue, In Medias Res presents a speech monologue that is algorithmically rearranged via a quicksort process—visually represented by short fragments (fractions of a second each) of video, first arranged randomly.

The conceptual foundation for this algorithmic approach draws direct inspiration from Brian Christian and Tom Griffiths’ book “Algorithms to Live By: The Computer Science of Human Decisions,”1 which explores how computational algorithms can illuminate human decision-making and provide insights into navigating complexity in daily life. The book’s examination of how sorting algorithms bring order to chaos particularly resonated as a powerful metaphor for human meaning-making processes and the passage of time—themes central to this installation. 

The technical realization of this work involved several intricate processes. The initial fragmentation was achieved by analyzing the complete, unbroken video in Max/MSP using transient detection to automatically identify syllable onsets within the speech, creating natural linguistic break points. These algorithmically determined fragments were then subjected to a custom quicksort algorithm programmed in Python, which systematically reorders the randomized segments toward their original sequence. 

The quicksort algorithm2, 3—a pivotal conceptual and technical element in this work—functions through a “divide and conquer” approach to sorting. Beginning with a set of disordered fragments, the algorithm selects a pivot element and partitions the remaining elements into two sub-arrays: those less than the pivot and those greater than it. This process recursively continues within each sub-array until the entire sequence is ordered. The algorithm’s efficiency lies in its average-case complexity of O(nlog n ), making it significantly faster than simpler sorting methods for large datasets. In the context of this installation, the quicksort visually manifests as a gradual emergence of coherence from chaos—a computational metaphor for the human search for meaning and order. 

During the disordered phase, video fragments play at four times their normal speed while maintaining the original speech pitch to avoid “chipmunk voice” distortion, creating a temporally compressed but vocally intelligible experience. This time-manipulation was carefully balanced through processing and combination in Adobe Premiere. The rapid juxtaposition of audio segments from different points in the timeline initially created unintended artifacts—clicks and pops from non-zero crossings—which were methodically addressed through audio restoration techniques in Adobe Audition to ensure sonic continuity despite the algorithmic disruption. 

As the algorithm progresses from disorder to order, the video undergoes a deliberate aging transformation. This effect was accomplished through a multi- stage process beginning with the extraction of a reference frame from the original A Dialogue, In Medias Res, 2025 ICMC ’25, June 8–14, 2025, Boston, U.S.A video footage. This frame was then manipulated in Adobe Photoshop using Adobe’s Firefly AI4 generation tools to create an artificially aged version serving as a keyframe reference. The aged keyframe was subsequently processed through EbSynth,5 which intelligently mapped these aging modifications onto the original video’s movement patterns, creating a fluid temporal transformation that unfolds in parallel with the algorithmic sorting. 

Concurrently, the voice undergoes its own aging process through subtle quality modifications programmed in Max/MSP, sonically mirroring the visual transformation with carefully calibrated audio processing that imparts the auditory impression of time passing. In a final twist, the voice morphs into instrumental sounds through Orchidea’s assisted orchestration process,6 which algorithmically generates a score that mirrors the emotional and structural transition of the piece. This work interrogates ideas of temporality, identity, and the transformation inherent in both human and computational processes, suggesting that order—even when achieved through random processes—can yield deeply human narratives.

Temporary Structures are short works for piano and computer graphics that explore the connections between musical and visual lines. The piece uses algorithmic methods to seamlessly integrate musical and visual art, aiming for both fluidity and depth. In these two works, digital sculptural forms shift shape along coordinates defined by pitch, timing, and dynamics drawn from the musical gestures. This process creates a constantly evolving visual structure that reflects the music and is inspired by the philosophy and smooth forms of parametric architecture. 

The premiere took place at the April in Santa Cruz Festival, where pianist Chia-Lin Yang performed alongside the accompanying computer graphics. The work has also been performed by pianist Eric Huebner at the Gassmann Electronic Music Series at UC Irvine, and by members of Dog Trio at the klub katarakt Festival for Experimental Music in Hamburg, Germany. 

Beyond live performances, the work has been presented in various formats: as an installation featuring video and a synchronized Disklavier at the FFEMF Festival at the Crisp-Ellert Art Museum in Saint Augustine; as a wall-mounted video artwork at the Zeitgeist Gallery in Nashville and at the Performing Media Festival in South Bend, Indiana; and as a screening at WAVEFORMS, presented by the Museum of Science, Boston.

This multimedia work explores neural activity through the integration of neuroscience and art. By processing calcium imaging data from the hippocampus CA1 and retrosplenial cortex (RSC) of a mouse brain, the project transforms complex brain functions into dynamic audio- visual experiences.

Utilizing TouchDesigner, neural data is visualized in real time, with calcium traces influencing the movement, size, and shape of visual elements. The soundscape is similarly shaped by Max/MSP, where the same neural data informs musical composition, manipulating filters and dynamics to create an immersive auditory experience. 

Additional musical layers are composed in Logic Pro, interacting continuously with the visuals to form a feedback loop that enhances the overall presentation. This work reveals the intricate relationships between brain activity, sound, and visual art, showcasing the potential of interdisciplinary collaboration in understanding and expressing the complexities of the brain.

Contained is an ambisonic/haptic virtual reality (VR) and immersive dome-based project that approaches anthropocentric spaces as haunted acoustic architectures, non-places permeated with the sonic spectres of unseen forces. Correspondingly, it seeks to invert the traditional, visually-biased audio-visual hierarchy via a positioning of spatial and haptic audio as the central elements within an immersive cinematic experience, emphasizing sound’s ability to not only permeate space and surround the auditor, but also its capacity to penetrate, saturate and vibrate a listening body, forming an intense relational bond between self and environment, whether material or virtual. 

This project evolves from and roots itself within an acoustemological approach: acoustemology is best described by Steven Feld as engaging “sound as a way of knowing” [2]. Contained explores how concepts rooted in the discipline might bear an impact not only on how immersive audiovisual experiences are created, but also on how they might enable a unique, profoundly embodied encounter. 

Thematically, this project presents an auscultation of our Anthropocentric milieu integrating field recordings, 360o camera footage and 3D scans of urban corporate towers, logistical networks, industrial areas and other non-places [1] as well as urban encampments and derelict locales that are resonant with both the heard and unheard acoustic emanations of the technotope we have become dependent upon for our survival. In doing so, it approaches sound as a material that can be apprehended as both corporeal and abstracted: in addition to the airborne, audible sound of the subject spaces, Contained integrates the electrical, vibrational and mnemonic emissions that permeate our everyday.

Foresta-Inclusive: (ex)tending towards is made up of two parts: 1) the ForestaInclusive sensing infrastructure installed in a forest and 2) the in-gallery connected immersive installation (ex)tending towards. The sensing infrastructure is comprised of three sculptural sensor pods that are battery powered and send live data to an Internet of Things (IoT) prototyping platform. The sculptural sensor pods can be installed in different forests unobtrusively onto the trunk of trees and sense phenomenon such as: soil and air temperature/humidity, particulate matter (.1 μm – 10 μm), light level, wind, volatile organic compounds (VOC), C02, and rain. The in-gallery installation uses the real time data and materializes it with visual, sonic and olfactory elements. Often the work is shown during the winter, and so in this case I use a past recording from a local forest protected by the rare Charitable Reserve in Blaire, ON. CA, in lieu of live data. (ex)tending towards is comprised of two projected visualizations, sculptural elements, a four-channel sound system, and a touchless scent interface.

In response to the temporal difference between tree and human individuals, this work explores ways to slow down human engagement, and to make visible the daily experience of a tree. The aim of the work is to find ways to demonstrate the absolute liveliness of the natural world as it unfolds all around us – yet more often than not – beyond our limited sensory perception. The first visualization materializes data as a particle flow field that gently undulates and is affected in real time by changing data.

Inspired by tree rings as evidence of yearly experience, the visualization is structured in the same manner and visualizes the last 24hrs of the tree’s life, where the outer ring shows contemporary values and each subsequent smaller ring the values from the previous hour. To interact with this visualization, there is a one-meter-tall cork cylinder that is also a scent sculpture, which releases the scent of geosmin (the scent of a forest after it rains) every time it rains in the forest. To interact, the participant uses a simple gestural interaction to move spatially into the visualization. The slower one moves, enables the participant to inspect each ring. The interface is embedded in soil, which also contain a set of sculptural sensor pods. Next to the visualization is a point cloud visualization of the tree at the rare Charitable Reserve. The point cloud was captured by a LIDAR scan of the forest at rare using a very large drone and rendered using Touch Designer. This point cloud is also affected in real time by live data. Like the visuals, the sonic elements materialize the forest data in a generative sound experience that balances between mimicry and poetic memory of forest experience. In its entirety this installation creates an embodied exploratory space where the deep time of a tree’s life is remembered, and the human body is slowed down in the engagement.

Liminal is a single-user, AI-assisted interactive audiovisual installation that merges gesture recognition, real-time sound control, and generative visual design grounded in classical Chinese aesthetics. The work draws upon the poetic motif of the “Peach Blossom Spring” as a metaphor for immersive escape and cultural reflection, reimagined here through contemporary computational media. At the heart of Liminal is a custom gesture-mapping system driven by com- puter vision. A high-frame-rate camera captures the participant’s hand and body movements in real time. The left hand triggers and modulates sounds derived from traditional Chinese percussion and xiao (bamboo flute), while the right hand controls timbral and articulatory variations of guqin-like textures, such as harmonics and arpeggios. These movements simultaneously influence a custom 3D particle system, generating visuals reminiscent of dynamic ink wash paintings. The result is an evolving visual environment that fuses digital abstraction with references to natural landscapes and lacquered ornamentation. Sound in Liminal is shaped through a real-time gestural mapping system. Using a high-frame-rate camera, the installation tracks the participant’s hand and body movements to control and modulate sonic elements derived from guqin, xiao, and percussive textures. Each gesture dynamically alters parameters such as pitch, articulation, layering, and spatialization—allowing the participant to sculpt a continuously evolving soundscape through motion alone. Designed for one-on-one interaction, the installation maintains focus and clarity by tracking a single participant at a time. This enables highly responsive audio-visual interplay and encourages an intimate, reflective mode of engagement. Each interaction becomes a unique and ephemeral composition, situated at the intersection of body, machine, and cultural resonance. Liminal invites participants into a space where gestures function as both input and authorship, transforming embodied presence into real-time audiovisual expression. It offers a sensory environment where memory, movement, and technology coalesce—evoking a digitally mediated encounter with introspection and transformation.

“24 Cards” is a computer music composition consisting of any number of participants for any duration. It is presented as a stack of interactive cards. Each card details a musical cell of the composition; on the reverse side, a QR code provides access to all other components of the piece. The cards represent individual elements of the composition as graphic depictions of software patches. Visitors are encouraged to take a card, and the set is replenished daily, ensuring the piece evolves with audience interaction. Whether explored individually or collaboratively, 24 Cards invites participants to engage with its modular structure and reimagine its possibilities.

This playful work seeks to escape the confines of a piece of music composed for and with a computer. This is partly done by conceiving patcher programming software as ‘score-instruments’, which are graphically reproducible computer programs for sound synthesis whose form contains sequential memory for execution, including chance-based operations. These musical cells can be distributed, recreated, repurposed, and reused from the cards on which they are presented. It is an emergent composition consisting of any number of participants for any duration. The piece’s form is an ecosystem of sound, with each cell contributing an element of sound design within it. The work was partly inspired by the motions of conferences, where attendees exchange cards and connect using their devices. What if this activity formed the mechanism of a composition?

Rings. . . Through Rings reimagines Hong Kong’s military cartographic heritage through an innovative sound art installation. Utilizing historical maps as the foundation, the project transforms archival geographical data into laser-etched vinyl discs, serving as both physical artifacts and sound-generating devices. This artistic reinterpretation emphasizes three significant sites: No. 55 Ha Pak Nai, Tung Chung, and Lau Tau at the Lantau Island Forts, symbolizing the intersection of natural landscapes and military infrastructure [2]. The installation features an integrated system composed of three key elements. At its core are custom laser-etched vinyl discs, crafted from PVC and embedded with encoded geographical data. When these discs are played on specially modified turntables, their physical grooves, shaped by both data and material properties, generate distinctive sonic textures. These turntable stations, three in total, are enhanced with real-time processing capabilities, inviting participants to actively manipulate and shape the sound output through direct interaction. Complementing this tactile experience is a TouchDesigner-based audiovisual system, which responds to turntable gestures by rendering dynamic visual projections and immersive spatialized soundscapes [1]. Together, these components form a responsive and exploratory environment that bridges data, sound, and performance. Participants physically engage with the turntables, altering rotation speeds and playback direction, which in turn influences real-time audio processing and projection mapping. This interactive feedback loop creates a multisensory experience, bridging past and present through tangible sonic exploration. The work demonstrates technology’s capacity to preserve and reinterpret cultural heritage through sonic immersion and interactive design. By mapping cartographic data onto sound, Rings. . . Through Rings not only explores historical memory but also invites reflections on the evolving relationship between landscapes and human intervention.

Summerland is an installation for an array of 24 antique telegraph sounders. These Morse code receivers, which mechanically produce tapping sounds, were the dominant form of long-distance communication for most of a century, requiring a skilled listener to translate the taps — sonic representations of dots and dashes – into language. These devices are arguably the first digital-to-analog converters, turning binary information into an audio stream of rhythmically modulated clicks. It is also, by its use of fluctuating voltages to create sound, the primogenitor of the audio speaker, the telephone, and most modern sound reproduction technology.

Invisible action at a distance – what we now call telematics – has always been a hallmark of the supernatural. Once the telegraph appeared, making instantaneous long-distance communication possible, it did not seem unreasonable to many that a ‘spiritual telegraph’, as it was often called, could use other invisible but verifiable forces to communicate with the world beyond [1]. From its beginning, information technology has always been entangled with myth and dreams of transcendence.

Summerland recuperates this most ancient of electric sound producers, using modern, computer-controlled forms of digital transmission to reanimate this prototypical apparatus, so it can speak once again. It uses speech and text from two major figures in 19th-century communications as source material which drives the sounders, each speaking in their own particular idiom. One of those voices is that of Samuel Morse himself, the inventor of the electro- magnetic telegraph. Excerpts from his writings and letters are translated into the dot-dash code that bears his name – a cipher once understood by many but now on the verge of extinction. The words and letters of the texts, while holding true to the rhythmic structures of Morse code, are often broken up, distributed among multiple sounders, and subjected to accelerations and deaccelerations based on a rule set of first- and second-order Markov chains. The other is that of the medium Kate Fox, whose experiences of ghostly ‘rappings’ in her house a few years after the invention of the telegraph led to the founding of Modern Spiritualism, and the craze for communicating with the dead that has lasted into the present day [2]. Rather than focusing on the glyphic character of written language as I did for Morse, I made recordings of transcripts of her séances, and inspired by the (more-or-less failed) early telephony experiments of J.P. Reis, which used rapid series of clicks in an attempt to recreate the human voice [3], I tried to do the same with my choir of sounders. I subjected these recordings to a number of different forms of spectral analysis, and developed several mapping strategies to somehow (at least subjectively) recreate something vaguely vocal. Given the mechanical limitations of the sounders, I knew this entire approach was somewhat absurd, but perhaps no more so than communication with the dead.

This conversation is created by a generative algorithm which uses complex decision-making structures to fashion a never-repeating dialog, in which layers of utterances, reproduced as taps and clicks, retain the character of language while remaining forever out of reach. In the context of contemporary magical thinking about media, Summerland looks back at the archaeology of communications, a séance which seizes from the ether the dead voices of Morse and Fox, materializing their words in streams of clicks through a medium — the sounder — no longer able to articulate. Thus the promise, and ultimate failure, of communication with the past is built into the very attempt to make the sounders speak. However, the psychic and electromagnetic forces we can summon can still, in the act of materialization, evoke the dimly-seen ghost, the unnerving rap of unseen knuckles on the medium’s table.

the ground beneath our feet, the air inside our lungs grew from a desire to extend  my work with infrasound into environments where the high sound pressure levels  needed to create simultaneous tactile and auditory sensation with loudspeakers  would be impractical. Since infrasonic energy propagates much more readily in  solids than in the air, designing a system that could present ultra-low frequency  compositions mechanically seemed to be a fruitful path. 

This led to the creation of the “haptic hammock,” a hammock chair with Dayton Audio TT25-8 tactile transducers mounted on each side of the support  frame. The hammock is supported far from the fulcrum at the frame’s legs and the  transducers are mounted close to the hammock hooks, resulting in an efficient  transfer of energy from the transducers to a person seated in the chair. While very  low frequency energy is strongly presented, the elasticity of the hammock’s  support webbing damps higher frequency energy, softening transients that have potential to be physically uncomfortable. 

The hammock chair provides a cocoon-like micro-environment for the person  seated in it. I found this to be a very pleasureable, safe place and I wanted to  create a shared experience for visitors while still preserving this sense of security.  To do this, I have developed a communication network based on the Seeed Studio  MR60BHA2 mmWave contactless breathing and heartbeat detection sensor. The  detected state of each visitor directs an instance of the composition, which is relayed to the transducers on their and on other visitor’s hammocks via a Max/MSP patch. This system allows for flows of information between visitors  without breaking the visual and acoustic privacy afforded by the hammock. 

While this system allows nonverbal communication between the visitors  seated in the hammocks, it does not include anyone else in the environment.  Additionally, there will be times when there is only one visitor, and something is  needed to move their experience forward. Both concerns have been addressed via  the inclusion of a seismic accelerometer in the composition’s control system. The  accelerometer is sensitive enough to detect the activity both in the immediate  vicinity of the device but also in other parts of the building and heavier activity  like road traffic at significant distance. The inclusion of the accelerometer ensures an always-evolving experience for any number of participants while connecting  them to their greater surroundings. 

As we watch the continuing genocide in Gaza and abductions of our  neighbors by government agents in broad daylight in the United States, there can  be a strong desire to retreat into a belief that these things don’t concern us, that  these tragedies are things that happen to other people. At this moment, it felt  especially critical to create a work that tries to dispel the myth of independence,  the false belief that one person’s life can exist in isolation from the lives around  them, human or otherwise. It is my hope that this work will provide a space in  which people can reflect on the depth of our interdependence and renew their commitments to mutual support.

Entrainment, originally entitled Entrainment718, was inspired by a hundred-foot stretch of the Brooklyn F train that connects the subterranean to the above-ground. On this transitional stretch, the visual interplay of regularly-spaced pylons against  haphazardly strung high-intensity work lights causes a skewing of parallax— looking out the window, depth perception becomes distorted, as though suddenly  careening through a starfield. Entrainment is an exploration of that sense of  disorientation, and the hypnotic and transcendental states that often emerge from the repetitious visual, auditory, and haptic polyrhythms experienced aboard a  moving train. This project continues to grow, include footage from rail-based public  transportation from around the world. 

Audio consists of a multi-channel sound system arranged linearly. The original  musical composition (16mins) consists of several textural layers that are distributed  spatially, running up and down the multi-channel array in gated sequences. The effect is that of a passing train, while also evoking the rhythmic quality of being on  board the train. Each channel is synced with the video panel directly behind it and  when that audio channel is active, a variety of visual effects is applied to the  corresponding panel. In this way, one can visually track the sonic placement of  sound in space (source-bonding [1] à la Denis Smalley’s Spectromorphology). 

Additionally, a haptic channel plays infrasonic polyrhythmic patterns. 

The auditory (score length vs spatial distribution), visual (footage vs effects  programming) and haptic layers are all cycles of different lengths. As they loop, the layers stack in new combinations. This gestalt of sensory information that drift in  and out of synchronization is an example of the nested or overlapping rhythms described in Henri Lefebvre’s Rhythmanalysis [2], in which the body becomes a  metronome that not only observes but feels—embodies—temporal perception. 

Most notably, the panoramic is actually a single-channel portrait-mode video (60mins) shot on a camera-phone, repeated 14 times and mirrored vertically. Each  column is an instance of the original footage offset by 23 frames; in essence, they  are pulling from the “memory” of the video, and placed side by side, they become  stitched together to form what appears to be a panoramic view. The perceived elongation of the image is achieved through a repeated temporal and spatial displacement. 

As a phenomenological experiment, Entrainment draws from a number of concepts  from neuroscience and psychotherapy: 

– The frame delay between each offset column is variable; when processing in real time, the number of frames delayed can be modulated (via OSC or MIDI) to slow  down or speed up the cascade of images. This variability plays off the elastic  perception of time—when cortisol spikes under acute stress, people often report a  slowing down of time. Memory becomes denser as more sensory information is  processed per unit of time. 

– The linear arrangement of the images and sound is designed to emulate the lateral  eye movement experienced in deep sleep. Side-to-side eye movements are tied to  internal visual processing and memory consolidation. REM sleep plays a key role  in processing emotional memories, especially those with strong visual or affective  content. The amygdala and hippocampus are both highly active during REM, and  lateral eye movements may support integration of experience, emotional regulation,  and the formation of semantic associations. 

– The left-right travel of auditory, visual and haptic cues mimics the use of such  techniques to treat PTSD in EMDR (Eye Movement Desensitization and  Reprocessing) therapy. The multi-sensory rhythmic patterning causes bilateral  stimulation, activating both the left and right hemispheres of the brain. This is  thought to help integrate fragmented traumatic memories, which are often stored in  a disjointed or somatosensory form (right hemisphere) and not fully processed by  language- and logic-dominant regions (left hemisphere). 

Entrainment only refers to such concepts as a compositional tool for artistic  expression; the use of such media-rich environments to support therapeutic efforts has yet to be fully explored. However, Entrainment is designed with flexibility to  respond in real-time to data streams (e.g. EEG) to facilitate interdisciplinary study.

I made this project after wandering around the world’s most active glacier, in Ilullisat, Greenland, with my mother and 5-year-old daughter and the film’s cinematographer, Troy Fairbanks. All synthesizers are made out of processed field recordings from this trip. The slow film stills and music are increasingly interrupted by audiovisual glitches, representing tipping points of our warming climate; the timing of these glitches was determined by a Max patch converting sea ice extent data to probability. The vocal music was created collaboratively with the members of Moving Star vocal ensemble from an open score I composed. The music was recorded by Jeff Cook at 2nd Story Sound, mixed by Michael Hammond of Big Ship Audio, and the spatial Dolby Atmos mix was created with Sean Winters. The film itself premiered at the IMAX Theatre at the Denver Museum of Nature and Science. 

As I struggle to give my child the same intimacy with the wild I enjoyed , I find myself wondering: will there be winter in the world when she is my age? So many films invite you to think about the facts and figures of climate change; I wanted to offer people something different: a glimpse at the experience I had in Greenland, a chance to slow down and feel, and be with the earth’s ice as it melts and spills its way through climate change, to witness their own feelings as music washes over them. I hope it resonates with you. 

A note on this immersive installation: MELT: the memory of ice as a film is a work of cinema; sounds and images in the shape of a cinema screen. As an installation, I wanted to truly use the gallery space to do something more than just press play on a movie, to explore the imaginative possibilities of transforming and relocating the sounds and visuals, to create a new kind of space for the audience to wander through. The visual transformations, especially, allow me to understand the images in a different, more imaginary way: a kind of inner topography. So I’ve titled this installation work MELT: the memory of ice (topographic remix).