WindGate

A breath-controlled instrument, built for a contest where you weren't allowed to use your hands.

Tangible Music Lab - Out of Control 2025

A celebration of musical instruments, their technologies, and the bodies that play them, organized by the Tangible Music Lab, University of Arts Linz

Kunstuniversität Linz

Video by Ulla Rauter.

The brief

The Tangible Music Lab at Kunstuni Linz runs a contest called Out-of-Control: instruments, the technology inside them, and the bodies that play them. The 2025 rule was Hands Off: build a digital instrument you can play with anything except your hands. People turned up at the Tabakfabrik with things you play with your elbows, your knees, your feet, your tongue.

I went with breath.

Partly because it's the obvious one; a wind instrument without hands is barely a stretch. Mostly because breathing is the only thing on that list you can't switch off. You can hold it for a while. You can't decide not to do it. Which means the instrument always has an input whether you're playing it or standing there thinking about something else, and anything you do to shape the sound is really you interfering with a process that was already running.

For a contest called Out-of-Control that seemed like the right answer. This is my first real attempt at building a usable controller, and from the start I wanted it to grow into something more than that: not a device sitting between me and a separate synthesizer, but one object you play, wear, and look at, all at once.

Call for instruments · Event details

How it works

Nine TCRT5000 infrared distance sensors sit behind a 3×3 grid of small flaps set into a printed mask that sits about a centimetre off your face. Blow on a flap and it flexes. The sensor behind it reads how far.

Each flap answers to one patch of breath in front of the mouth. Breathe straight ahead and the middle column does most of the work. Turn your head, drop your jaw, push the air out one side of your mouth and it redistributes. So it ends up being less about how hard you breathe than where you aim it, which turned out to be the more playable of the two. One limit worth naming: the flaps only register the exhale. They rest flat on the inhale and there's no inward flex to read, so breathing in currently looks the same as not breathing at all. Whether a future version should read the inhale too, and whether that should mean subtracting from a texture rather than mirroring the exhale and adding to it, is a real open question I don't have an answer to yet.

A XIAO ESP32C3 reads all nine at 100 Hz, smooths them with a moving average, runs them through a per-person calibration and normalises everything to 0-1. Out the other end it's MIDI CC 20 through 28 over Bluetooth, into a PlugData patch.

Nothing is a discrete trigger. It's all continuous, so there's no wrong note to play, just textures that wander off while you're busy trying to breathe normally.

You can't see what you're doing, and that's the point

Put the mask on and the first thing you lose is the ability to watch your own hands, except there are no hands to watch. You can't see your breath land on the grid the way you'd see a finger find a key. It's not breath control in general that's hard, it's more specific than that: airflow is genuinely difficult to localize on a surface sitting directly in front of a part of your body that can't see itself.

So you end up playing on other channels instead: listening, small constant adjustment, and a kind of bodily memory for where the grid actually sits relative to your mouth. That's a real usability problem. It's also, I've come to think, the most interesting thing about the instrument. Not knowing exactly what you're about to trigger turns playing into something closer to listening while guessing, and I'd rather protect that uncertainty than design it away.

The sound engine, and why it's the wrong one

A modified version of Sound Codex's photosynthesis patch in PlugData, a fork of Pure Data. I've been calling it Botanica.

It's the wrong engine, and I'd rather write that here than have someone find out by listening to it. I also spent a while calling it granular in my head, which turned out to be wrong too: pull the patch apart and it's actually about twenty parallel voices firing impulses into resonant filters, closer to a bank of small excited resonators than to sliced-up grains of recorded sound.

Botanica is lovely regardless of what you call it. It's organic, it modulates itself, it tends to resolve whatever you give it into something pleasant. Point nine channels of erratic human breathing at it and it politely sands the lot down into ambient. All the struggle that's actually in the interface, the part where you're wrestling your own diaphragm for control of a sound, disappears somewhere between the sensor and the speaker.

Right now, if I'm honest, WindGate is still just a controller. The mask sends nine channels out as MIDI to my laptop, Botanica runs there, and the thing I'm actually playing is that patch, not the mask. WindGate should sound like a machine being made to run on something as unreliable as a person. At the moment it sounds like a nice afternoon.

The plan isn't a harsher version of the same idea, it's a different relationship entirely: get a microphone and a synthesis engine of its own inside the mask, so it picks up my voice and reshapes it with no laptop anywhere in the signal path. Voice becomes the raw material, and breath becomes the hand that reshapes it, impulse by impulse, in real time. I've been calling that embedded engine the Word Scrambler when I'm thinking about what happens to the voice, or the Synth Engine when I'm thinking about the sound process itself. Neither name is quite right yet, and I've stopped trying to force the decision, because picking one now would close off a question I'd rather keep open a while longer.

The hardware's fine. It's the sound, and what it's even made of, that I keep changing my mind about.

Extension, not prosthesis, but not only that

I keep reaching for the word extension rather than prosthesis to describe what the mask does: it becomes part of what my body can do, not a tool my body operates from a distance. But that's the flattering half of the idea, and I don't want to only take the flattering half. Strapping a synthesizer to your face doesn't just extend you, it also makes your own body less predictable to itself. You gain a new way of shaping sound and lose the ability to fully know what you're about to do with it. Both are true at once, and the second half is at least as interesting as the first.

Breathing differently

A contest about bodies raises the question of whose. Lung capacity varies. Inhale and exhale aren't symmetric for everyone. And five minutes of performance is a lot to ask of some people's chests.

What's actually built:

• Adaptive calibration. It learns your range at the start of a session instead of assuming one, and treats inhale and exhale as separate curves.
• Sensitivity adjustment, so it works for people who breathe quietly and people who don't.
• Low-effort mode: lower thresholds, for smaller lung capacity.
• Pause and resume. You can stop and rest without the patch losing its state, so the music waits instead of falling over.

Building it

Blender for the shell and the nine sensor mounts. The awkward part was holding the sensors at a consistent distance from the airflow without the housing itself turning into a wind tunnel and reading its own turbulence. Printed in PLA. A Grove shield handles power and wiring so there's nothing rattling around loose inside a thing you strap to your face.

Base model of the mask: free model
Original photosynthesis patch (Sound Codex): photosynthesis

Specifications

Playing it

26 June 2025, Tabakfabrik Linz, at the Out-of-Control contest.

Where it is now

As of this writing, WindGate is still, honestly, just a controller. The sensing side works reliably: nine channels of breath data, clean and calibrated, going out over Bluetooth. What isn't built yet is the part that would make it an instrument in its own right, a microphone and a synthesis engine living inside the mask instead of on a laptop next to it. Until that exists, what I'm playing is Botanica through a breath controller, not WindGate.

Built for the Tangible Music Lab's Out-of-Control Contest, part of the Postdigital Lutherie programme at Kunstuni Linz, 2025.