Skip to main content
DOMAIN 05

Experimental & Interactive Media

Projection mapping, biofeedback art, haptic design, and mixed reality performance for artists who refuse to stay in a medium.

25 skills 65 activities 34 free resources levels 5 → 9

0 of 25 skills marked done · saved on this device

Where do I start?

Start where the work stops being trivial and stops being impossible. That's your tier — jump in below.

Working tier

Apply it to your own project without a guide; debug under constraint.

L5 · Solid Control 4 activities

Digital Fabrication for Interactive Art

Custom enclosures, mounts, and tangible parts are what separate a breadboard demo from an installation that survives a gallery.

Walk away with: Exported STL + a photo of the paper mockup hugging the real board, plus a 3-line note on wal...

Model a parametric snap-fit enclosure for an Arduino + sensor in free CAD, sized to your actual board with cutouts for the USB/power port and one sensor window.

Exercise constraints

Free Tinkercad or Fusion personal license; no printer needed - validate fit with a 1:1 paper printout wrapped around the real board.

You walk away with

Exported STL + a photo of the paper mockup hugging the real board, plus a 3-line note on wall thickness and tolerance chosen.

Recreate a guided laser-cut press-fit box generator result (e.g. a finger-joint enclosure) and adjust the kerf value, documenting how kerf changes joint tightness.

Exercise constraints

Free MakerCase / boxes.py in browser; export SVG only - no machine time. Time-box 60 min.

You walk away with

Two SVGs (default vs adjusted kerf) + a note explaining why the second fits better for 3mm material.

Design a cable-management + strain-relief detail for one cable exiting your enclosure (a relief loop, a screw-down clamp, or a printed bushing).

Exercise constraints

Sketch + CAD only; justify against the field failure mode 'underestimates cabling.' Time-box 45 min.

You walk away with

Annotated CAD/sketch showing where the cable enters, how it is anchored, and what stops it yanking the board.

Take an existing fragile prototype housing and redesign it for teardown/reinstall: tool-free access panel, labeled ports, and a mounting feature.

Exercise constraints

Document the before/after; address the 'maintenance and teardown not designed' failure mode. Use only free CAD.

You walk away with

Before/after CAD renders + a 5-step teardown checklist anyone could follow without you present.

L5 · Solid Control 4 activities

Experimental Game Design

Play is a rigorous interaction-design lab; an experimental rule-set forces you to design feedback loops, not just visuals.

Walk away with: The one-page rules sheet + a short note on what surprised you in playtest and one rule you c...

Design and playtest a one-page physical/analog 'art game' built on a single experimental constraint (no winning, no words, or no eye contact).

Exercise constraints

Paper + household objects only; playtest with 2 people. Time-box one afternoon.

You walk away with

The one-page rules sheet + a short note on what surprised you in playtest and one rule you changed because of it.

Recreate a small branching-narrative game in Twine following a beginner guide, then add one mechanic that subverts player expectation (a choice that silently changes earlier text).

Exercise constraints

Free Twine; publish to a single HTML file. Time-box 2-3 hrs.

You walk away with

Playable .html + the Twee/source and a note naming the subverted expectation.

Prototype a tiny interactive 'toy' in p5.js where the only goal is to discover the interaction (procedural, no instructions).

Exercise constraints

Free p5.js editor; one input (mouse or key). Time-box 2 hrs.

You walk away with

Shared p5.js sketch link + 3 sentences on what behavior players discover and how the system signals it.

Write a critique teardown of an existing experimental/art game (e.g. from itch.io), naming its core mechanic, its intended feeling, and where mechanic and meaning diverge.

Exercise constraints

300-500 words; one concrete redesign suggestion. Time-box 60 min.

You walk away with

The teardown doc with a labeled screenshot and your single most defensible redesign.

L5 · Solid Control 4 activities

Human-Computer Interaction (HCI) for Art

Installations fail when they're designed for a perfect participant who doesn't exist; HCI is the discipline that designs for the real one.

Walk away with: A tally table of failure points + the single biggest friction you observed and your hypothes...

Run a silent observation study of one existing public interactive (a kiosk, an elevator panel, a museum touchscreen): log every hesitation, mis-tap, and abandonment for 20 minutes.

Exercise constraints

Pen + tally sheet; observe only, no interviews. Time-box 30 min on site.

You walk away with

A tally table of failure points + the single biggest friction you observed and your hypothesized cause.

Build a p5.js interaction with NO text instructions, then test whether 2 people can discover the interaction in under 30 seconds using only affordances (motion, glow, response).

Exercise constraints

Free p5.js; one interaction. Test cold with people who haven't seen it.

You walk away with

Sketch link + a note on which affordance signaled the interaction and which testers missed it.

Storyboard the full interaction arc of an imagined installation: approach -> notice -> first action -> feedback -> deepen -> exit, marking where a confused participant could fall out.

Exercise constraints

6-8 frame storyboard, hand-drawn is fine. Address 'designs for a participant who does not exist.'

You walk away with

The storyboard + an annotated 'drop-out risk' at each frame.

Add an accessibility pass to an existing interactive sketch: a non-color signal, a larger hit target, and a no-timeout option; document each change against a stated accessibility gap.

Exercise constraints

Free tools only. Directly target the 'ignores accessibility' failure mode. Time-box 90 min.

You walk away with

Before/after sketch + a 3-row table mapping each accessibility gap to the fix.

L5 · Solid Control 4 activities

Interactive Storytelling & Narrative

Branching and participatory narrative is the structural backbone of immersive theatre, narrative games, and walk-through installations.

Walk away with: Playable .html + a node map screenshot showing the branch that the variable gates.

Author a branching story in Twine with at least 3 meaningful choice points and one state variable that changes an ending.

Exercise constraints

Free Twine; publish to HTML. Time-box 2-3 hrs.

You walk away with

Playable .html + a node map screenshot showing the branch that the variable gates.

Recreate a guided ink (inkle) tutorial story, then add a 'weave' that recombines a shared scene differently based on an earlier flag.

Exercise constraints

Free ink + Inky editor; export to a single playable file. Time-box 2 hrs.

You walk away with

The .ink source + compiled story and a note on how the flag changes the shared scene.

Design a spatial/walk-through narrative: map a real room or hallway and place 4 story beats at physical locations, triggered by arrival rather than clicks.

Exercise constraints

Floor-plan sketch + beat list; no build required. Time-box 90 min.

You walk away with

Annotated floor plan with the 4 trigger zones and what each beat reveals.

Write a critique of a published interactive narrative (a Twine game or interactive doc), identifying whether choices are real (change outcome) or cosmetic (flavor only).

Exercise constraints

300-500 words; cite 2 specific choices. Time-box 60 min.

You walk away with

The critique naming one real and one cosmetic choice, with a fix for the cosmetic one.

L5 · Solid Control 4 activities

Projection Mapping

It is the field's signature first artifact - turning a physical object into a screen forces calibration, alignment, and content discipline at once.

Walk away with: A 20-30s video of the mapped object from the calibrated viewpoint + a still showing your mes...

Map a single white cardboard box or stack of cubes: build the mesh, align corners to the real edges, and run a simple animated fill that respects each face.

Exercise constraints

Free MadMapper trial (watermark ok) or free HeavyM tier; one projector, dark room. Time-box one evening.

You walk away with

A 20-30s video of the mapped object from the calibrated viewpoint + a still showing your mesh overlaid on the geometry.

Recreate a guided projection-mapping tutorial in TouchDesigner using its CamSchnappr/keystone workflow on a simple surface, following the official curriculum.

Exercise constraints

Free TouchDesigner non-commercial; document each calibration step. Time-box 2-3 hrs.

You walk away with

Saved .toe file + a screen-recording of the calibration and the note 'what mis-aligned first and how I fixed it.'

Produce a projection-mapping setup checklist covering space, power, projector throw/lumens, ambient light, alignment, and a re-calibration procedure for when the object gets bumped.

Exercise constraints

One page; must include a 'fix alignment in 2 minutes' recovery step. Time-box 60 min.

You walk away with

The reusable checklist, tested by following it on your own box map from activity 1.

Repair a deliberately broken map: move the object 10cm, observe the misregistration, and recalibrate; log how throw distance and keystone interact.

Exercise constraints

Reuse activity-1 rig; free tools. Address 'underestimates calibration.' Time-box 60 min.

You walk away with

Before/after stills (misaligned -> realigned) + a short problem note on what calibration parameters you touched.

Advanced tier

Teach it, template it, ship it under real constraints.

L6 · Advanced Control 3 activities

Augmented Reality Experience Design

AR lives or dies on whether virtual elements feel anchored in real space; the design problem is integration, not novelty.

Walk away with: Hosted demo link + a screen-capture showing the object holding its position as the camera mo...

Build a marker-based or plane-detection AR scene in the browser with A-Frame/WebXR that places one virtual object convincingly on a real surface, with correct scale and shadow.

Exercise constraints

Free A-Frame + a phone browser; no app store. Time-box 3 hrs.

You walk away with

Hosted demo link + a screen-capture showing the object holding its position as the camera moves.

Recreate Google's ARCore 'Hello AR' style placement sample (or the Unity AR Foundation equivalent) and document the plane-detection and anchoring behavior.

Exercise constraints

Free ARCore docs + Unity Personal or web; one anchored object. Time-box half a day.

You walk away with

Build + a video of placing/re-placing the object, plus a note on when tracking drifts and why.

Design (no build) an AR interaction-context map for a real location: light conditions, sightlines, how users hold the device, and the integration failure that would break the illusion.

Exercise constraints

Site photos + annotations. Address 'concept outruns reliability.' Time-box 90 min.

You walk away with

Annotated location doc naming the single biggest real-world integration risk and a mitigation.

L6 · Advanced Control 3 activities

Gesture & Whole-Body Interaction Design

Body-as-input installations are a domain signature - and a calibration/latency minefield that punishes anyone who skips the body-as-system mindset.

Walk away with: Sketch link + a video of the gesture-to-response loop and a note on the latency you measured...

Build a webcam body/hand-tracking interaction in the browser with MediaPipe or ml5.js where a gesture drives a visible response (hand position scales/colors a shape).

Exercise constraints

Free ml5.js/MediaPipe + laptop webcam (no depth camera needed). Time-box 3 hrs.

You walk away with

Sketch link + a video of the gesture-to-response loop and a note on the latency you measured.

Define a gesture vocabulary for an imagined installation: 3 gestures, each with an ergonomic note (can a tired or short person do it?), a dead-zone, and a 'rest' state that doesn't trigger by accident.

Exercise constraints

Spec doc + body diagrams. Address 'sensor noise and latency go untreated' with a debounce plan. Time-box 90 min.

You walk away with

The gesture spec with explicit rest-state and false-trigger handling.

Add smoothing/filtering to your activity-1 tracker (moving average or one-euro filter) and document the jitter reduction vs added latency tradeoff.

Exercise constraints

Free tools; measure both before and after. Note that depth cameras (Kinect/RealSense) are discontinued - use webcam tracking as the free fallback.

You walk away with

Before/after motion graph or video + the filter parameters and the latency cost you accepted.

L6 · Advanced Control 3 activities

Light Art & Interactive Lighting

Light is the most legible medium in a room; controlling it via DMX/Art-Net and reacting to people/music is core installation craft.

Walk away with: Video of the three looks + the reactive mode, plus the sketch and a wiring photo.

Drive an addressable LED strip (NeoPixel/WS2812) from an Arduino/ESP32 with 3 lighting 'looks' and one audio- or motion-reactive mode.

Exercise constraints

Single strip + microcontroller (cheap); free Arduino IDE + Adafruit NeoPixel guide. Time-box one evening.

You walk away with

Video of the three looks + the reactive mode, plus the sketch and a wiring photo.

Patch a virtual DMX fixture in QLC+ and program a cue list (chase + a manual override), running it against the built-in visualizer with no hardware.

Exercise constraints

Free QLC+; use the visualizer instead of real lights. Time-box 2 hrs.

You walk away with

Saved QLC+ workspace + a screen-recording of the cue list running, with a note on DMX channel mapping.

Write an interactive-lighting design + safety checklist: power budget for your strip length, heat, flicker/photosensitivity caution, and a fail-safe default state.

Exercise constraints

One page; include amps-per-LED math. Address 'ignores accessibility/safety' (photosensitive seizure risk). Time-box 60 min.

You walk away with

The checklist with the calculated power draw and the explicit no-strobe-above-3Hz rule.

L6 · Advanced Control 3 activities

Sensor Integration (Motion/Depth)

Sensors are the input layer of every responsive installation; reading them cleanly and treating noise/latency is the whole game.

Walk away with: Serial-plot screenshot of the signal + a video of the threshold trigger and the sketch.

Wire one sensor (PIR motion, ultrasonic distance, or a capacitive touch pad) to an Arduino, print clean readings to serial, and trigger an output when a threshold is crossed.

Exercise constraints

One cheap sensor + microcontroller; free Arduino IDE. Time-box one evening.

You walk away with

Serial-plot screenshot of the signal + a video of the threshold trigger and the sketch.

Send your sensor data over OSC into Processing/TouchDesigner and map it to a visual parameter, confirming end-to-end latency feels acceptable.

Exercise constraints

Free tools; follow the OSC 1.0 message format. Time-box 2-3 hrs.

You walk away with

Video of sensor -> OSC -> visual + a note on the message address pattern you used.

Characterize and treat your sensor's noise: log raw data, apply a threshold + debounce + smoothing, and document false-trigger rate before/after. For depth, note that Azure Kinect/RealSense are discontinued and plan around Orbbec Femto Bolt or webcam tracking.

Exercise constraints

Free tools; quantify the improvement. Address 'sensor noise and latency go untreated.' Time-box 2 hrs.

You walk away with

Before/after false-trigger counts over a fixed test + the filtering parameters chosen.

L6 · Advanced Control 3 activities

Sound Installation & Soundscape Design

Spatial, walk-through sound is its own medium; arranging sources in a room and composing ambient environments is distinct from stereo mixing.

Walk away with: Audio export + the Pd patch and a note on how you avoided an obvious loop point.

Compose a 3-5 minute generative/ambient soundscape in Pure Data (or a free DAW) with at least two layers that evolve independently so it never loops audibly.

Exercise constraints

Free Pure Data; headphones fine. Time-box one evening.

You walk away with

Audio export + the Pd patch and a note on how you avoided an obvious loop point.

Design a multi-speaker spatial layout for a real room: plan source placement, listener path, and which sounds a person hears as they walk a route; simulate with 2 speakers + panning if you can't build it.

Exercise constraints

Floor plan + a 2-speaker panning test. Time-box 2 hrs.

You walk away with

Annotated floor plan + a binaural/stereo render that demonstrates the walk-through experience.

Make the soundscape reactive: feed one sensor/OSC value (presence or distance) into Pd to shift the mix when a 'visitor' approaches.

Exercise constraints

Free Pd + a sensor or simulated OSC value. Time-box 2 hrs.

You walk away with

Video/audio of the mix responding to the input + the patch and the mapping note.

L6 · Advanced Control 3 activities

Urban Media & Public Interactive Art

Public space is the hardest deployment context - weather, power, sightlines, unsupervised use, and crowds break studio-perfect prototypes.

Walk away with: The production plan with a site map, a power/weather risk table, and the unattended fail-saf...

Write a public-installation production plan for a real site: power source, mounting, weatherproofing (IP rating), sightlines, accessibility, daily maintenance, and an unattended fail-safe.

Exercise constraints

Use a real location you can photograph. Address 'prototype works in studio but fails with crowds.' Time-box half a day.

You walk away with

The production plan with a site map, a power/weather risk table, and the unattended fail-safe behavior.

Storyboard the unsupervised interaction: how a passer-by understands it in 5 seconds, what stops vandalism/misuse, and how it resets between users.

Exercise constraints

6-8 frames; explicitly design the reset and the abuse case. Time-box 90 min.

You walk away with

Storyboard with a labeled 'reset between strangers' frame and a misuse mitigation.

Build a small weather/operations runbook: a startup/shutdown sequence, a 'what to check each morning' list, and a remote heartbeat/monitoring idea.

Exercise constraints

One page. Address 'maintenance and teardown are not designed.' Time-box 60 min.

You walk away with

The runbook, concrete enough that a volunteer could open and close the piece without you.

L6 · Advanced Control 3 activities

Virtual Reality Experience Design

VR presence is built from locomotion, comfort, and interaction conventions; getting these wrong causes nausea and breaks immersion fast.

Walk away with: Hosted link + a capture of the gaze/grab interaction and a note on the eye-height you set.

Build a tiny WebXR VR scene (A-Frame) the viewer can look around in, with one grabbable/gaze-triggered object and a comfortable default scale.

Exercise constraints

Free A-Frame; works in browser, viewable in a cardboard viewer or desktop. Time-box 3 hrs.

You walk away with

Hosted link + a capture of the gaze/grab interaction and a note on the eye-height you set.

Recreate a Unity XR Interaction Toolkit starter scene against OpenXR and add one comfort feature (teleport locomotion or vignette on turn).

Exercise constraints

Free Unity Personal + OpenXR; document the comfort choice. Time-box half a day.

You walk away with

Build/capture + a note citing why you chose teleport over smooth locomotion for first-time users.

Write a VR comfort + presence checklist: frame-rate floor, locomotion options, UI distance, snap-turn, and a seated fallback; map each to the discomfort it prevents.

Exercise constraints

One page. Address 'designs for a participant who does not exist.' Time-box 60 min.

You walk away with

The checklist with each item tied to a specific nausea/comfort risk.

L6 · Advanced Control 3 activities

Wearable Tech & E-Textiles

Garments add electronics to a moving, sweating body - so soldering gives way to conductive thread, flex, washability, and comfort constraints.

Walk away with: Photo/video of the working soft circuit + a note on where thread shorts almost happened and...

Sew a simple soft circuit: a coin-cell, conductive thread, and one LED on fabric that lights when a sewn switch/snap closes.

Exercise constraints

Cheap LilyPad-style parts or copper tape + battery; follow the Adafruit/SparkFun e-textile basics. Time-box one evening.

You walk away with

Photo/video of the working soft circuit + a note on where thread shorts almost happened and how you routed around them.

Recreate an Adafruit wearable guide (e.g. sound- or motion-reactive NeoPixel accessory) on a sewable microcontroller and adapt the animation.

Exercise constraints

FLORA/Gemma-class board or ESP32; free guide + Arduino IDE. Time-box half a day.

You walk away with

Video of the reactive garment piece + the modified code and the wearability note (weight, battery placement).

Document a wearability + safety pass: strain relief at flex points, battery safety, washability/removal plan, and skin comfort; redesign one weak point from activity 2.

Exercise constraints

Annotated sketch/photos. Address 'concept outruns reliability' for a body that moves. Time-box 90 min.

You walk away with

Before/after of the redesigned flex point + the safety/wash checklist.

L7 · Expert 2 activities

Audience Participation & Crowd Interaction

Letting a whole crowd steer one artwork raises networking, aggregation, and crowd-behavior problems a single-user demo never reveals.

Walk away with: Demo link + a video of multiple phones changing the shared screen and a note on how you aggr...

Build a phone-as-controller crowd interaction: a web page audience members open to vote/move/color a shared big-screen visual in real time via WebSockets.

Exercise constraints

Free stack (Node/Glitch + p5.js); test with 3-5 phones. Time-box a day.

You walk away with

Demo link + a video of multiple phones changing the shared screen and a note on how you aggregated conflicting inputs.

Write a crowd-dynamics + load production note: what happens at 5 vs 500 users, rate-limiting, dominant-user prevention, and a graceful-degradation plan.

Exercise constraints

One page. Address 'prototype works in studio but fails with crowds.' Time-box 90 min.

You walk away with

The note with a concrete aggregation rule (e.g. averaged, not last-wins) and a fallback at overload.

L7 · Expert 2 activities

Biofeedback & Biometric Art

Driving art from heart rate or muscle signals turns the body into the instrument - which demands consent, calibration, and safety boundaries.

Walk away with: Video of the biosignal driving the visual + the signal trace and a note on baseline calibrat...

Capture one live biosignal (heart rate via a cheap pulse sensor, or breath via a stretch/mic proxy) into Arduino, then drive a visual whose intensity tracks the signal.

Exercise constraints

Cheap pulse sensor or webcam-rPPG/breath proxy; free tools. Time-box a day.

You walk away with

Video of the biosignal driving the visual + the signal trace and a note on baseline calibration per person.

Draft a participant consent + data-handling protocol for biometric art: what is captured, that nothing is stored, opt-out mid-experience, and a per-person calibration step.

Exercise constraints

One page. Directly address 'body data/biometric work lacks consent boundaries.' Time-box 90 min.

You walk away with

The consent script + data-handling note a participant could read in 30 seconds before opting in.

L7 · Expert 2 activities

Haptic Feedback Design

Tactile feedback is timing-critical: a buzz 100ms late feels disconnected, so haptics is a latency- and perception-design problem.

Walk away with: Video + the waveform/timing code and a note on the discrimination test result (which two got...

Drive a vibration motor / LRA from a microcontroller and design 3 distinct haptic 'words' (a tap, a rising alert, a heartbeat) tied to digital events; test if a blindfolded person can tell them apart.

Exercise constraints

Cheap vibration motor + microcontroller; free Arduino IDE. Time-box one evening.

You walk away with

Video + the waveform/timing code and a note on the discrimination test result (which two got confused).

Measure and tune event-to-haptic latency: log the delay between the digital trigger and the felt buzz, then minimize it and document the perceptual threshold you targeted.

Exercise constraints

Free tools; reuse activity-1 rig. Address 'latency' as the body-as-system failure mode. Time-box 90 min.

You walk away with

Before/after latency numbers + the note citing why sub-~50ms matters for feeling 'connected.'

L7 · Expert 2 activities

Live Performance Technology Integration

Live shows have no retry button; syncing projections, sensors, and instruments to a performer's timing under pressure is pure production discipline.

Walk away with: Video of a full run-through hitting cues on time + the cue list and the show file.

Build a cue-driven A/V rig where a single trigger (MIDI/OSC/keypress) advances synchronized visuals + audio + lighting, rehearsed against a fixed timeline.

Exercise constraints

Free TouchDesigner/QLC+/Pd + Chataigne as the conductor; one performer or a recording as the timing source. Time-box a day.

You walk away with

Video of a full run-through hitting cues on time + the cue list and the show file.

Write a show production case study: signal-flow diagram, a failure-recovery plan (what the operator does if a cue misfires live), and a soundcheck/tech-rehearsal checklist.

Exercise constraints

Address 'concept outruns reliability' with a live recovery procedure. Time-box half a day.

You walk away with

The case study with the signal-flow diagram and an explicit 'cue 7 failed, do this' recovery step.

L7 · Expert 2 activities

Physical Interface & Controller Design

Bespoke tangible controllers turn abstract digital systems into something a hand understands; the craft is mapping physical affordance to digital meaning.

Walk away with: Video of the controller driving software + the wiring/code and a note on why the physical ma...

Build a custom tangible controller from cheap inputs (a potentiometer, a force sensor, a rotary encoder) that maps physically to a digital parameter people can read without instructions.

Exercise constraints

Cheap parts + Arduino as a USB-MIDI/HID or OSC device; free tools. Time-box a day.

You walk away with

Video of the controller driving software + the wiring/code and a note on why the physical mapping is intuitive.

Iterate the controller's industrial-design feel: prototype 2 enclosure/knob variants, test grip and labeling with a user, and document which felt right and why.

Exercise constraints

Cardboard/3D-print mockups + free CAD. Address 'documentation is too thin for reinstall.' Time-box half a day.

You walk away with

Before/after of the two variants + the user-test note and a wiring/assembly doc for rebuilding it.

L7 · Expert 2 activities

Telematic & Networked Performance

Performing across locations makes latency and synchronization the artistic material itself, not just a technical nuisance.

Walk away with: Video of an action mirrored across both ends + the measured round-trip latency and how you d...

Build a two-location synced visual: send OSC/WebSocket data between two machines (or two browser tabs simulating remote sites) so an action in A appears in B, and measure the round-trip latency.

Exercise constraints

Free stack (Node/WebSockets or OSC over LAN); a second device or VM stands in for the remote site. Time-box a day.

You walk away with

Video of an action mirrored across both ends + the measured round-trip latency and how you designed around it.

Write a latency-as-material design note: choose to hide, embrace, or compensate for delay, and define a sync strategy (shared clock vs event-driven) with a degraded-network fallback.

Exercise constraints

One page. Address synchronization explicitly. Time-box 90 min.

You walk away with

The note naming your latency strategy and what the piece does when one side drops.

Frontier tier

Fuse across domains and extend the field. Earned, not entered.

L8 · Cross-Domain Mastery 1 activity

BioArt & Living Systems

Living media evolve and decay on their own timeline, demanding lab care, safety, and ethics far beyond any deterministic interactive system.

Walk away with: Timelapse/video of the living signal driving the output + a logged care plan and a note on h...

Prerequisite chain

Requires Sensor Integration plus basic biology/biotech familiarity and lab-safety/ethics grounding before working with living media; treat consent, containment, and care-over-time as non-negotiable.

Build a plant/environment-as-sensor piece: read a living-system signal (plant electrical activity via a galvanic probe, soil moisture, or growth via timelapse) into Arduino and drive a responsive output, with a care/keep-alive plan.

Exercise constraints

Cheap probe/moisture sensor + a houseplant; free tools. No regulated organisms - use plants or food-safe cultures only. Time-box several days (living timeline).

You walk away with

Timelapse/video of the living signal driving the output + a logged care plan and a note on how the piece changes as the organism does.

L8 · Cross-Domain Mastery 2 activities

Immersive Experience Design (Multisensory)

True immersion is the integration discipline - visual, audio, tactile, and scent must agree or the illusion collapses; this is where the domain's 'integration is the failure mode' lesson peaks.

Walk away with: Video walkthrough showing all channels firing in sync to the same trigger + a sensory-map do...

Prerequisite chain

Combine prior audio, lighting, projection, and sensor skills; do not attempt before you can independently ship at least two of those single-channel artifacts. Multisensory work fails when one channel is bolted on rather than co-designed.

Design and prototype a small multisensory 'pod': one enclosed corner where synchronized sound, light, and one tactile or scent cue respond to a participant's presence, all co-designed to a single emotional beat.

Exercise constraints

Reuse your sound (Pd), light (Arduino/QLC+), and sensor rigs; free tools + cheap parts. Time-box a weekend.

You walk away with

Video walkthrough showing all channels firing in sync to the same trigger + a sensory-map doc binding each channel to one intended feeling.

Produce an integration case study analyzing where your pod's channels almost desynced and how you bound them to one clock/trigger, plus the accessibility accommodation for a participant who can't use one sense.

Exercise constraints

Address 'failure mode is integration, not concept' and 'ignores accessibility.' Time-box half a day.

You walk away with

The case study with a sync/timing diagram and a stated single-sense fallback experience.

L8 · Cross-Domain Mastery 2 activities

Interactive Installation Design

This is the domain's capstone planning skill: orchestrating space, visuals, sound, and responsive elements into one coherent audience experience.

Walk away with: Video of a stranger using it unaided + a full system diagram, the source files, and a reset-...

Prerequisite chain

Assumes working competence in sensors, real-time visuals/projection, and sound, plus production planning; the artifact is an integrated system, not a single-channel demo. Design teardown, reinstall, and maintenance from the start.

Build a complete small interactive installation: a sensor input drives synchronized visuals and sound in a defined space, documented end-to-end as a working system (approach -> response -> reset).

Exercise constraints

Reuse TouchDesigner/p5.js + a sensor + Pd; one corner of a room. Time-box a weekend.

You walk away with

Video of a stranger using it unaided + a full system diagram, the source files, and a reset-between-users behavior.

Write the install/reinstall + maintenance documentation pack for that installation so a different technician could set it up without you: bill of materials, wiring, calibration, startup/shutdown, and teardown.

Exercise constraints

Address 'documentation too thin for reinstall' and 'teardown not designed.' Time-box half a day.

You walk away with

The doc pack, validated by tearing down and rebuilding your own piece following only the document.

L8 · Cross-Domain Mastery 1 activity

Mixed Reality Performance

Blending live performers with real-time digital elements means coordinating human timing with tracking and visuals - the hardest sync problem in the domain.

Walk away with: Video of a full rehearsed run where visuals track the performer + the cue sheet and the show...

Prerequisite chain

Requires theatrical/production basics plus prior sensor, projection, and live-integration artifacts; rehearsal time and a recovery plan for tracking loss are mandatory before any audience is present.

Stage a one-performer mixed-reality scene: body/marker tracking (webcam/MediaPipe) drives real-time projected or screen visuals that respond to the performer's movement, rehearsed to a fixed cue sequence.

Exercise constraints

Free TouchDesigner/Unity + webcam tracking (depth cameras discontinued - use webcam as the free path); one performer. Time-box a weekend.

You walk away with

Video of a full rehearsed run where visuals track the performer + the cue sheet and the show file.

L8 · Cross-Domain Mastery 1 activity

Robotic & Kinetic Art

Moving mechanical elements add physical force, safety risk, and motion choreography to interactivity - mistakes here can injure, not just glitch.

Walk away with: Video of the responsive motion + the code, a wiring photo, and a documented safe-motion enve...

Prerequisite chain

Requires mechanics/robotics basics plus physical computing; mechanical safety (pinch points, force limits, emergency stop) must be designed before powering any moving piece near people.

Build a small kinetic piece: a microcontroller drives a servo/stepper to choreograph a repeating motion that changes when a sensor detects a person, with a defined safe range of motion.

Exercise constraints

Cheap servo/stepper + Arduino + one sensor; free tools. Bound the motion mechanically so it cannot pinch. Time-box a weekend.

You walk away with

Video of the responsive motion + the code, a wiring photo, and a documented safe-motion envelope / e-stop.

L9 · Visionary 2 activities

Volumetric Display & Holographic Art

Creating true 3D-in-space imagery is an emerging frontier with severe hardware constraints; the visionary move is to make something compelling within (or around) those limits.

Walk away with: Video of the floating 3D illusion from multiple angles + the Blender file and the 4-view ren...

Prerequisite chain

Requires 3D modeling/animation plus an understanding of volumetric/holographic display limits; most learners must simulate the medium, since real volumetric hardware is rare and costly. Treat the constraint as the creative problem.

Build a free Pepper's-ghost / reflective pyramid volumetric illusion: model a rotating 3D form in Blender, render the 4-view layout, and display it through a clear acrylic/CD-case pyramid on a phone or monitor.

Exercise constraints

Free Blender + a DIY acrylic pyramid (cheap); no specialized hardware. Time-box a weekend.

You walk away with

Video of the floating 3D illusion from multiple angles + the Blender file and the 4-view render, with a note on the medium's viewing-angle limits.

Write a practice essay/spec proposing a volumetric piece for real hardware (e.g. a Looking Glass-class light-field display or a swept-volume display), naming the technical constraints and how your content design works with them.

Exercise constraints

Research real volumetric display constraints; no hardware purchase. Time-box half a day.

You walk away with

The spec/essay with a content-design approach explicitly shaped by the chosen display's resolution/viewing-cone limits.

Free Resource Library

Everything Here Opens. No Cost to You.

34 curated, free-to-open references. Filter by what you need.

Arduino Documentation Docs

Canonical reference for boards, sensors, and the Arduino language; the spine of every physical-computing activity here.

docs.arduino.cc
OpenXR 1.0 Specification (Khronos) Docs

Royalty-free cross-vendor XR API spec; target this (not a single headset SDK) for portable AR/VR/MR.

registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html
Unity XR Manual Docs

Entry point for Unity VR/AR/MR projects, XR plug-in management, and the XR Interaction Toolkit.

docs.unity3d.com/Manual/XR.html
OSC 1.0 Specification Docs

The message protocol that glues sensors, audio engines, lighting, and visuals together across machines.

opensoundcontrol.stanford.edu/spec-1_0.html
TouchDesigner Documentation (Derivative wiki) Docs

Operator reference, Python, and tutorials for the dominant real-time node-based installation tool.

docs.derivative.ca/Main_Page
MadMapper Documentation Docs

Official docs for projection mapping, LED, DMX, and laser output.

docs.madmapper.com
Resolume Support & Manual Docs

Advanced Output / projector blending and warping reference for projection-mapped shows.

resolume.com/support/en
Adafruit Learning System Docs

3,300+ free, build-grade hardware guides - wearables, sensors, NeoPixels, soldering.

learn.adafruit.com
Processing Tool

Free/open-source creative-coding language; fastest path from idea to sketch-on-screen.

processing.org
p5.js Tool

Browser-native creative coding; zero install, runs on phones, easy to publish/share.

p5js.org
TouchDesigner (non-commercial) Tool

Free non-commercial license of the real-time visual programming environment used across the field.

derivative.ca/download
Arduino IDE Tool

Free IDE/CLI for programming microcontrollers and reading sensors.

arduino.cc/en/software
Pure Data (Pd) Tool

Free visual dataflow language for audio, control, and sensor-driven sound/soundscape work.

puredata.info
Blender Tool

Free/open-source 3D for modeling assets, mockups, and content for XR and volumetric work.

blender.org
Unity Personal Tool

Free tier of the engine for AR/VR/MR experience prototypes and interactive simulations.

unity.com/products/unity-personal
QLC+ Tool

Free/open-source DMX/Art-Net/sACN lighting control for light art when a hardware console is out of reach.

qlcplus.org
TouchDesigner Curriculum (Derivative) Course

Free structured 100/200-level course series for the field's core real-time tool.

learn.derivative.ca
Processing Tutorials Course

Free step-by-step lessons covering coordinates, motion, interaction, and data.

processing.org/tutorials
p5.js Tutorials Course

Free guided tutorials; pairs with the editor for instant in-browser results.

p5js.org/tutorials
The Book of Shaders Course

Free step-by-step GLSL fragment-shader book for generative/light visuals.

thebookofshaders.com
openFrameworks Learning + ofBook Course

Free C++ creative-coding lessons and the full ofBook for performance-critical installation code.

openframeworks.cc/learning
SparkFun Tutorials Course

Hundreds of free electronics/sensor/e-textile guides with wiring diagrams.

learn.sparkfun.com/tutorials
MDN WebXR Device API Course

Free reference for building AR/VR that runs in a browser, no app-store gate.

developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API
r/TouchDesigner Community

Active troubleshooting and patch-sharing for the dominant installation tool.

reddit.com/r/TouchDesigner
r/creativecoding Community

Cross-tool generative/interactive work, critique, and source sharing.

reddit.com/r/creativecoding
Processing / p5.js Discourse Forum Community

Official help forum spanning Processing, p5.js, and Processing.py.

discourse.processing.org
Arduino Forum Community

Sensor wiring, debouncing, power, and microcontroller debugging.

forum.arduino.cc
NYU ITP Community

Reference program and public archive for physical-computing and interactive-media pedagogy.

itp.nyu.edu/itp
Chataigne (community + source) Community

Free OSC/DMX/MIDI show-control 'conductor' with an active modules community for wiring installations together.

benjamin.kuperberg.fr/chataigne
OpenXR (Khronos) Spec

The royalty-free open standard for XR; write once, run across headsets.

khronos.org/openxr
OSC 1.0 Specification Spec

Transport-independent message protocol for inter-device control - the lingua franca of installations.

opensoundcontrol.stanford.edu/spec-1_0.html
WebXR Device API (W3C/MDN) Spec

Open web standard for AR/VR in the browser.

developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API
DMX512 / Art-Net (via QLC+ docs) Spec

Practical reference implementation of the DMX/Art-Net lighting-control standards.

qlcplus.org
SIGGRAPH Immersive Pavilion Spec

Field benchmark for what 'accepted' immersive/interactive work looks like; submission criteria double as a quality rubric.

s2026.siggraph.org/program/immersive-pavilion