This research project is focused on pitch perception and music education and involves elements of acoustics and psychology. My responsibility was to create a projection-installation that echoed an experience of synesthesia by developing, coding, and manipulating patches and functions in Max8/MSP and C in order to detect and interpret changes in the live video signal via a perceptron structure.

​

Fluctuations in the values of different regions of the video feed correspond to changing voltage values of the program's input. These voltages and their relative rate of change are then utilized by the visualizer program to generate output sounds and their corresponding visuals. This is done by assigning specific output choices to specific input signals. The program alters the parameters of the difference function in real-time. This process is somewhat analogous to how the iris of the human eye responds to bright or dim light. Different pitches, tones, intervals, and visual outputs are generated by the program by considering both the changes in the input feed and the fluctuation of the changes (of the input feed) over time.

​

​

​

The following videos are examples of the final installation and some of its visual/computational components.