PERANAKAN MUSIC TILES
Conventionally, exploring the textures and patterns of Peranakan tiles is limited to visual and tactile observation. This interactive interface enriches the learning process by introducing an auditory dimension, mapping specific audio beats to distinct elements of the tile's design. Ultimately, this multimodal approach enables participants to recall not only the look and feel of the textures, but also their corresponding sounds, enhancing the overall texture learning experience.
WHAT ARE
PERANAKAN TILES?
HISTORY OF PERANAKAN TILES
Ceramic wall tiles were common in Peranakan houses back in the early 1900s. These prevalence in their use lead to it being labelled as the the term "Peranakan Tiles". However, these tiles have varying origins, ranging from Chinese, Malay to European roots. Some of these tiles were imported from places like Britain, France and Japan, reflecting the diversity of the Peranakan culture and their cohesive integration of different cultures.
CAN WE LEARN VISUAL TEXTURES WITH
SOUNDS?
THE INTERACTION CONCEPT
Users place the different components from two sets of tile designs into a empty frame to collage their own tile. Each tile has their own unique musical beat that represents different Peranakan Instruments, which layers with other components as the collage is formed, ultimately creating their own tile design and their own musical composition.
THE PHYSICAL INTERFACE
The physical interface was designed with a modular concept, where each component represents specific parts of a Peranakan tile design. This modularity allows users to customize their own tile design and compose their own musical composition. The interface was 3D modelled in Blender, with considerations to allowances in the frame design to allow for the pieces to fit in effectively.
TOUCHDESIGNER VISUALS
Placing a physical tile component triggers its corresponding pattern on the digital interface, providing the user with immediate visual feedback of a successful input. Furthermore, the on-screen visuals and graphs react dynamically to the audio beats, creating an immersive experience where the user's tile collage appears to come to life.
SYNTHESIZING THE MUSIC
The music was synthesized with VCV Rack, where each component's beat were carefully crafted to mimic the different Peranakan Instruments through the adjustments of the sound's acoustics and audio properties such as timbre and pitch. The beats follow the same clock rhythm, allowing for the different beats to sound cohesive when it is being layered together.
COLOR RECOGNITION AS A TOOL
To enable the input of the physical components to communicate with TouchDesigner and VCV Rack, a webcam was used to capture the image of the tile collage. This image is then processed in TouchDesigner, where color data is recorded and processed into node networks that translate the data into triggers for the audios and visuals.