What if we could listen to paintings sing ?

Speculating about the way that we perceive the world. I was interested in understanding if we could experience certain senses in a whole new way. Considering the subjectiveness of abstract paintings, the initial thought arises from speculating if we could add the layer of sound to understand and experience paintings in a unique way.

Research

Taking two different approaches , I have tried to speculate the two different paths through which we could possibly play around with converting sound to colors and colors to sound

Chromesthisia

A variant of synesthesia, 'CHROMESTHESIA' is a cognitive condition wherein sound results in an involuntary and automatic perception of colour, shape or movement. Individuals with this ability perceive and experience the world in an individual and unique way. Different individuals with the same condition react differently to the same stimuli. Therefore not everyone sees the same colour for the same sound. The relation between the two is based on your environment, cultural as well as personal influence

Neil Harbisson & Adam Montandon

Neil Harbisson is the world's 1st cyborg. Born with Achromatopsia he could only see the world in black and white. Due to his condition, he and scientist Adam Montandon worked on a unique solution to help Neil experience colours. By attaching a chip to his brain, Adam Montandon created a device that translates colours to sound. So whilst Neil can see colours, due to the integration of his brain and the sensor, Neil Harbisson now perceives colour through sound

Emotional

Most Synesthetes state that the colours that they might see might not always correlate with others, additionally, NCBI states that for Synesthetes, emotions, culture and the understanding of the environment, also play a role in which colour they associate with which note. Due to this, I wanted to explore the possibility of showcasing this sensation through correlation emotions and colours.

I then followed the following steps, to hopefully replicate the experience
#FFFFFF
#FFF0FF
#90FFFF
Emotions
Mellow, Calm
Chord Progressions
C-G-A-F
C-Am-F-G
#FFFFFF
#FFFFFF
#FFFFFF
Emotions
Mellow, Calm
Chord Progressions
C-G-A-F
C-Am-F-G
#C01818
#A81800
#480000
Emotions
Dark, Passion, Anger
Chord Progressions
Cm-Fm-Bb-G
Ebm-Abm-Db-Bb
F#m-Bm-E-Am
Audio Credit - https://www.youtube.com/watch?v=iIL9bFq2M5k
#C01818
#A81800
#480000
Emotions
Dark, Passion, Anger
Chord Progressions
Cm-Fm-Bb-G
Ebm-Abm-Db-Bb
F#m-Bm-E-Am
Audio Credit - https://www.youtube.com/watch?v=iIL9bFq2M5k

Observations

  • A colour often evokes a variety of emotions, for example, red could be used for anger as well as passion. Therefore it becomes difficult to narrow down the required emotion for the specific colour
  • Similarly, whilst there are certain preferences when it comes to emotions and chord progressions. Musicians often disregard the notion that there can be a direct correlation between the two.
  • The Method lacks repeatability and is highly subjective to each individual

Technical

For this approach, I based my research on the findings and understandings of Neil Harbisson and Adam Montandon. Sound and light both are waves. Humans can hear within a range of - 20 to 20,000 Hz See with a range of - 405–790 THz Adam Montandon was thus of the belief that if we could slow down the frequency of light by a certain degree then they start to become audible. In order to understand which note subscribes to which colour, Neil had to go through a learning curve, after which he developed the Sonochromatic scale
‍
For my experiment, I stuck to the device developed by Adam Montandon and the scales created by Neil Harbisson.

Sonochromatic Scale

Visualizing songs

For the 1st experiment, I decided to the deconstruct popular jingle ‘twinkle twinkle little star’ , the courus of ‘Hey Jude’ and the famous track called ‘Demons by Imagine Dragons’ For this, I first, broke down the songs to their corresponding notes and then, based on the Sonochromatic scale, provided colour to each of these notes, to showcase what these sounds would look like

The Beatles - hey jude

twinkle twinkle little star

Imagine Dragons - demons

Vocalising Colours

In this experiment, I decided to let go of the music, and randomly assign a colour to each of the lines. After which I correlated each of those colours to a corresponding note to create short tracks

Track

Observations

  • The Technical Approach has higher repeatability and less dependency on environment, cultures and emotions
  • Due to its mathematical conversation, the color and note relation stays consistent
  • Other factors however, like the intensity, octave, and energy is not accounted for or included in the conversion process
next project