• A machine learning system lets visitors to a new art exhibition experience the sounds of colours through the artist Kandinsky’s eyes.
  • The abstract art pioneer had synaesthesia, a condition that can make people hear sounds when looking at colours and shapes.

Have you ever heard colours?

As part of a new exhibition, the worlds of culture and technology collide, bringing sound to the colours of abstract art pioneer Wassily Kandinsky.

Kandinsky had synaesthesia, where looking at colours and shapes causes some with the condition to hear associated sounds. With the help of machine learning, virtual visitors to the Sounds Like Kandinsky exhibition, a partnership project by Centre Pompidou in Paris and Google Arts & Culture, can have an aural experience of his art.

An eye for music

Kandinsky’s synaesthesia is thought to have heavily influenced his painting. Seeing yellow summoned up trumpets, evoking emotions like cheekiness; reds produced violins portraying restlessness; while organs representing heavenliness he associated with blues, according to the exhibition notes.

Virtual visitors are invited to take part in an experiment called Play a Kandinsky, which allows them to see and hear the world through the artist’s eyes.

a picture of the 1925 painting 'Yellow, Red, Blue' by Kandinsky
Kandinsky’s synaesthesia is thought to have heavily influenced his 1925 painting Yellow, Red, Blue.
Image: Guillaume Piolle/Wikimedia Commons

In 1925, the artist’s masterpiece, "Yellow, Red, Blue", broke new ground in the world of abstract art, guiding the viewer from left to right with shifting shapes and shades. Almost a century after it was painted, Google’s interactive tool lets visitors click different parts of the artwork to journey through the artist’s description of the colours, associated sounds and moods that inspired the work.

But Google’s new toy is not the only tool developed to enhance the artistic experience.

Artist Neil Harbisson has developed an artificial way to emulate Kandinsky by turning colours into sounds. He has a rare form of colour blindness and sees the world in greyscale. But a smart antenna attached to his head translates dominant colours into musical notes, creating a real-world soundtrack of what’s in front of him. The invention could open up a new world for people who are colour blind.

Data

How is the World Economic Forum addressing challenges raised by the Internet of Bodies?

Recent technological advancements have ushered in a new era of the “internet of bodies” (IoB), with an unprecedented number of connected devices and sensors being affixed to or even implanted and ingested into the human body.

The IoB generates tremendous amounts of biometric and human behavioral data. This is, in turn, fuelling the transformation of health research and industry, as well as other aspects of social life, such as the adoption of IoB in work settings, or the provision of new options for entertainment – all with remarkable data-driven innovations and social benefits.

The World Economic Forum recently released a special report on this developing area. The August 2020 report explores how IoB raises new challenges for data governance that concern not only individual privacy and autonomy but also new risks of discrimination and bias in employment, education, finance, access to health insurance and other important areas for the distribution of social resources.