A music toy demo transforms drawing into sounds with a novel way to promote children’s abilities of innovation and interest in music. 

By using the listening and touching sense could let children feel that they can create their music by scrawling in a relaxed manner.  



Individual Project

(with a helper of technical realization)

09/2017 - 11/2017

With the improvement of the living standard in China, many parents begin to focus on children's musical education which they think is right for kids’ intelligence. However, many children have lost their interest in music, for they are forced to learn. Meanwhile, the research demonstrates that children like to express themselves by scrawling. Painting and music are both of artistic forms in which practicing children’s ability of innovation and creation.

How about combining drawing with music to enhance children’s perception and interests of music while improving children’s innovative capability? Using the sense of listening and touching can make children feel that they can create their music by scrawling in a relaxed manner. 

The project primarily targets the children who are 4-7 years old; this period is the best age to form the sense of art. I went to a kindergarten and randomly interviewed ten parents and kids to learn about their experience and opinion of music learning and my design concept. Here are the most common responses.

The biggest challenge for this project is to design the technical solution for the idea. How to record the picture and transform it into a melody is the problem that confuses me. 


In the beginning, my design subject was a board under the picture, installed with some grid-shaped thin-film pressure sensors which connect with a note. The grid will fall over when the painting brush passes by.

However, the cost of using multiple sensors is too high, and it can only get a very imprecise position data due to the large size of each piece of the thin-film sensor.


I substituted the IR infrared sensors for the pressure sensors, which was to establish a coordinate system in the mainboard. With infrared led light, the painting brush can be located by the infrared detecting module which tracks the painting brush all the time.

This scenario is about several children playing the game “Draw a Song.” The boy is recording the chirp of the birds by Daughter Part. The girl with a hair clasp is marking her drawings. The other girl is listening to the music being transformed from the pictures into the melody which is mingled with sound recorded from nature.

“Draw a Song” has a main body and several subdivided parts. The function of the main body is to record the contents drawn by the users and translate the positions of lines into points in the coordinate system and create a melody made of corresponding musical notes. Additionally, the main body will transmit signal by Bluetooth to subdivided parts to play the sound recorded in them by their locations, which joins the main melody during the process of playing.


The function of the subdivided parts is to record the sounds from the real world and occupy two coordinates of an area of the paintings to wait for the playing signal from the main body.

1. Press the “Record” button on the Main body

2. Use a pen to draw a picture.

3. Press the “Stop” button on the Main body.

4. Optional *Use Subdivided Parts to record some related voice in 2-3s.

5. *Put the Subdivided Parts no matter where on the picture you’d like to.

6. Press the “Play” button on the Main body then you can hear the unique song of the picture you drew (with the related voice if you have used the Subdivided parts).

With the help of a friend of technique realization, my function realization began with adjusting the Positioning IR Camera and putting musical notes to the coordinate system in Positioning IR Camera. Then we combined the other functions with the main board of Arduino and began tests after modulation of single modules in a week.

Adjusting lines and resetting the Bluetooth sensor to fixed up the problem that he subdivided parts could not receive the playing signal from the main body, for there was something wrong with the connection.

Fixing the problems of the inconsistency of playing notes: Enhancing the precision of the standard axis so that the line can be identified in a more specific position. The more precise coordinate positions can be, the more accurately the points of paintings can be captured and recorded. 

This is the final model I made after fixing the bugs. It is mainly focused on the realization of technology and function, instead of the outlook design.