This augmented reality app teaches children to program robots

Programming a robot: something most kids would love to do. “RosieReality” makes it possible – even if it’s only in augmented reality. The ETH spin-off plans to use the new technology to teach young kids about programming and robotics.

“In 15 years’ time kids will find it strange for anyone to make a distinction between the real and the virtual world.” What sounds like the words of a computer geek in a science fiction film comes from the mouth of a young man who seems pretty grounded: 27-year-old Sélim Benayat is a biophysicist and co-founder of RosieReality, a start-up whose mission is to introduce children to programming and robotics using an AR robot.

Rosie is a friendly exploration robot from a distant galaxy. While traveling through space she got too close to earth and crashed on the planet. Rosie now needs the help of children to recover her memory and explore planet earth. This is the story behind the Rosie-App, which the developers hope to become a worldwide success.

Alternative to an “expensive toy”

The story starts with Sélim Benayat and Peter Spence, a former engineering & innovative design student at Imperial College London, playing around with a robot for pre-school children. The project was further developed in the Autonomous Systems Lab at ETH Zurich and marked the start of a close friendship between Benayat and Spence.

However, the ambitious young researchers soon noticed that the hardware involved in building a real robot-toy is far too complex and expensive: “Such a toy would not be affordable for parents or schools,” Benayat says. So they looked into alternatives to make robotics more accessible. This provided the initial spark for “RosieReality”.

Fusing virtual and real worlds

If you install the app on your smartphone, you see the real environment with Rosie as a virtual robot figure, in some ways as we know it from Pokémon Go, for example. The virtual robot can be "programmed" using simple commands: Players give Rosie instructions by moving the smartphone and selecting virtual objects. In contrast to games that are completely set in a virtual world - such as Minecraft - the real world merges with the virtual world - real "Augmented Reality".

What sounds relatively simple in theory is tricky to put into practice: one of the biggest challenges is to add a tangible and credible third dimension to a two-dimensional object, the smartphone screen in order to provide the user with a real 3D world experience. This is known in the jargon as “physicality”. This is where design specialists like Spence are needed. Interaction patterns need to be clear enough for everyone to understand. AR experiences are a new way of interacting with digital objects inhabiting the real world – users need some training.

New app keeps children’s interest for longer

Thousands of children ages 8 and up have tested the app in recent months. Some of them spent up to 20 minutes playing with Rosie the robot – a relatively long time compared with other apps. Other tests were a “complete flop”, as Benayat freely admits. Anyone who tries out the app can understand these failures: the first thing you have to grasp is that the virtual robot does not move by itself or via the touchscreen – you have to “programme” it by moving the smartphone and using the correct 3-dimensional programming command associated to Rosie’s movements. This is where the education in programming and robotics starts for the kids.

Despite these teething problems, Benayat is confident that the project has a bright future. And he’s not alone in this: the project, which is currently based at the Wyss Centre in Zurich, a joint research and development centre run by ETH and the University of Zurich, has attracted initial funding from the Gebert Rüf Foundation, and later raised the majority of its seed capital from Silicon Valley-based venture capitalists.

Target: 100 million users

The company was founded in January 2018. Spence and Benayat were joined by the robotics specialist Florian Maushart, who Benayat tempted back from the USA to Switzerland to co-found RosieReality. RosieReality already employs nine people, including some 3D game specialists. Benayat reckons the company will expand to at least 15 people in the first quarter of 2019.

Over the next three years the young entrepreneurs want to achieve 100 million users worldwide, making the company one of the major competitors on the gaming scene. In the future the company aims to generate money through monthly subscriptions. The content will be produced like a TV series, with new interactive episodes added continuously. Such episodes could allow the user to help Rosie explore the Sahara Desert, the Himalayan mountains, or many other exciting places.

“Making the world a slightly better place”

But there’s still plenty to do before this happens. Currently, the Rosie app only runs on the newer generation of iPhones and is still being tested. New users from all over Switzerland are added every week. Every detail of their behaviour is recorded (in an anonymised way) so that it can be incorporated in the further development of the software. Here Benayat and his colleagues are interested in more than simply bringing a new educational toy onto the market. “We want to make the world a slightly better place,” says the biophysicist with genuine intent.

This may sound like virtual happiness, but Benayat calls it “social interaction”. Soon several users will be able to interact with Rosie at the same time and in the same real-world location – in so-called “shared experiences”. Here the players will have to support each other to make sure Rosie does what she’s supposed to. Is that going to work? “Parents are starting to acknowledge that games are not a bad idea per se,” Benayat is convinced. “And they think it’s cool that some of this collaborative and educational work takes place in the real-world amongst real friends.” This is one of the many advantages of Augmented Reality.