After an earthquake, drones to the rescue?

Scaramuzza (pictured) believes that small, nimble drones that can fly autonomously through buildings to map damage and spot survivors could be of huge help to search-and-rescue teams. Meinrad Schade

Davide Scaramuzza develops autonomous drones that rely on their own cameras and sensors to navigate dangerous and unpredictable search-and-rescue situations. He tells swissinfo.ch about some of the most promising applications — and risks — associated with autonomous drones.

A personal mission

There are two reasons why I work on autonomous drones. The first is personal: I come from the region of Umbria in Italy, which has been devastated by earthquakes. Several villages dating back to the Middle Ages have been destroyed in the last decades, like Amatrice. My hometown was fortunately not hit, but I always wanted to develop robotic technologies that can help save people’s lives. 

PLACEHOLDER

The second reason is because search-and-rescue is one of the toughest and most challenging problems in robotics: If you have a drone that can fly in such complicated environments, among rubble and other things, then you can solve many other robotics problems. I want to build drones that are very small and can communicate with each other and can therefore be sent inside a building to look for survivors.

In my lab we are using onboard cameras that allow a drone to identify open space, avoiding all the obstacles on the way, and to build a map of the environment. Building a map is very important for rescue operations, because after an earthquake for example, the [original maps] might not be usable anymore. You want a drone to rebuild a 3D, photorealistic map of the environment and the location of the victims, so the search-and-rescue people can plan their action more efficiently.

Drones with an eagle eye

What if you want a drone that can do things that are easy for animals, but difficult for robots? You need a super-human, super-animal eye, and in our opinion there are three challenges that need to be overcome: you need sensors that are super-fast, have no motion blur and have a very high dynamic range – meaning they can see both in low and very bright light.

In the last four years, we’ve been working on a new class of cameras invented in my department in ZurichExternal link  called event cameras, which promise to solve all three of these problems. Event cameras have smart pixels, and every time a pixel detects a change of intensity, it triggers an event. What you get is not an image, but information corresponding to everything that is moving in the scene. So, we can say an event camera is a motion sensor.

Event cameras are useful for obstacle avoidance: they can detect motions at a rate of 1 million times a second, whereas a standard camera can only do up to 100 times a second. In low light, a standard camera cannot see anything, but an event camera can still see texture…allowing the drone to navigate without crashing. In September last year we did the first autonomous drone flight with an event cameraExternal link [see video below].

This is terrific not only for drone applications: you may remember that the first Tesla fatality that occurred two years ago was caused because a white truck passed the Tesla on a white background, so the camera could not tell the difference. Using these new types of cameras, this should not be a problem anymore. 

Curious about drones?

Check out our crash course on drone basics, from technology to regulation.

Still have questions?

Let us know in the comments below!

The making of an autonomous robot

The commercial drones that exist nowadays rely on GPS to navigate, which works only outdoors. When you want to have a drone navigating inside a building, for example to search for survivors, then search-and-rescue authorities rely on visual line-of-sight [VLOS], meaning you have to continuously track the drone with your eyes or use FPV [first-person-view] goggles. The problem with this is that VLOS does not work in a building with many rooms to explore, and after 50-100 metres you would lose the FPV connection. For all these reasons, there is a need for drones that can operate completely autonomously – beyond VLOS and any wireless communication.

In my lab, we are working on AI [artificial intelligence], which is a new way of programming robots. Typically, to programme a robot you have to tell it how to behave in basically any circumstance by writing thousands of lines of code.

In the last few years, artificial neural networks have been developed: instead of writing a set of instructions that the robot should follow, these networks consist of showing the robots how to solve a task by example. You train the robot, for instance, to recognise an opening by showing it thousands and thousands or even millions of images of openings. That means that the algorithm would be able to work with any sort of opening, and not just rectangular openings.

The patterned rug and boxes placed in this room at the University of Zurich help Scaramuzza and his team test out their drones’ ability to map their environment and avoid obstacles. swissinfo.ch

‘Home of Drones’

Switzerland is the country of drones. We have about 80 drone companies and 40 drone start-ups, and a very good ecosystem thanks to great support from the Swiss National Science Foundation, and [the Swiss innovation agency] Innosuisse. Their support of technology transfer has over the last ten years brought Switzerland to the forefront of drone research and industry.

Of course, there are regulatory barriers, but I would say that Switzerland is also at the forefront [of drone regulation] because it’s a small country, and we have a very good communication with the Swiss Federal Office of Aviation, and they’re very open to new applications. There are already two companies in Switzerland that are allowed to fly autonomously over people’s heads and beyond VLOS. One is MeteomaticsExternal link, which uses drones as probes for example to check the weather instead of sending balloons. The other is MatternetExternal link, which is doing last-mile delivery.

The biggest concern about drones from a public perception is privacy. As long as drones don’t store any data, it’s fine. But if you store data, either on board the drone or on a server, then you have to make sure you delete any sensitive data which includes faces, and license plate and house numbers.

To be honest, it seems to me that people are not really concerned about safety, although they should be. Sometimes I see people jogging, and pilots flying their drones very low but the joggers don’t seem to care. When you get too close to a drone that does not have a protective ring around the rotors – which is not allowed, by the way – you could actually get hurt, especially your eyes. You might eventually want to have parachutes or even airbags.

I was at the WEF Drone Innovator’s NetworkExternal link in June, and there was a presentation by Airbus that said by 2035, there would be 2,000 drones flying over our heads in Zurich. That means we would need a very accurate air traffic control system to avoid collisions. So, some infrastructure needs to be put in place.

The future of entertainment?

We are also working on autonomous drone races – we predict this could be a form of entertainment in the future. There is a community developing autonomous racing cars, so we are also developing autonomous racing drones in a project sponsored by Intel. It is about beating the best human pilot in a drone race.

It really tackles open robotic challenges, because if you are able to succeed at this very difficult task of having a drone fly super-fast around a course and beating a human pilot, then automatically you would be able solve search and rescue of people, especially in complex environments. It’s a motivator for us.

Aerial Futures: the Drone Frontier

Aerial Futures: the Drone FrontierExternal link is an ongoing event series created and curated by swissnex Boston, aimed at exploring the changes to policy and society that accompany the growing adoption and implementation of professional drone technology. swissinfo.ch publishes stories on the events and topics corresponding to the Aerial Futures series, in collaboration with swissnex Boston. You can follow the series on social media using the #DroneFrontierExternal link hashtag.