Q&A: The future of ‘sense and avoid’ drones

Sense and avoid drones are one of 10 emerging technologies for 2015 highlighted by the World Economic Forum’s Meta-Council on Emerging Technologies.

Once unmanned aerial vehicles – better known as drones – are reliably able to avoid collisions with humans, objects and each other, they could become an everyday sight in our skies, performing a multitude of tasks.

We talked to Red Whittaker, research professor of robotics at Carnegie Mellon University, about what is driving progress in drones and how we can expect to see them being used.

How will future drones with ‘sense and avoid’ capability differ from current unmanned air vehicles?

Future drones will sense, think and act to avoid collisions and safely achieve their intended purposes. Today’s drones cannot sense or think in that way. Current drones can’t react to others or interact with their surroundings. They can’t respond to what they cannot see. Some current drones can fly on their own, but even the best of them fly blindly relative to the high standard that will be needed in the future.

What does it take for drones to be able to fly themselves?

They need to be able to understand and respond to an unpredictable local environment. Essentially, they need to emulate the way that bees, birds and fish do a great job of avoiding collisions even when they’re travelling together in close proximity, by changing altitude or trajectory.

And what are the breakthroughs in technology that are enabling this to become a reality?

We’ve seen a convergence in the development of a whole range of necessary areas. One particularly important thrust is the speeding up of the “sense-think-act” cycle through special purpose software and hardware that enable massively more parallel processing. A self-driving car might need to go through the sense-think-act loop only, say, ten times a second to appear skilled. But to get stable, agile flying in close quarters that must be closer to 100 or 1000 times a second. We’ve seen big improvements over the last few years in optimizing and accelerating that control loop.

We’ve also seen remarkable progress in sensor technology. New depth cameras can sense depth, range and geometry – not just pictures. That is invaluable for judging where it is safe and unsafe to fly, and how to avoid collisions. Readers may be aware that projectors are now so advanced that you’re starting to see them in smartphones. With a projector to project checkerboard line patterns into the world, and a camera to observe them, you can model the world within your field of view – and do this with sufficient range, accuracy, and speed in a variety of light conditions. This is much improved beyond technology that was possible just a few years ago.

Another example: inertial sensors are becoming much better, smaller, cheaper and more robust. These are central and critical to a drone’s balance and control. Inertial sensors act like a human’s inner ear to sense motion and verticality. These sensors used to be big, cumbersome, expensive mechanical gyroscopes with motorized moving parts. They are now economical, accurate, miniature devices with no moving parts.

These are only a few examples. The same story of incremental but cumulatively transformational progress applies everywhere from propeller technology to materials, processing algorithms, sensors, communications – all of the elemental building blocks of being able to fly from A to B while staying out of trouble.

What are the uses to which drones are currently being put?

The biggest commercial use now is probably in agriculture, monitoring crops to make sure inputs like water and pesticides are used as efficiently as possible. Drones are also transforming the surveying industry, including construction, inspections for real estate sales and evaluation of infrastructure. They make it much easier to survey trees, to check the health of forests. Traffic monitoring and control is another under-appreciated area of use.

There are some stunning examples of videography from drones, in the shape of things like city tours. Earlier this year CNN used a drone for direct broadcast of the news for the first time, in the commemorations of the 50^th anniversary of the Selma civil rights march. And of course there are also monitoring uses for security purposes. As the adage goes, information is power; and drones are a powerful tool for generating information.

But the big value proposition for drones is when they are able to move from merely observing things to actually doing things.

What could become possible?

Potentially, a lot of things we now send out crews of people to do – lifting and carrying things, delivering things, climbing poles, stringing wires, search and rescue, trimming trees. Think of drones as being robots that can fly and hover in the air, and think of the advances in the pipeline in next-generation robotics, and you get a sense of the possibilities.

The technology is a long way from maturity. I would say we are now at something like 2% of what’s destined to become possible, once we can overcome the hurdles of safety and reliability that would open the door to allowing drones to share the airspace.

What are the prospects for reaching an acceptable level of safety and reliability?

There will always be glitches, even with mature and routine technologies – that’s a statistical inevitability. Even human-piloted passenger airplanes are not immune from hardware, software, human error or maliciousness. Sense and avoid drones will be no different.

As drones become used for a wider range of purposes, there will always be a need to balance the benefits with the risks – for example, we may initially want to allow certain uses in good conditions but not in darkness or adverse weather. These discussions never stop with any technology, and drones will be no exception.

Author: Red Whittaker, research professor of robotics at Carnegie Mellon University

Image: A camera drone flies in New York City, March 12, 2014. REUTERS/Mike Segar