Iris Automation operates a CubCrafters XCub as a target during its Casia flight tests, fitted with tundra tires so it can land on the Nevada desert
Iris Automation’s Casia detect and avoid computer vision system is a leading candidate for equipping drones to avoid other aircraft. The system, which uses off-the-shelf cameras, has been extensively flight tested during the last three years.
In the USA the FAA allows waivers and exceptions for drones to operate beyond visual line of sight (BVLOS) under Part 107 and Part 135, but the rules and regulations haven’t been written yet by the FAA for more routine operations. Just what a detect and avoid (D&A) system will have to do comply to serve in a BVLOS role in the future is yet to be determined precisely. Part 135 certification is currently the only way for small drones to carry the property of another for compensation beyond visual line of sight.
But Iris Automation has gathered test and operational data in staggering quantities. The company has tested its Casia system on drones flying at 400 ft and below over the Nevada desert encountering fixed wing aircraft and helicopters, including the company’s CubCrafters XCub. The encounters include having Casia initiate avoidance maneuvers, which often result in the drone spiraling down out of the way of the aircraft. Casia can distinguish between a light aircraft, an airliner, a helicopter or a military jet and records the speed of the target to work out the drone’s avoidance maneuver.
A single camera system on the XCub has been used to evaluate how this could detect and identify other general aviation aircraft
Inspection duties
Casia’s cameras enable the system to detect and avoid aircraft from 0.75 miles (1.2km) away on average and up to 1.2 miles (1.9km). In three years of testing over the desert within 30 to 100 miles (50 to 160 km) of Reno, Nevada, Iris engineers have recorded 16,000 encounters between Casia-equipped drones and aircraft.
The company focuses on drones used in industrial inspection applications rather than small package delivery drones. Any drone that has a maximum gross takeoff weight of less than 30 lbs is not going to be able to fly long enough to engage in BVLOS, believes John Damush, CEO of Iris Automation.
Acoustic sensing and onboard radar are also candidates for D&A on drones. However, seeing and avoiding is what regulators have written into current regulations. Damush also believes an optical system provides a nice trade off in terms of size, weight and available power and cost.
The tests over the Nevada desert occur daily and often include 50 encounters in a single morning of operations between a drone and an aircraft like the XCub, according to Iris’ marketing director Aaron Rudger. The flight tests collect data that is used to evaluate the performance of the computer vision system and ensure it is identifying the approaching aircraft at sufficient distance and the type of aircraft.
“Cameras lend themselves towards remote operations and as a proxy for human eyeballs looking out the window,” says Damush.
“The difference in our approach is that we’re not trying to provide a remote pilot view, but rather a sensing system that detects aircraft that could be a collision risk, and then alerts an onboard system to take an automatic maneuver to avoid that potential collision.”
Casia is focused on dealing with non-cooperative targets, for example a crop duster popping up in front of a drone flying at low level near a farmer’s field. The system can be configured with anywhere from one forward facing camera or six if a safety regulator decides to require 360˚ coverage.
Computing power
The cameras feed data into the Casia central computing module which runs computer vision and machine learning algorithms to detect an aircraft, classify it, determine its speed, distance and bearing from the drone. It is important to be able to discern if it is an airliner or a crop duster because the performance is different. If Casia mistakes a Boeing 737 for a general aviation (GA) aircraft the system is going to think that the aircraft is very close. Closure rate and direction determine what the Casia system does in terms of commanding an avoidance maneuver. The GA pilot is also instructed in the flight tests to maneuver to avoid the drone.
All encounters between a drone and an aircraft are flown with some altitude separation and none are head-on, which is considered too dangerous for an actual flight test. Head-on encounters are evaluated with simulation. Iris does ten times as many encounters in simulation as in actual flights and uses the flight test data to validate the simulations.
“We record all of the video that the system is processing. After the flight we download it into our computer system that continuously monitors the algorithms that are running machine learning,” says Rudger. “It means the Casia system is getting better all the time at identifying things.”
The drone encounters are always being flown against different types of aircraft and against different types of backgrounds. In addition about 20 customers are using a total of 50 Casia systems on drones.
“Our customers using Casia in the field are saying – ‘I was using it and all of a sudden, a crop duster flew over next to us where we were operating our drone’,” Rudger says. “Our system flashes a warning to the remote pilot that there’s a detection. And then when the drone executes an avoidance maneuver, it also notifies the remote pilot. Most of the time the customer didn’t even know there was an aircraft out there.”
Environmental calculations
The horizon in the background where the drone is operating is a very critical feature used by Casia in determining the distance, speed and bearing of an intruder aircraft. The system has to accommodate things like the angle of the sun and whether it is low or high in the sky.
“We have a computer simulator we created in house that has incredibly high fidelity in reproducing all of these different environmental dynamics,” Rudger says.
Iris did a series of flights with an Applied Aeronautics Albatross drone during 2019 in Kansas, flying in a powerline inspection scenario to demonstrate Casia system performance. The 22 lbs, long-range drone has an endurance of four hours and a range of 155 miles (250km). The work with Kansas State University and Evergy utility company was part of the FAAs Integration Pilot Program. Iris is still part of this FAA program, which is now called Beyond and is focusing on BVLOS development. There were no intentional intruder scenarios during the Kansas flights.
Iris-run test flights using an Albatross equipped with Casia and a GA aircraft in Nevada show it is very difficult for a pilot to see a drone. Iris has experimented with a Casia system on its XCub but does not intend to develop a product for use on aircraft yet. The drone avoiding the aircraft can also initiate maneuvers more effectively and spot the aircraft on camera from much farther away. The GA aircraft therefore has the right of way.
“The FAA has given us guidance that their expectation is that detect and avoid capability will require a full 360˚ field of view,” Rudger says. This requirement may depend on the level of traffic in certain airspace. It takes five cameras to give Casia an all-around view. A single camera Casia system for a drone costs about US$9,000 while a multi camera system will be more expensive.
“The single camera system provides an 80˚ field of view,” Rudger says. “Most customers are not adopting the 360˚ field of view until they get a better sense of what the FAA is going to require for BVLOS. Meanwhile the single camera system improves safety and enables customers to still have a useful payload for things like inspection and data collection.”
Casia is not designed to spot air vehicles as small as the 22 lbs Albatross, but some customers have been doing flight tests to see if the system can detect drones. “We are seeing data from customers that the system does perform,” says Rudger. “It does not perform as well, because that is not the way it was designed. It detects drones, but the classification library hasn’t been designed to include drones.”
Integration and application
Censys Technologies Corporation of Daytona Beach, Florida, makes the 18 lbs Sentaero BVLOS drone, which can fly for more than an hour at a range of 55 miles (88 km). It is a fixed-wing, tilt-rotor drone that can carry a 3 lbs (1.3kg) sensor package. The company can also field a mobile command center when minimizing data turn-around is essential and customers need to get a greater output from their field teams. The company handles all of the regulatory interaction needed with the FAA to conduct BVLOS operations.
Censys manufacturers the drone and integrates the Casia D&A system on it but does not offer flight services. The Sentaero drone can cost anywhere from US$30,000 to US$130,000 depending on payload, according to Kyle Miller, director of business development at Censys.
A Censys Sentaero drone before flight
Modular payloads that can be swapped out include a wide variety of things such as LIDAR or infrared, thermal or very high megapixel cameras. Cameras can be mounted on a gimbal or fixed in the fuselage.
“Our air vehicle is only as valuable as the sensor package we carry and how far we can carry it,” says Miller.
Sentaero can inspect 50 miles (80km) of pipelines during a day of BVLOS operations with remote pilots moving frequently to new locations. “Long linear inspection is where BVLOS shines,” Miller says. Electric utilities, oil and gas, engineering and surveying and mapping companies and small drone service providers are using the drone. “We are not a half million dollar drone system, so we have customers that range from four person companies to 40,000 person companies,” he says.
Currently Censys has few competitors for a drone that can fly BVLOS. Anyone who wants to do it has to build a safety case with enough reliability, redundancy and the necessary equipment to obtain FAA approval. Miller says the main competitor now is manned aviation, but that will change as more drones are approved for BVLOS.
A major advantage of a forward-facing D&A camera is that unlike a human pilot it doesn’t get tired, it doesn’t blink and it is much more accurate, according to Miller. He confirms that often Casia is executing an avoidance maneuver before the operator of a Sentaero drone flying a BVLOS mission even knows there’s another aircraft in its airspace.
The drone will descend to the left or right to a pre-set safe altitude and loiters to avoid an aircraft. Since operations are conducted at 400ft and below, it cannot descend very far without hitting the ground. Casia has a high rate of detection and no false positives, Miller reports, especially with the latest software.
Censys flight tests its BVLOS drone near Daytona Beach International Airport, Florida where there are several flight schools and Embry Riddle Aeronautical University’s fleet of training aircraft. Every new drone manufactured completes 20 test flights before delivery to a customer. The test drones make avoidance maneuvers every day in this high volume traffic area.
“Thankfully, we have never had a close call, but that’s because of the Casia hardware on board,” Miller says.
Censys drones have had thousands of encounters with aircraft using the Casia system and all of this data is shipped to Iris Automation for analysis. Iris shares drone / aircraft encounter data with the FAA Beyond program and industry and universities involved with drones. However the startup company hasn’t had the time to publish any technical papers on the anti-collision lessons learned from analyzing this data.