Vision Based Precision Landing

[reedchristiansen]

Hello,

I thought I would post an update of our ongoing effort to put fixed wing UAVs right where we want them.

Here is a video that shows a couple landing runs using vision only. This is especially usefull when GPS is not all that reliable. It is also more accurate then a GPS/laser solution.

Here is the link:
http://www.procerusuav.com/video/target_prosecution_vision-only.wmv


The user steers the UAV using left mouse clicks in the video screen. Once a landing spot has been identified, the user can either left click and hold on the desired landing location or if their is enough texture, the user can right click once to activate the auto tracker, and the uav will guide itself to the landing location.

This is very much a work in progress - the hard part is correcting for wind and camera misalignment which is what we are doing right now. It is one thing to shoot a couple videos on a no wind day, it is another to make it work all the time in most reasonable weather conditions.

Reed

[Dan_Jones]

Very, very interesting. What kind of processor or computing power is needed for this type of control? Do you use webcams or NTSC/PAL cameras? Good job so far!

[reedchristiansen]

Very, very interesting. What kind of processor or computing power is needed for this type of control? Do you use webcams or NTSC/PAL cameras? Good job so far!

Dan,

We use analog video cameras (BW AV) and do all processing on the ground. It takes quite a bit of CPU to do the stabilization and feature tracking. The loop is closed at about 10 hz with pitch and roll cmds being sent back to the airplane. We are also working on an onboard solution using and FPGA w/DSP.

Reed

[d_wheel]

Looks interesting but a better name might be "precision CRASHING". :cool:

Later;

D.W.

[reedchristiansen]

Looks interesting but a better name might be "precision CRASHING". :cool:

Later;

D.W.


The word disposable comes to mind when think of potential applications. You could consider it a terminal guidance algorithm. Landing in the traditional sense is pretty far down the list of uses.

Reed

[Unterhausen]

that's pretty cool. We would need a more disposable airframe

[Dan_Jones]

Landing in the traditional sense is pretty far down the list of uses.

Reed

If the airspeed is maintained by flaps or throttle, I would think that landing would be a possibility. The camera may need to be pointed downward to increase the airplanes angle of attack as the airspeed slows. I wouldn't want my airplane to be a missile and I don't have any disposable tents! :D

[wadiprawita]

Dear Reed,

Very impressive achievement.

What is the algorithm used for object tracking? Is it implemented onboard UAV (if yes using what processor), or onboard the ground station?

Thank's

-doni-

[rob10000]

related newspaper article.. Great stuff! http://www.deseretnews.com/dn/view/0,1249,660207484,00.html

[HELModels]

Wouldnt a "cargo net" make a better landing target?

[reedchristiansen]

Dear Reed,

Very impressive achievement.

What is the algorithm used for object tracking? Is it implemented onboard UAV (if yes using what processor), or onboard the ground station?

Thank's

-doni-


None of the vision algorithms are implemented on-board. The video is sent to the ground via a BWAV analog video transmitter/rx. It is captured using an uncompressing USB or firewire (low as latency as possible) video capture device. A reasonably powerfull laptop running windows is needed to avoid dropping frames - atleast 1.5ghz... The pixel coordinates from the feature tracker on the laptop are combined with wind estimate information and current uav roll and pitch angles to produce new roll and pitch commands which are sent back to the autopilot.

reliable wind information is critical for both pitch and roll control. If everthing is lined up, the target will appear as an unmoving object growing in size. the object will likely not be centered in the screen due to wind. If things are not lined up (misaligned camera, AOA changes, boundery layer effect on wind, inacurate wind estimation, gusting wind, ....etc) then the vision algorithms can do some correction during the last 5 or 6 seconds as the object is big enough and close enough to start "moving" in the video.

Their are lots of variables, if only the wind never blew, then things would be simpler....

That's probably allot more answer then you wanted...I must be venting :)
Reed

[wadiprawita]

Dear Reed,

I'm new on this "machine vision" things. Right now I'm pursuing graduate degree on UAV control. The basic waypoint following algorithm has been implemented and tested onboard our UAV. Right now I'm trying to implement some kind of on board machine vision. I'm using blackfin processor.

So, what kind of vision / tracking algorithm do you use, is it 2D pixel correlation or another algorithm (I assume it's another algorithm, because pixel correlation is sensitive to zooming / distance change and rotation).

Thank's

-doni-

[Tuner]

What API or Vision Software do you use. I am familiar with almost all the packages. im real curious with what you settled with.
Have you ever consider the CMU Cam for maybe doing some pre processing or even doing all the work.

I have found great difficulty with a lot of the machine vision software when it comes down to low latency. I have been looking more into the ACE 16k and implementing a Visual Flow type method this obviously has major limitation. A great company that yo might want to check out is Analogic Computers.

[wadiprawita]

I'm still in early state in machine vision research. Right now I'm still familiarizing with Blackfin Processor (along with my other activity in basic AHRS and autopilot development). I haven't found any Vision Library that have been ported to Blackfin other than Blackfin Multimedia SDK, so I think I will stick with this. I will not use CMU cam, because I think It's very limited for aerial application. I will use video decoder (like frame grabber) along with the Blackfin.

I haven't found the algorithm yet, so I posted in this forum.

Thank's

-doni-