Terrain Matching Systems on UAV's [Archive]

View Full Version : Discussion Terrain Matching Systems on UAV's

vector_vortex

Jan 06, 2007, 04:38 AM

Hi,
I was thinking about supplements to an INU system to reduce error. With the increasing availability of high-ish resolution satellite imagery has anyone experimented with terrain matching to determine position and attitude. Of course this is a large amount of processing to be performed on-board, but maybe someone has experimented with ground based processing. If anyone has done anything like this I'd love to know how you approached it and what results were found.

Thanks

Joel

Blue Sky

Jan 15, 2007, 02:56 AM

What you want is called SLAM. Simultaneous Localization And Mapping.
You can google up a lot of links for it. It's used alot in robotics. I believe it was Rodney
Brooks who said "The world is it's own best map." I don't know much about it and I've had little free time lately so I can't be of much help except to point you in the right direction. This is a good link to get you started. http://www.google.com/search?hl=en&safe=active&client=opera&rls=en&hs=dlg&sa=X&oi=spell&resnum=0&ct=result&cd=1&q=2d+simultaneous+localization+and+mapping+for+mic ro+air+vehicles&spell=1

-Dave

Unterhausen

Jan 15, 2007, 09:24 AM

I've worked on a couple of systems that did this. I'm assuming you'd use cameras to do this, and I'm not sure how well it would work with limited resources. Interesting problem though.

Wulffy

Jan 27, 2007, 01:23 PM

Take a peek at SmartCam3d (http://www.landform.com). It does 'hybrid' synthetic vision, overlaying NTSC video onto synthetic vision (CG 3d Terrain Depictions). In my reading about the app, I understand that it needs airframe position (lat/long/alt), flight parameters (pitch/roll/heading/ias/etc.), the video, and camera displacement angles. Once it has all of those, it can function as depicted here (http://www.youtube.com/watch?v=bC-D0XFvszg). Manual is downloadable here (http://www.smartcam3d.com/documents/SmartCam3DManual.pdf).

I have been trying to see if it is available to the general public, and to see what pricing exists. Unfortunately, I am not able to make any of these determinations without resorting to writing to them, and frankly, I am having too much fun getting back into the whole RC/RPV/UAV scene, that I have been away from for so long, that I haven't taken the time yet to inquire...

clolson

Jan 27, 2007, 03:28 PM

I've posted this link before, but if someone is looking for a more "roll your own" low cost system, you might take a look at FlightGear. I've rigged up some "glue" code to drive FlightGear in real time from simple nmea gps sentences, a Microbotics MIDG-II, and a Xbow MNAV. The data can be fed to the ground station in real time via a radio modem (for example) and fed straight into FlightGear.

The MIDG-II is a really good sensor and yields pretty good correlation between the synthetic (FlightGear) view and the live on board camera view. In the movies linked below, the camera is angled 45 degrees down.

http://baron.flightgear.org/~curt/UAV/Rascal110_2/

You have to scroll down to the December 12, 2005 link to download two movies showing the synthetic view synced with the live camera view (and you may have to download the divx 6 codec if you don't already have it ... details also at the above link.)

(Note that we are in minnesota where we don't have much for terrain, but FlightGear has pretty accurate terrain based on SRTM-2 data so this should work even better in non-flat areas.)

Curt.

Wulffy

Feb 03, 2007, 05:17 PM

clolson

Feb 03, 2007, 06:48 PM

Hey Curt. I am interested in learning more about what you have done there. It is definately of interest. Can you please provide additional information? TIA.

Hi Wulffy,

The information I have posted is here:

http://baron.flightgear.org/~curt/UAV/Rascal110_1/

And here:

http://baron.flightgear.org/~curt/UAV/Rascal110_2/

And here:

http://baron.flightgear.org/~curt/UAV/MicroGear

Beyond that, I'll do my best to answer any specific questions ...

Curt.

Wulffy

Feb 12, 2007, 11:54 PM

Hi Wulffy,

The information I have posted is here:

http://baron.flightgear.org/~curt/UAV/Rascal110_1/

And here:

http://baron.flightgear.org/~curt/UAV/Rascal110_2/

And here:

http://baron.flightgear.org/~curt/UAV/MicroGear

Beyond that, I'll do my best to answer any specific questions ...

Curt.

Curt,

I have had a chance to spend some time looking at the majority of the information on your site.

You have a nice fleet of aircraft, a wonderful cabin, a beautiful dog that is about 10 times the size of my fox terrier (and seemingly equally as entertaining and rambunctious), and a great looking family - your girls are so cute - you and your bride have done extremely well! The pics out in the Mojave bring back some memories - I was out there for a few weeks during the summer of 2000, doing a first-of-type non-group RVSM modification in an IA 1124 WestWind II. Man, does it ever get flipping hot out there - being bent upside down under an instrument panel in the cockpit of a beer-can cooking out on the ramp was not my idea of enjoying the desert... (It was my first exposure to the evaporative style of 'air conditioners' running in the hanger. I was surprised at how efficient they really work!)

I have been looking to build an RPV for over 20 years (I have dug out some old correspondence that I drafted back in the mid 80's to some NASEU representatives, trying to convince them that I was not off of my rocker for wanting to build an RPV/GCS combo, and soliciting their assistance). I allowed life to always get in the way, and would still be doing so if I wasn't getting cranky and selfish in my maturing years. I am nearing 40 (yeah, I know, not THAT old... :-), and I am darn tired of waiting to fulfill my (adult)life-long dream...

With my recent professional activities being steered toward EVS (Infrared Camera Systems on Part 23/25 aircraft) and SVS (Synthetic Vision now being installed on a semi-regular basis (I have installed the Chelton EFIS - nice package with CrossBow AHRS and FreeFlight WAAS GPS)), I am finding that I need to dust off the dreams and pursue them.

Anyways, back to the subject matter; I see where you make mention of a couple of different sensor packs and radios. If I am not mistaken the pair of combos were:

The MIDG II with AeroComm Radios
The MNav/Stargate combo with MaxStream Radios

I put a query into the folks at Microbotics, Inc. Holy chevys, their hardware sure isn't cheap. I was quoted a price of $6,750.00+ for the MIDG II. I have a request into Crossbow for pricing on the MNav/Stargate combo. I have been watching what has been going on with the WPS and anxiously awaiting the WPS2 threads here at RCGroups, not to mention the Altitude hold activity of recent. I came across a site that is informative for those looking to get pricing/ideas on IMUs: http://damien.douxchamps.net/research/imu.php. Damien is also a photographer with a nice eye... It is amazing how small the technology is getting, not to mention how expensive it still is. Take a peek at http://www.memsense.com/products/mag3.asp. What the folks have going over on the Paparazzi site is encouraging as well with the active tracking antenna and the IMU solution that appears to be interfaced with FlightGear on their update articles.

Regarding your experiences, which combination of sensor/radios do you favor (foregoing the issue with the MNav - yes, I read the thread on sourceforge... :-)?

Regarding the Home-Grown code for getting FlightGear to respond to external inputs, I would be interested in seeing what you have available. I may be getting a couple of things mixed up, but I recall a 9Feb07 update where you are making reference to an application called MicroGear. Is this a spin-off of FlightGear customized to be the interface and display for controlling the RPV?

Anyways, thanks for taking the time. Keep updating us (via your website and posts herein) as things progress for you and your team. We are anxiously watching and waiting...

clolson

Feb 13, 2007, 11:59 AM

Curt,
The pics out in the Mojave bring back some memories - I was out there for a few weeks during the summer of 2000, doing a first-of-type non-group RVSM modification in an IA 1124 WestWind II. Man, does it ever get flipping hot out there - being bent upside down under an instrument panel in the cockpit of a beer-can cooking out on the ramp was not my idea of enjoying the desert... (It was my first exposure to the evaporative style of 'air conditioners' running in the hanger. I was surprised at how efficient they really work!)

I have been looking to build an RPV for over 20 years (I have dug out some old correspondence that I drafted back in the mid 80's to some NASEU representatives, trying to convince them that I was not off of my rocker for wanting to build an RPV/GCS combo, and soliciting their assistance). I allowed life to always get in the way, and would still be doing so if I wasn't getting cranky and selfish in my maturing years. I am nearing 40 (yeah, I know, not THAT old... :-), and I am darn tired of waiting to fulfill my (adult)life-long dream...

With my recent professional activities being steered toward EVS (Infrared Camera Systems on Part 23/25 aircraft) and SVS (Synthetic Vision now being installed on a semi-regular basis (I have installed the Chelton EFIS - nice package with CrossBow AHRS and FreeFlight WAAS GPS)), I am finding that I need to dust off the dreams and pursue them.

Anyways, back to the subject matter; I see where you make mention of a couple of different sensor packs and radios. If I am not mistaken the pair of combos were:

The MIDG II with AeroComm Radios
The MNav/Stargate combo with MaxStream Radios

I put a query into the folks at Microbotics, Inc. Holy chevys, their hardware sure isn't cheap. I was quoted a price of $6,750.00+ for the MIDG II. I have a request into Crossbow for pricing on the MNav/Stargate combo. I have been watching what has been going on with the WPS and anxiously awaiting the WPS2 threads here at RCGroups, not to mention the Altitude hold activity of recent. I came across a site that is informative for those looking to get pricing/ideas on IMUs: http://damien.douxchamps.net/research/imu.php. Damien is also a photographer with a nice eye... It is amazing how small the technology is getting, not to mention how expensive it still is. Take a peek at http://www.memsense.com/products/mag3.asp. What the folks have going over on the Paparazzi site is encouraging as well with the active tracking antenna and the IMU solution that appears to be interfaced with FlightGear on their update articles.

Regarding your experiences, which combination of sensor/radios do you favor (foregoing the issue with the MNav - yes, I read the thread on sourceforge... :-)?

Regarding the Home-Grown code for getting FlightGear to respond to external inputs, I would be interested in seeing what you have available. I may be getting a couple of things mixed up, but I recall a 9Feb07 update where you are making reference to an application called MicroGear. Is this a spin-off of FlightGear customized to be the interface and display for controlling the RPV?

Anyways, thanks for taking the time. Keep updating us (via your website and posts herein) as things progress for you and your team. We are anxiously watching and waiting...

Thanks for your kind words! Yeah, the first time I was out in Mojave it was flipping hot! Hilariously, they were shooting the end to a Jodie Foster movie that weekend up there (Flight Plan). They had fake snow, plowed tarmac, the cars were rigged to have vapor coming out of the exhaust (like it was -20F) and they even had a fake manhole cover with vapor coming out (like NY city in the winter.) We got to go right up to the edge of the set and watch the action. It was hilarious how convincing their cold weather and fake snow and fake manhole covers were even though it was summer in Mojave ... I was ready to put on my coat, hat, and gloves!

The MIDG is expensive, but it is very good. If you need a high quality attitude estimate, they are tough to beat.

The XBow MNAV has very similar sensor hardware, but their open-source code leaves *much* to be desired. They don't handle things like prolonged constant rate turns where the sensed gravity vector is not down. They don't have a good yaw angle estimate. There's is only based on magnitometer sensing and filtered with a bit of gyro data. So you better have your magnitometer calibrated perfectly if you want to use the ahrs heading estimate, otherwise you better stick to gps ground track and live with that.

We've seen other issues with the software too that I've posted elsewhere, so I won't keep beating that horse.

For just the MNAV, I think Xbow is quoting prices in the $1500-$1750 range per unit. The recommended StarGate is probably about $500. However, a nice gumstix will run you $150-200 depending on which variant you buy and I'm finding that the gumstix works every bit as well (and even has some advantages over the stargate.)

We haven't done detailed testing, but our results (in terms of range and data throughput) have been pretty comperable between the Aerocomm's and the Maxstreams.

Right now I'm leaning towards the MNAV + gumstix + maxstream as my favorite balance between capability and cost. Of course, the MNAV is a huge do-it-yourself project, it's not a plug-n-play autopilot. But it can get there, especially with the software work that I'm in the middle of.

I kind of feel like I'm blazing a new trail here with the MNAV and some of the work we are doing. Not many have an MNAV up and running as part of a working autopilot. But I think once the various issues are press through and engineered around, and the software work is complete, this will become an extremely attractive package.

I think you will be able to put together a very sophisticated and very flexible and open system for maybe in the $2500 range for hardware. Software would be open-source. That doesn't count the ground station or airframe ... just the onboard bits that would turn a run of the mill R/C airplane into a fairly sophisticated uav.

The really exciting thing (for me) about the gumstix/stargate flight computer and the open-source software, is that you aren't necessarily limited to a simple altitude/speed hold + limited waypoint follower.

You could start to build in more sophistication ... like each waypoint (or maybe I should say route segment) has it's own unique altitude and speed. You could pretty much throw away the limits on number of way points. You could even start thinking about doing more sophisticated or adaptive routing.

I'm really getting exited about this combination of hardware, but at the moment the MN weather is kicking me in the butt! Sure, we are hearty folk and can get our airplanes up in the air in brutal temps, but when you are testing new hardware out for the first time and may need to do at-the-field surgery, +20F is about my threshold of pain. :-)

You brought up MicroGear which I mentioned on my web page. This is what I'm calling (initially) my flight computer software package. It derives from the basic MNAV open-source software, but has been substantially updated and rearchitectured. I've left the kalman filtering stuff as untouched as possible (since I don't understand that stuff as well) but just about everything else is completely different.

This software will run on the stargate (or gumstix), it will read and integrate the sensor data from the MNAV to produce an attitude and location estimate. It will run the autopilot modes. And it will communication with the ground station (right now I'm leveraging FlightGear and related tools for that.)

It's all a work in progress (as are most of the projects mentioned here) but I'm still having fun and I still have high hopes! A few of my initial naively high hopes have been dashed along the way, but life and reality has a way of recalibrating your focus and your goals.

Still, I think the MNAV + gumstix + reasonable air-ground communcation link + FlightGear will be about the coolest thing to hit the low-mid range uav world since sliced bread. :-) I have no plans of trying to upstage things like the globalhawk or the pathfinder project that could fly at obscene altitudes for days on solar power alone ...

Curt.

Wulffy

Feb 17, 2007, 04:41 PM

...The really exciting thing (for me) about the gumstix/stargate flight computer and the open-source software, is that you aren't necessarily limited to a simple altitude/speed hold + limited waypoint follower.

You could start to build in more sophistication ... like each waypoint (or maybe I should say route segment) has it's own unique altitude and speed. You could pretty much throw away the limits on number of way points. You could even start thinking about doing more sophisticated or adaptive routing.
...
This software will run on the stargate (or gumstix), it will read and integrate the sensor data from the MNAV to produce an attitude and location estimate. It will run the autopilot modes. And it will communication with the ground station (right now I'm leveraging FlightGear and related tools for that.)
...
Still, I think the MNAV + gumstix + reasonable air-ground communication link + FlightGear will be about the coolest thing to hit the low-mid range UAV world since sliced bread. :-) I have no plans of trying to upstage things like the globalhawk or the pathfinder project that could fly at obscene altitudes for days on solar power alone ...

Curt.

Thanks for the reply...

Regarding full-featured-navigation, once the platform has matured, that is exactly what I am excited about. Being in the Avionics (for Piloted-Airframes) industry, and being schooled on Flight Management Systems for these airframes, I have some, admittedly probably-too-aggressive, expectations as to what a nav system onboard our airframes should be able to do...

However, this thread, along with the Catalina Is and WPS(2) threads, is exciting as it seems that what I am envisioning in my noggin is achievable with some time and diligent efforts...

From the Catalina thread:

...a plug for FlightGear (www.flightgear.org (http://www.flightgear.org/)) here. (When you have a hammer, every problem looks like a nail, right?) :-)
... Then you can pipe your telemetry information into FlightGear (disabling it's built in flight dynamics) and connect FlightGear up to one of the available multiplayer servers...
...There would be some effort for a person to get up to speed with all this, but there are at least basic examples available for all of this if anyone wants to fiddle around with any of this...
I am interested in learning more about the details of accomplishing what you have described herein. Does one have to recompile the app, or is it something that can be facilitated via the GUI? Earlier, you made mention of this as well, and was what I was inquiring about when I asked for more details (I realize I did not detail what I was 'looking' for ).

Also you made mention of an interpolation utility that you used to take GPS data and generate linear pseudo-pitch/roll/yaw approximations. Is this also something that is available? (Trust me, I fully understand that one is NOT to attempt to predicate navigation on the generated displays ... ).

I am interested in this so that I can begin to get up to speed with using FlightGear as a possible man-machine interface for my models .

Thank you.

-t

p.s. Just got my EM-411 talking with my laptop (I just bought the USB Eval board from SparkFun. After a couple of erroneous shipments, I finally received what I ordered) - WOO HOO - I am going to go around driving with the thing to see if GE will work with the cached images when connectivity is lost. Sorry for being off topic, just a bit geeked!

clolson

Feb 19, 2007, 04:11 PM

I am interested in learning more about the details of accomplishing what you have described herein. Does one have to recompile the app, or is it something that can be facilitated via the GUI? Earlier, you made mention of this as well, and was what I was inquiring about when I asked for more details (I realize I did not detail what I was 'looking' for ).

From a high level perspective, FlightGear is a 3d application that can draw a 3d representation of the world from any position and orientation. The world can contain user created models. FlightGear by default has a built in flight dynamics system to drive the view point and can model a variety of aviation systems and a variety of other things surrounding the world of flight (weather, navigation systems, other players in the virtual world, etc.)

FlightGear has the ability without recompiling to turn off it's built in flight dynamics and let it's view point be driven from an external data source.

You may need to put in some effort to take your available data and format it into the appropriate FlightGear data structure and send it over, but you don't need to make any changes to flightgear itself.

Also you made mention of an interpolation utility that you used to take GPS data and generate linear pseudo-pitch/roll/yaw approximations. Is this also something that is available? (Trust me, I fully understand that one is NOT to attempt to predicate navigation on the generated displays ... ).

What I created was an ugly hack. The interpolation was not done in real time (but was done later by loading in the entire track file and replaying it. It was that interpolation code that attempted to come up with plausible pitch/roll angles to coincide with the recorded motion, but that was crude and didn't take into consideration any of the aerodynamic properties or limitations of the airframe. Still, it was a lot better than nothing.

I am interested in this so that I can begin to get up to speed with using FlightGear as a possible man-machine interface for my models .

Well, FlightGear isn't going any place, so it's there to download and try out whenever you are ready to go. :-)

Regards,

Curt.

Wulffy

Feb 19, 2007, 05:31 PM

...You may need to put in some effort to take your available data and format it into the appropriate FlightGear data structure and send it over, but you don't need to make any changes to flightgear itself...Kewl! Can you please point me towards where the data structure is defined, and also how to go about directing the app to listen to external data? TIA.

...What I created was an ugly hack. The interpolation was not done in real time (but was done later by loading in the entire track file and replaying it. It was that interpolation code that attempted to come up with plausible pitch/roll angles to coincide with the recorded motion, but that was crude and didn't take into consideration any of the aerodynamic properties or limitations of the airframe. Still, it was a lot better than nothing...Yeppers, I derived that from the dialogue you posted. Can you make the source available for this little tool? I already have some placebo NMEA sequences, as well as real-world sequences that I'd be interested in testing with, just to get the hang of things.

Well, FlightGear isn't going any place, so it's there to download and try out whenever you are ready to go. :-)

Regards,

Curt.I already have it installed and have actually used it when I got my Saitek hardware (X52 Joystick, Throttle, and Rudder Pedals :D ) for Xmas.

clolson

Feb 19, 2007, 06:01 PM

Kewl! Can you please point me towards where the data structure is defined, and also how to go about directing the app to listen to external data? TIA.

If you can find your way to the source code, take a look at .../src/Network/net-fdm.hxx

That defines a structure, and that particular source code file is "public domain" which means you can do anything you want with it ... even include it in your own proprietary application to make it faster to build a communication link to FlightGear.

Yeppers, I derived that from the dialogue you posted. Can you make the source available for this little tool? I already have some placebo NMEA sequences, as well as real-world sequences that I'd be interested in testing with, just to get the hang of things.

This is already part of the source code ... look in utils/GPSsmooth/

Regards,

Curt.

Wulffy

Feb 19, 2007, 06:37 PM

Thanks Curt, I will give it a go! -t

msomby

Oct 30, 2008, 09:39 AM