Photogrammerty solution...

[panomaniac]

Hi guys. I'm looking appropriate solution for Hi-res vertical shots for photogrammerty. My intention is to build UAV that would be capable to fly at given altitude in grid pattern countioniously taking vertical pictures. The platform I plan to use is senior TM. I have read many posts here about various autopilots at the market, but I'm not shure wich one I need for this project and haven't seen anyone who is doing it. I have intended to use it this way: Manually fly plane to x location and x altitude, flip a switch, and plane flies parralel lines pattern, crusing at fixed altitude and activate shutter servo for a givven sec or even gorund distance. What kind of autopilot would I need to be capable of that? Also Any other suggestions are wellcome....

[}{ell}{unter]

Hi,
I think a plane is not the best solution for photogrammetry.
Problem is that you need 60%+ overlap to create stereo models. Depending on how high, the plane's speed, what fixed lens you use, etc., you should have a photo every 0.5 seconds or less.
Another problem is the shutter speed of the camera. When moving fast you should have a shutter speed of 1/3000 of a sec. or even faster to reduce blur on the photos in order to post process them to orthophotos.

Something for the near future is to film you traject and post process it with frame stiching to create 1 big photo of the area. I know some guys who are working on this in Ghent University in Belgium.

A better platform is a heli I think. You can reduce the blur much better because you can hover over every point...

[Ceros]

Which auto pilot to use - any and none is the answer, because none of them wil do the job sufficently accurate - unless you are prepared to spend some time modifying them, or adding to their standard ability to maintain a flight path - which as Hell Hunter said, will mean you are going to have to keep a steady overlap - or spend a lot of time stitching frames together.

As far as the actual picture goes - well, you are going to need a decent high res digital still camera (Imperx makes good 8, 10, 12 and 16Meg units) - the amount of megs' per frame a function of the altitude you intend to shoot at.

Now add to this a decent frame grabber to dump high res pic's to solid state onboard memory very fast indeed at 0.5 secs is it?.

A 10Meg digital still camera will set you back a $1000, a frame grabber capable of this will be another 1000.

Now you need an SBC in PC104 or PCI formatt, with a backplane, power supply (to control everything) - and things quickly get heavy if you want to do this properly - which means a decent sized UAV.

After all said and done - your autopilot is going to be the easy part of the project.

It can be done - but do not underestimate what it wil cost you in terms of hardware to do it properly. You'll be lucky to come away with much change from $10 000, and more more likely 15 - 20 if you are wanting to get decent quality pictures to work with.

Still interested - start off by taken a look at the electronics that John Deere uses for its ground based DGPS signal generation - incorporate that into your UAV (just bin the heavy steering wheel motor they use and replace it with something much lighter for the rudder) and you'll have a DGPS source for your UAV that will give you a real world 10cm flight path accuracy!

..... yup, I see lots of work here.

[mlbco]

My company builds UAVs that already do this mission. We developed our own software to stitch the images. The aircraft is fully autonomous and can fly for 5 hours at 40 mph. We normally take photos at 1000 ft AGL using a camera that is automatically triggered by the flight computer for 50% overlap. The attached images show an example of several square miles imaged at 3"/pixel. We are still refining the image stitching algorithms.

Steve Morris
President, MLB Co.
www.spyplanes.com
smorris@spyplanes.com

[air]

Steve, That's the most impressive imagery I've ever seen from a UAV bar none.
How do you manage collision avoidance flying over built up areas like that?
Any more information on your camera / stabilisation system and stitching / orthorectification process?
I understand if you'd prefer not to share.
The output is stunning in any case.

[mlbco]

Air,

The imagery was taken a few years back before the FAA went crazy with definitions of model aircraft vs UAV. Now we do this work in foreign countries.

The camera setup is dirt simple, a Canon SD900 mounted in the wing and triggered by a servo. The flight computer calculates when the prescribed overlap has occurred and triggers the photo. At 1000 ft AGL we get approximately 3"/pixel resolution. The UAV can be easily programmed to fly a back and forth flight path that insures image overlap on successive passes of a specified region. In military exercises we often fly autonomously beyond telemetry range, take photos, and return with the data stored on-board.

The image stitching software is still in development and runs on a PC laptop. At the moment it takes about an hour to stitch 250 images but we expect the performance to improve with our latest software. I can't discuss the details of the stitching algorithms, but we may offer the software as a stand-alone product when it is finished.

I've attached some photos of the Bat 3 UAV, our main product. It weighs 23 lbs and can fly for about 6 hours at 40 mph. It also has a 3-axis stabilized gimbal video system. One of the photos shows a military beach landing in Australia from 2500 feet (check out the hovercraft on the beach!). An entire US naval battle fleet was protecting the area and never knew we were imaging them.

Thanks,

Steve

[Jack Crossfire]

The average Taco Bell employee in Silicon Valley can write an image stitching algorithm. The trick is going from still photos to 3D models without an army of Maya monkeys in Bangalore painstakingly plotting points.

[mlbco]

Jack,

If you only care about matching pixels in successive images what you say is somewhat true, but if you want to match an ensemble of images from successive linear passes and maintain an overall level of geo-location accuracy, then the problem is much harder. If it weren't then I would be able to buy some COTS software to do this with 1000's of images and I haven't found such software yet. If you know where I can buy it tell me now so I don't waste anymore money!

Steve

[wadiprawita]

Dear All,

Any recommendation on auto mosaic / auto stitching software for overlapped aerial photos ? It seems that many freeware (such as hugin) always assume that the photo is taken from one point (so, if used for overlapped aerial photos taken by R/C from different point, the result will be distorted).

regards

-doni-

[air]

Hi Steve,
First of all many thanks for your reply.
The level of verticality you achieved in that imagery was outstanding for such a simple camera setup, it gives me great hope!

On the image stitching, builder has had great success using the freeware program Autostitch (http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html) which does everything automatically when you feed in the output imagery.

I'm exploring some open GIS source packages (GRASS and OSSIM) for the orthorectification portion & I have both packages running now but I've yet to evaluate them fully.

[wadiprawita]

Hi Air,

I have download Autostitch, and it does a very good job (You use the demo version or one of the commercial version).

But still it doesn't do planar stitching, this is mentioned in the web page :

Q: Does Autostitch support planar stitching, such as flatbed scans or aerial photographs?

A:The demo version of Autostitch assumes that the camera is rotating about a point, so distortions will be visible when stitching multiple views of a planar surface. We hope to add planar stitching functionality in the future.


Regards

-doni-

[air]

Hi Doni,
I just use the demo version.
I actually emailed Matthew Brown about the planar stitching a few days ago but didnt get any reply. I also asked him if any commercial products that license their code provide the option. I'll report back if he replies.
Boulder's output seems to be excellent in any case.
I think that if you correct your photos for lens distortion first and they're pretty vertical, autotstitch won't introduce any additional distortion.

[zlite]

We (http://diydrones.com/) use PTGui (http://www.ptgui.com/), which works pretty well for this application. We use a 6megapixel pocket digicam, shooting at 0.7 second intervals. Here's an (low rez) example of one pass over the Lawrence Bekeley Lab Cyclotron. W're also getting 3cm resolution....

file:///C:/Users/Chris/AppData/Local/Temp/moz-screenshot-1.jpghttp://api.ning.com/files/uhOdKQpmzW81RgA3Hdp8nH6V1JO4wkNZkKShSqLBFNM_/pano.jpg

[ehx]

Serious work requires elevation distortion/displacement correction. At the lower altitude most here fly this is the main cause of positional error. Just look at any vertical photo. Find a telephone pole or tree - which is vertical in the real world - and then look at the coordinates of the bottom and top of the feature. They won't be the same. Especially if near the edge of the photo. For serious mapping this error needs to be corrected and you need a Digital Elevation Model (DEM) to do it.

This is what makes things much harder than just stitching adjacent images. If you want an accurate map you need to account for elevation-based distortion. A DEM (of sufficient accuracy) must exist or you must create one - via a stereo model or with laser range scanning, etc. This is not easy. Given this info there are literally dozens of software programs which can create highly accurate maps, but none are inexpensive (< $1000) that I know of. If anyone has tried any that seem to work well let us know.

[Ceros]

The key to all this (from a cost effective DIY or hobbyists perspective at least) is to implement as high a resolution digital still camera as can be afforded - keeping in mind UAV speed (i.e. the amount of frames you need to take for a given distance travelled per unit time required for the camera to proces each frame - which becomes increasingly important as the camera resolution rises).

In short: the higher you fly, the greater the area per frame and the more one can afford to trim off each frame - and the more one can trim off, the less distortion will manifest its self when stitching the frames together.

However, as said, the higher one flys, the greater the resolution requirement for the camera - suffice to say that 10Mb single frames at 5 -6 frames per second is realistic with good quality consumer camera's nowadays, is it not (?).

As far as memory goes, fire-wireing to the camera a standalone memory card of what ever type and memory amount one needs to use, will deal with the extra memory requirement.

Does this seem to be a realistic and acceptable, if not complete solution to the problem - or am I missing something that would suggest the above caould not be used?

[air]

Ceros, I dont know of any lightweight camera that can fire at 5-6 frames per second continously - can you provide an example?
Why would you need such a fast frame rate - assuming a slow flying UAV (say 30-40Km/h) one could get good overlap at much slower framerates once the altitude is over say 150 metres.

Zlite, you should try running those images through autostitch & comparing the outputs.
It shouldnt take much more than 20mins of your time (autostitch is totally automatic) - although processing will take a while.
Builder told me that he turns off "Auto Straighten" in the options (this has to be set each time you open the program).
There is another frontend to Panoram Tools called "Hugin" and there's a tutorial on their site that describes the process for stitching flat images but i've not tried it with quality input images as of yet. The results shown in the tutorial look impressive.

Ehx - Both GRASS and OSSIM both support orthorectification with DEMs to the best of my knowledge but as I say I've yet to try them out, bit of a learning curve there although on first impressions GRASS looks a bit easier to get going.
For our purposes I would imagine that obtaining ground control points through traditional ground survey would be the best option given the high horizontal resolutions we're working at.
Would you care to share any thoughts on using stero pairs for DEM extraction?
How feasible do you think this is for us?

Finally, for the group, can anyone share some insight on how large scale projects manage the conversion of multiple source images into larger raster blocks covering a large area?
In particular I'm interested in how to handle the transition from one macro block to another.

[Ceros]

Hi Air - sure, but "lightweight" is a very broad term.
Could you please qualify what is mean;t by the term "lightweight"?

If you are thinking in terms of around 500 - 1000 grams, then no - well, let me put it like this - difficult. But if your UAV can carry a few kilo's (say 2 - 3 or 4kg's), then yes - and for not much more cost than a good quality consumer/prosumer type digital still camera would cost you - and which would give you imagining quality far far quicker, and of siginificantly higher quality than that achievable with consumer type digital still camera's.

[air]

Ceros:
At present I just use a consumer camera, similar to what Steve is using. I modify my cameras heavily though, all up weight including switching power supply & electronic triggering is about 60g for a 7MP CCD camera. They take a lot of work to get to that weight but for me it's worth it.
I'd still be interested in learing about the high framerate digital camera options though.
I have a rotory wing platform that could handle the weight & it could prove useful for small areas.

[wadiprawita]

Dear Air,

So, the key in Autostitch is turning of the "auto straighten" ! Thank's.
Keep us informed about Your progress in GRASS ans OSSIM.

Regards

-doni-

[air]

Kudos have to go to Builder for the tips. I'm sure he wont mind my sharing his advice which was:
"I turn off auto straightning, and set jpeg quality to 100 percent; scale to 50 percent usually. select pictures with generous overlap of 40-60 percent and close to perfectly vertical for best results."

I usually use 100% scale myself in the hope that it's preserving some data, the info on autostitch is a bit sketchy so it's use is a bit of a black art.

[Ceros]

Ceros:
At present I just use a consumer camera, similar to what Steve is using. I modify my cameras heavily though, all up weight including switching power supply & electronic triggering is about 60g for a 7MP CCD camera. They take a lot of work to get to that weight but for me it's worth it.
I'd still be interested in learing about the high framerate digital camera options though.
I have a rotory wing platform that could handle the weight & it could prove useful for small areas.


I like the sound of that - " I modify my camera's heavily ...." gooooood, because then this will interest you.

I see no problem being able to rattle off 7Mp frames at half a dozen per second - 10 x 16Mp per second is possible, but that would cost a sack load of money.

So back to reality - well, you're going to have to bin the idea of using a consumer type digital camera and replace it with an industry machine vision head unit - something like that produced by Leutron or Imperx or Sony or Hitachi ( -there are hundreds to choose from) - for the purposes of these notes I'll illustrate a solution using an Hitachi KP-F500 PCL camera. Full details can be found on:

www.aegis-elec.com/products/hitachi-KP-F500_PCL.html

This is a typical example of a lightweight industrial vision head interfaced through a Cameralink - and the CamerLink interface method is probably the best method around today for taking high rate mega-pixel frames with low to zero latency - fact in this particular example you could, with carefull choice of frame grabber fire off at a rate of around 16 frames per second constantly @ 5Megs p/frame. Although I see you have quoted 7 mega-pixel in your case, I wonder if that is true effective pixels or if that figure is the amount of pixels on the image sensor given by the camera manufacturer. Camera manufacturers are like motorcycle manufacturers - they are (in)famous for quoting figures given for circumstances and/or definetions that are seldom repeatable in real life circumstances, and total pixel versus effective pixel is one example of published camera data where consumers often get an "inflated" impresion of just what their camera is capable of.

In any event this camera uses a 2/3" ccd sensor, with a pixel size of 3.45 x 3.45 microns over a 60mm square area - in plain speak, that is going to provide a siginificantly higher quality picture than the sensors found in a lot of higher resolution off-the-shelf type consumer digital still cameras which are advertised with a higher pixel figure. I don't wish to go off into a long write-up about just what the true value of x amount of pixels per frame means in consumer type digital camera's, other than to say, that as a standalone figure to picture quality/detail/resolution, it is misleading. The true realworld quality of a picture obtained from any consumer type digital camera needs to be evaluated not just against the camera's amount of pixels per frame, but againstthe background of the sensor type (ccd or cmos), the size of each pixel (usualy given in microns), the shape of the pixel (oblong or square), the image sensor scanning frequency, the wave lengths that the sensor is designe dto work best at - and a whole bunch of other factors.

In short, an understanding of all these parameters in their correct context often means one can find a 4 megapixel camera that will produce far better quality pictures than a 6 mega pixel camera, as an example.

Anyhow, nuff said on that - back to the subject matter in hand.

This camera, controllled through a small SBC (single Board Computer) of PC104 form factor, interfaced through a PC104 or PCI type framegrabber using CameraLink, could easily allow one to fire off extremly high quality pictures 5Mega pixel plus frames with micro second latency, which could be "dumped" instantly to solid state PCMCIA type solid state memory - meaning: the frame grabber outputs to an onboard removable type solid state memory card (you can get SD type cards of up to 32Gbytes nowadays).

What does it cost? Well, its a function of just how many frames per second one wants, and how large each frame needs to be (e.g. 10fps @ 16Mb per frame would cost in the region of Euro 20 - 30 000!). It rises expotentionaly quite quickly, but a system as I have described above, I belive could be put together for just under $3000.

Weightwise - all up less than than 5 - 6 lbs absolute max, and with careful selection of parts I believe 3 - 4 pounds would be just possible.

[ehx]

I was just pointing out that a photogrammetric solution is more than stitching images together. It implies a rigorous correction of errors like elevation displacement, camera lens distortion, etc.

Now a stitched image without these corrections may still be useful. If the area you are imaging is really flat a DEM-based correction may not even be necessary. Ultimately you decide just what errors you can live with.

An example of the type of software that can stitch and geo-correct images like Steve Morris collects is ERDAS OrthoBASE. It can be a mostly automatic process depending on the data input quality. The problem, for a hobbyist, is the cost goes into the tens of thousands of dollars. For a contractor doing this on a weekly basis it may be a very good investment.

I've installed and used GRASS, but it was some time ago. It looks like some ortho-rectification routines were added a few years back. Is anyone using GRASS for ortho-rectification? The price is right. The question is does it work well?

[air]

Hi Ceros,
Thanks for jumping in with a huge chunk of information there!
My cameras are slightly old school & the sensor is 1/1.8in IIRC which is bigger than the newer 12Mp offerings on the market. I totally concur on your thoughts regarding sensor size etc.

That information on the machine vision heads is invaluable and a great eye opener for me.
Just wondering what the 3 - 6lbs gets made up of?
I guess you'd have the camera (110g for the one you listed) and the frame grabber and PC104 board with CF storage etc (around 250g perhaps) - what makes up the rest of the fat?

I think Tom Harper and the rest of the guys doing archaeological work would be really interested in those frame rates for their high res low altitude work.

If someone could jump in with some input on steroscopic orthorectification etc I'd have a better understanding of why we might benefit from such high frame rates.

With my current low level of GIS knowledge it appears to me that my slow framerate consumer cam would provide me with the overlap that I need for merging of images etc.

A high framerate would allow me fly lower but then I'd be covering much less area with each shot. Also relief distortion becomes much more of an issue when the maximum terrain relief is more than 10% of the flying height.

For me I'd be flying at roughly 200m covering an area of approx 200m x 300m in a frame. At a 0.5 fps camera speed and 11m/s ground speed I'd have approx 93% overlap along the flightline.

Could a 1fps 16MP solution be realised for equivalent cost to the 16fps 5MP option you proposed? For me it's all about covering the largest area possible at the highest resolution possible in each frame.

[Ceros]

The weight was a worst case scenario - in reality s you have observed, all up it would actually be a lot lighter.

In real world terms the higher resolution frame rates allow:

- one to fly [so much higher] while being able to maintain the same degree of ground detail and resolution (and a lot more)

- cover larger areas per frame (again with no loss of detail)

- and lastly, the increased height allows a greater degree of relief distortion to be accomodated.

However, if you are wanting to maintain your 200m altitude, and the same lens type (i.e. surface area coverage per frame) then no - there is little benefit to be had by going down this path.

This realy is all about flying higher to get larger surface areas per picture while loosing no detail or resolution, and ultimately having less frames to stitch together.

[air]

That's great on the weight.

I understand fully the benefits of higher res image capture but I dont see the need for the high framerate at present.
The first 16MP cameralink cam I came across was this one (http://www.imperx.com/machine_vision/megapixel_digital_cameras/16_meg/index.html).
It weighs over half a kilo though without lens.
One disadvantage of large sensors is that they're going to have large and heavy lenses to go with them I guess.

Struggling through getting Grass running on cygwin at the moment.

[Ceros]

Air - dont forget that the 16Meg camera you looked at IMPERX is a thermo cooled camera - that accounts for a large percentage of the mass/weight. Its overkill for most aerial survey applications.

C type Lenses - come a dozen a dime - cheap, simple and light.

High frame rate. I see no requirement for it in your use - its generally related to camera resolution - you get it as part and parcel of whatever camera you choose.

[air]

Ceros - Would you mind recommending a compatible camera, lens, framegrabber and processor / memory interface that would be optimal for this type of work?

I've done a lot of searching and found another 16MP unit at http://www.framos.de/www.dir/en/produkte/kameras/digop/sonst/prod.dir/2356/index_en.html

However the site's selling this gear aren't too informative and I'm having difficulty figuring out what I need to build a complete system.
No worries if this is too big a request.

[Ceros]

Air - only pleased to help.

Some questions first:

1) Do you need 16Megs per frame? Can I suggest 8 - 10Megs (using a 2/3" - 1" progressive scan image sensor of CCD type the pictures you get will be far far superior to anything I have seen shown on this forum to date - so that means excellent because there are some very good examples on the forum).

2) What is the max frame rate you think will be required (now or in the future)?

3) How much surface area do you wish each frame to cover (in length and breadth - feet or meters).

4) At what altitude do you anticipate taking pictures from?

5) At what speed will the UAV fly at when taking pictures?

6) Do you have 12Vdc avalible on board, or do you have space to add another Lipo pack to provide 12Vdc for around 2amp/hr?

[air]

Hi Ceros,
Thanks, you're being very generous with your time & expertise across the forum.

1) I guess I'm not sure what I need really as I've no experience with the output of these industrial standard cameras. Going along with what we discussed eaarlier it would be nice to have the highest resolution available.
However if there is a very non linear relationship between the pixel count in a camera and it's size and / or cost then it might be better to go with a lower pixel count one as you suggest.

2) I think 2fps should be suffficient for everything I'd be doing.

3) At the moment I usually do about 300x200 squares using a 7.2MP for a theoretical resolution of roughly 10cm, this seems to be what I'm achieving in practice.

4) These are normally taken at about 200m

5) Approx 12 m/s

6) Providing a suitable power supply is no problem.

While I'm posting, it might be no harm to mention this thread from 2004 which covered similar topics
Low Altitude Large Scale Reconnaissance (http://http://www.rcgroups.com/forums/showthread.php?t=253849)

Tom Harper shares some interesting pieces of knowledge in it.

[bat]

Sorry for the hijack but I have a question I am sure some of you can answer. Feel free to PM me a response so that this thread stays on topic.

I just got back from an archaeological dig in Isreal at tel Dan. As we began digging we came across a rock structure, and it was decided to shoot in the points of most of the rocks using a total station http://en.wikipedia.org/wiki/Total_station

The process took 2 plus days and seemed horribly inefficient in the field. Thats when I came up with the idea. That I should reserach using a 50ft polecam with two cameras to generate stereo photographs. from the stereophotos generate a 3d topo of the archaeological feature. the pole came used in conjunction with software andother surveying equipment should provide a fast economical solution for recording in detail emerging features.

I would use a rc helicopter instead of a pole but I found out gps is not percise enough

The question is can anyone provide me with resources that would help me realize this idea?
software
field techniques
tools and equipment

Thanks
Thomas Beyl

[air]

Hi Thomas,
I did exactly what you describe using my maxi joker heli for an Irish university recently. They had already done a total station survey so I didnt recreate the 3D model for them but I've done it successfully on other projects using PhotoModeler.
It is quite time consuming to connect the tie points in the software but the accuracy achieveable is very good in my opinion and it is also a non invasive measurement which can be useful for sensitive sites.
Feel free to pm me if you need any further info.
Regards,
Fionn.

[stasilein]

The question is can anyone provide me with resources that would help me realize this idea?
software
field techniques
tools and equipment


Hi Tomas!

I'm just back from the field in Armenia where I dug a small Early Iron Age fortress. For excavations I used a 4 m aluminium ruler with a photo tripod on top to get vertical pictures from ca.4,5 m height. This method covers ca.4x5 m square per one shot. We put 4-6 red nails in this frame and measured them with a total station. After picturing the excavation area we made picture corrections and assemblings in Global Mapper. For pictures from higher distance I used EasyStar electroplane with camera looking aside or down.

Stas Vasiliev
St.-Petersburg
Institute of Archaeology, Russia
stasilein@mail.ru

[bat]

spasibo Stas!

I'll have to talk it over with the Dig director to see if we can do something like that when we dig in 2010.

[datapolo]

There may be an opportunity to carry out aerial mapping of several large West African cities. This is competing with satellite imagery so I will be looking to offer higher resolution and maybe lower cost. I am a consulting engineer and have been building up my RC aerial mapping capabilities for some time to assist with development issues. I am presently occupied with other work and if this project becomes more real I will not be able to dedicate a great deal of time to the field work. I am looking for an assistant/partner or maybe even a corporate entitiy to work with. I believe that this will be a very useful first project and will almost certainly lead to more work. If you are interested please feel free to contact me at datapolo@yahoo.com

Thanks,
Mike