Image Processing

[radiohound]

Image processing was mentioned briefly in another part of this UAV forum, but I thought it deserved its own thread.

What product is anyone using or thinking of using out there?

Image processing is one of the last things on my list of necessities, but it would be very cool to implement, and would really heighten the capabilities of any UAV. It would also be pretty high on the complexity scale, throwing my KISS (keep it simple stupid) slogan out the window.

One device I have pondered, that seems do-able, and is microprocessor compatible is the CMUCAM. It has the following features, and about 47 commands.

The CMUcam2 consists of a SX52 microcontroller ( http://www.ubicom.com/
products/sx/sx.html ) interfaced with an OV6620 or OV7620 Omnivision
CMOS camera (http://www.ovt.com) on a chip that allows simple high level
data to be extracted from the camera’s streaming video. The board communicates
via a RS-232 or a TTL serial port and has the following functionality:
• Track user defined color blobs at up to 50 Frames Per Second*
• Track motion using frame differencing at 26 Frames Per Second
• Find the centroid of any tracking data
• Gather mean color and variance data
• Gather a 28 bin histogram of each color channel
• Manipulate Horizontally Pixel Differenced Images
• Transfer a real-time binary bitmap of the tracked pixels in an image
• Arbitrary image windowing
• Adjust the camera’s image properties
• Dump a raw image (single or multiple channels)
• Up to 160 x 255 Resolution**
• Supports Multiple Baudrates: 115,200 57,600 38,400 19,200 9,600
4,800 2,400 1,200
• Control 5 servo outputs
• Slave parallel image processing mode off of a single camera bus
• Automatically use servos to do two axis color tracking
• B/W Analog video output (PAL or NTSC)**
• Flexible output packet customization
• Multiple pass image processing on a buffered image
• Works with the OV7620 or OV6620 module

http://www-2.cs.cmu.edu/~cmucam/

On a side note, I was thinking about image processing ..... this could be used as a range finder of sorts to detect altitude above ground features. Overall altitude is easy to obtain with the use of an altimeter. But finding the height of the plane over ground features is much more difficult/expensive (laser rangefinders, radar... etc). If frame to frame pixel movement can be detected (image shift), and suposing your camera was pointed straight down, then altitude over the ground could be determined as follows:

The field of view on any camera is given in degrees. Say a camera has a 41 degree field of view. When the camera is a foot off the ground, this field of view is a small area, and at any speed, the image shifts very quickly. At 100 feet, the area covered by the field of view is much larger. It takes a longer time to shift the image. This effect is noticed easily when you are on an airplane. At 30,000 feet it takes forever to change your "image" of what is down below.

What is needed is a known speed reference, but with a gps this is easy to obtain

One anoyance would be tilting of your plane, giving you error in altitude above ground. This could be somewhat compensated for by using infra red attitude/horizon sensors.

Any other good products out there for us?

[JettPilot]

It is simple, its just a matter of writing the programming :D . The electronics part of it would be very simple to put together and it should be very reliable. I dont think its worth all the trouble just for altitude above ground though, you need to find something better :confused: . Altitude above ground can be done with a GPS and a terrain height database, much simpler and much better ;) . I good use for the camera would be to keep from running into stuff, maybe low level treetop flying witout running into anything :eek: .

A CMOS camera sucks, if you are going to go to that much expense and trouble, might as well use a good CCD camera that would give much better data to the processor :) . I do like the idea though of a UAV that sees, the coolness factor alone would make it worth doing :cool:

[direwolf]

It depends on what problems you want to solve. That low-res camera is ok for tracking large objects like roads and the horizon, but you couldn't use it for small feature detection. You need optical zoom, or more resolution, or both.

Last year we experimented with some 1280x1024 Omnivision cameras, but they're very, very slow. It's almost impossible to take a still picture without blur. This year we're going with Sony industrial block cameras (FCB-EX780B)...
http://www.expandore.com/product/Sony/Colour%20Block%20Camera/Colour%20Block%20Cam.htm
Connected to a frame grabber on our on-board Arcom Viper PC-104 computers. Custom image server software packages the frames into TCP packets and sends the packets to our ground computers which process the images.

I haven't researched the use of cameras to track altitude, although it's an interesting idea. You could probably use optical flow analysis to track blob movement. There's going to be a maximum altitude at which the optical flow between successive frames will be undetectable, however. Also, as you mentioned, the ever changing attitude of the aircraft would be a problem.

In fact....
http://www.isy.liu.se/cvl/ScOut/Theses/Papers/A_Moe_lic.pdf
There ya go!

Another way to solve the attitude problem is to simply throw out the frames that are taken when the aircraft is not parallel to a virtual ground plane. This is one trick we're using to get high precision lat/lon coordinates of reference features in video frames.

-- Brian Stone

[Arp]

Until now, my choice of cameras has been limited to various USB webcams. I hear that other robot-builders sometimes use FireWire, but none of my boards supports this interface. Recently, I have preferred the "spca50x" Linux driver, resulting in camera choices like Creative Webcam Notebook.

I am currently hoping that a Linux driver will eventually appear for either Creative Webcam Live Ultra or its notebook version (would be even better). Both sport 640x480 pixel CCD sensors. Unfortunately, to pull a decent framerate, one would need USB 2.0 -- which is still rare on embedded mainboards.

Since I currently lack USB 2.0, and available processor speed is limited too (either 266 or 300 MHz) I somewhat doubt if I can use maximum resolution. Seems more likely that I must settle with 320x240.

[soccercisco]

If so, I have a couple of questions regarding Image Acquisition and Processing

[direwolf]

We've got a team that is going to start working with a CMU Cam in a couple of weeks. We're developing a quad-blade helicopter that will travel indoors, and we need to devise ways to maintain perfect position without drifting. One thing that we're going to experiment with is using CMU Cam to do optical flow alaysis to detect the relative motion of the floor to keep the helicopter from drifting.

-- Brian

[soccercisco]

:o I gave up though... I don't think is what I was looking for... I switched to a BoosterCam...

Good Luck.

I figured out how to use it though, if you have questions let me know.