Visual target recognition/following

[zozie]

Well just wanted to share what I've been working on.

I started working on visual target acqusition a few years ago but never managed to get enough computing power to make it actually work. That has now changed by using 5 mini ITX boards with 1ghz CPUs. I can now visually "lock on" to objects without the aid of IR or US sensors. All it takes is two cameras. At this point it can only process 500*500 pixels but that should change as CPUs get faster. At this point, my mini LST will follow any object you select form its field of view, so this week I'll be trying it on a plane. It should follow another rc plane strictly visually without any other aid. I'll be using a hangar 9 60 stick converted to electric. It can at this point keep a set distance from objects so there SHOULD be no collision but of course I need to try that :o
If interested I'll keep you posted.

[aschuck]

That is a very cool project.
Please keep us informed.
Alex

[zozie]

Thanx I will :D

so far so good, but the problem I need to overcome now is still speed. As you can see on the video, it's OK as long as the object doesn't move too quickly. When it does, it may loose it for a sec. It can lock on it again, but it makes mistakes. I got my mini LST to follow and other rc car, but then when it moved too abruptly, it lost it and picked up something else. It should be a bit easier on a plane/heli cause they can move up and down if you know what I mean.... so we'll see if there is a collision or not. I have to try it to find out, cause the distance between them cannot be more than about 30-40 feet :eek:

[Tuner]

http://www.anafocus.com/home.php?opcion=7&subopcion=200&id=23

Thats all I have to say about that.

[only_way_99]

Hi zozie
I’m interested in your project very much
also i was prepare to make exactly what you did
uav tracking system

my experience is in programming in c under windows and Linux

Embedded Linux

Embedded systems design
I’m working now on blackfin DSP bf537

so let's talk about what i have notice in your project

first i saw that you are using 500X500 camera why ????
this large amount of pixels need very high speed processor also it must be DSP processor

Why you don't try 120 x 90
also you don't need colored pictures camera
in the gray scale each pixel is represented in 8 bits only
but in colored pictures each pixel contain large amount of bits
also the camera which you are using is very bad for application like what you are making
Take a look in those sites

http://www.surveyor.com/blackfin/

http://www.fastvideo.ru/english/products/smart/fv250bf.htm

we can use gray scale camera to minimize the processing needed to be done
Also the servo motors is slow and the program may be fast but the mechanical part is slow so it may not give you the fast response which you need

That is all what i notice
but i will read more in your web site to understand what you have done
and I hope to work with you in this project
See you soon...

[zozie]

Thanx for your thoughts, but ideally I want more like 2000x2000 pixels to make it worthwhile. And it cannot be grayscale. I need it in colors so different objects are easier to distinguish ;)

[only_way_99]

objects can be distinguished easily in grayscale images ( it is used in any object tracking systems ) the aim behind what we are working in it to track an object so fast
so there is no need for color images

and put in your mind the delay in processing and the delay in the mechanical part and we need fast object tracking ...........
so we have to minimize the processing time as much as we can it is our only choice if you have any other choices just tell me to think with you :D

second:

i saw some thing i think he made what you need

http://www.youtube.com/watch?v=bWl33urh2w8

but I’m still searching for the algorithm which he used to make this program
but I’ve reach to the materials and lecture notices of CS335: Principles of Multimedia Systems
also he used OpenCV i'm still reading in it

[zozie]

I used opencv before but it was missing the features I was after so never touched it in 4 years..... Open cv does not need colors you're right, but my project does.

[only_way_99]

so what can you do to solve the delay problem ?

[zozie]

Other than the physical limitations of the servos, I just ordered 5 mini ITX boards with Dual core 1.6GHZ Intel atoms. we'll see how they do.... Will certainly be faster than the 1.0GHZ AMDs. I've tried it on a machine with an Intel core 2 quad 2.83ghz and it was nearly perfect so I just need more cpu juice. Plus dropping windows for Linux may help too but not sure if I'll ever do that due to drivers.....

[eddymoore]

Perhaps an obvious and silly question, but what's the development language? Have previously come across people complaining about the performance of their image proc software, then they tell you they're using Java and you loose all sympathy.

[zozie]

Its about 60% C++ and 40% asm so not much to get rid of there...

[Dimitris76]

I am sorry in advance if this sounds stupid, but why so much processing power is needed for locking on and tracking optically a target?

My new Panasonic Lumix DMC-FX500 digital camera does exactly that (optically tracks in order to keep in focus) at any zoom level, any moving object speed with almost 100% accuracy... you can touch the screen on any object you want and then just sit and observe the square frame following that object nomatter how you move around the camera.

[zozie]

Well I'm sure they used a more advanced algorythm then what I can caugh up at home by myself, but mine also calculates rough distance so it can follow or stay away from an object....

[only_way_99]

are you going to buy this
http://www.mini-itx.com/reviews/atoms/

if yes
tell us please about it and how they connect 5 boards with each other

[zozie]

Well instead of playing with networking the boards, I found a nice one with a Core 2 Duo CPU which will fit nicely on a Stick 60. I'm getting the plane today after work :D I hope it will live more than 5 mins :D
Anyone has a rough idea as to how much weight a 60stick can carry around? I'll be running the e-flite 90 motor with 8S lipos.

[Tuner]

What are the FPS you think will be adequate

If you cant accomplish what you need with 60fps then it will be astronomical in cost and weigh to much to do this. I can recommend a few board level cameras that can accomplish this.

If their is one thing I can promise you is that currently the Power and Weight requirements are going to make this very difficult to use in a small UAV!!!

If you are talking a larger gas powered UAV then I would recommend Checking out the Bi-Eye from www.analogic-computers.com OR any of their FPGA's IP cores.

If you are doing object tracking as a means of stabilization then Optical-flow with a low resolution sensor is the more efficient way to go you can even accomplish some obstacle avoidance with Optical-flow.

I have researched what you are talking about in great depth and the power and weight requirements are quite high. Also the cost is quite high. Most 2000x2000pixel cameras that work at high frame rates of 120-1,000 frames per second are around $15,000-$35,000 maybe you are already aware of this though or dont need these frame rates

FYI Atom cpu or really any CPU will not work nearly as well as a DSP FPGA combination its just the nature of a non application specific processor. I would consider the Blackfin DSP/CPU / or the New TI OMAP DSP/CPU.

I have seen what you are talking about done with 2 NVIDIA Graphics cards if that helps.

The FPGA solution from analogic are awesome! Any other solution would require 1ghz TI DSP or better and they suck a lot of juice.

The best solution I have found is a Hybrid solution. Grayscale low resolution camera for high speed tracking combined with a color high resolution low speed (60fps) camera for finding the objects and relaying this to the low res camera. I put many many hours into R&D and that company still has the best tracking system(that only costs $3,000 not $30k) because they used my hybrid approach.
High res low speed 60fps for finding object once found low res high speed for tracking it.

[zozie]

More videos are coming tomorrow :D
It can now follow and avoid multiple objects, while observing their speed and calculating the possibility of a collision.
Anyone in the Toronto area want to join in?

[Buzz_Lightyear]

I've worked with OpenCV and I can say with a fair degree of confidence that OpenCV is pretty optimized so will most likely be faster than your own homebrew tracking system.

[AdamKt1]

I agree with Tuner. Very interesting.
Tuner can you please suggest the cameras you are talking about. Can gray scale cameras be used for long distances like 1000 meters?
Secondly I shall love to know about your Hybrid solution.

[AdamKt1]

Tuner, what do you think about the suitability of dsPics for the application.

[tekrunner]

What's the latest on this? Any open source target tracking solutions being developed?

[BeefStake]

I am working on target tracking and identification using SURF descriptors.
My work will be released opensource eventually, most likely as an engine library implementation of SURF and then later my object tracking solution.

I am using feature based recognition because it is very robust in real world environments.
Because my system is designed for use in UAV applications I am trying to optimize it as much as possible but I imagine that it will not be running onboard in it's current state. Most likely realtime video will be relayed to a ground station and processed there.
To solve the tracking issue when objects are moving quickly or behind other objects etc I am using a Kalman filter to estimate the probable position of the object.
This approach is incredibly robust and I have used to to excelent effect on the ground atleast.

[mentari]

I am sorry in advance if this sounds stupid, but why so much processing power is needed for locking on and tracking optically a target?

My new Panasonic Lumix DMC-FX500 digital camera does exactly that (optically tracks in order to keep in focus) at any zoom level, any moving object speed with almost 100% accuracy... you can touch the screen on any object you want and then just sit and observe the square frame following that object nomatter how you move around the camera.

That is because they are using a http://scratchpad.wikia.com/wiki/FpGa with a high level design flow tool that converts control loops and C directly into Verilog. FPGA, http://scratchpad.wikia.com/wiki/TileraMulticore and http://scratchpad.wikia.com/wiki/PicoChip are different means of achieving parallal processing. Something which a linear processor like a PC can't do very well. For the embedded space you want to fit an object tracking device you would have to go FPGA route and purchase expensive expensive design flow tools ($40000 + ).

[taratharanath]

Hello How much this would cost ? :confused:
http://www.anafocus.com/home.php?opcion=7&subopcion=200&id=23

Thats all I have to say about that.

[jaguarbite]

Zozie ..Probably same thing is already launched in the market by www.Procerusuav.com called onpoint vision system (previously it was softeare based but not it is embedded hardware based.