See & Avoid

[Tuner]

Im not really fluent with the FAA guidlines and am learning more everyday.
Would anyone be able to confirm my suspicion that if you demonstrated a UAV with electronic based see and avoid capbilities then this aircraft would be able to fly with out having a observer.

I know that using machine vision and cameras you can do this with off the shelf software and hardware. Granted its going to be a heafty system that would way 30+lbs.


Is a visual system enough or must it be radarbased/3d.

I know of a imaging chip that has the potential of taking my 30lb system and getting the same results from 2 IC;s and DSP controller combo.
Analogic Computers (http://www.analogic-computers.com/)

I just wanted to see if anyone had opinions on how to tackle the see and avoid in order to open up commercial UAV markets.

Scott

[kd7ost]

I'm no expert so can only state that you are likely correct. But the FAA will have to approve the system through some source. It would have to be subject to the same type of rating system they use in all aviation.

Dan

[mwraight]

I'll try and give you what I know so far. If I were at work I could give you the links...but going to the FAA website and searching for documents with UAV in it will bring up most of the newest info. The legislation seems to be in limbo at the moment...and things will change and things will be added.

The FAA is actively working on regulations for UAVs. Most of their focus is on the larger sized UAVs. They are working on a certification and training program. So far there has been only one UAV that was given an Experimental type certificate. This UAV has to be followed either by another manned craft in flight or by a ground crew in constant visual contact with the UAV. I believe the craft is also required to be fitted with an operating transponder with altitude encoding. I think the bigger UAVs are heading towards legislation that puts them in the actual airspace structure and control.

Now, as for the little UAVs - like the one I'm working on (5' wingspan and 6 pounds) - the FAA doesn't have anything worth saying on them at the moment...at least what I can glean from reading the documentation. I'm planning on getting in contact with the FAA and see where the regulations are going on the smaller sized UAVs. So what would constitute See & Avoid for the smaller planes is open for debate.

Since I'm working on the training program for a UAV I plan to incorporate common sense approach to flying. Stay out of controlled airspace (i.e. busy airports and above 17,999' Mean Sea Level).

Being a full scale pilot and former flight instructor and having numerous friends flying airliners I'm trying to keep abreast of all the developments in legislation.

I have a feeling that the smaller commercial UAVs might end up needing the same capabilities as the larger UAVs...they will probably end up subject to some type airframe and training certification.

The FAA has stated that they are very interested in developing rules that allow commercial UAVs to operate and they realize this is the new up and coming thing. But the first UAV to get sucked into the intake of a 747 or bringing down an airliner or flying into a TFR (temp. flight restriction) area has the potential to change all that.

Hope that helps. I would just keep a watch on the FAA website...or call your local FSDO (flight standards district office) and ask them about it. They will be more than happy to help explain the rules, regs, etc.

[shedao]

I've been doing a tiny bit of research on this myself given that I contract for the FAA, we're building a new rulemaking system for their CFRs. But the FAA is a quite large organization so it may take some time. They announced the following in September of 2005:


http://www.faa.gov/news/news_story.cfm?type=fact_sheet&year=2005&date=092005
Operation and Certification Standards
To address the increasing civil market and the desire by civilian operators to fly UAVs just like any other aircraft, the FAA is developing new policies, procedures and approval processes.

* At FAA Headquarters in Washington, D.C., a team of experts from various parts of the agency is working on guidance that will increase the level of oversight in a step-by-step fashion without being overly restrictive in the early stages.
* Developing and implementing this new UAV guidance is a long-term effort and is still a “work in progress.”
* More immediately, the FAA is reviewing certification requests from several UAV manufacturers. The first airworthiness certificates in the “Experimental” category (for research and development, crew training, or market survey) will be issued before the end of 2005. These certification efforts provide an excellent opportunity for the FAA to work with manufacturers and to collect vital technical and operational data that will help improve the UAV airworthiness certification process.
* The FAA has asked RTCA – a group that frequently advises the agency on technical issues – to help develop UAV standards. RTCA will answer two key questions: How will UAVs handle command and control, and how will they detect and avoid other aircraft?
* The FAA also participates in the NASA-sponsored ACCESS 5 effort to understand the difficulties of integrating UAVs with piloted aircraft. Although the group’s work focuses on high-altitude, long-duration UA operations, many of the standards will be applicable to other types of UAVs.
* The FAA continues to work closely with its international counterparts to harmonize standards, policies, procedures, and regulatory requirements.


FAA Unmanned Aircraft Systems Branch:
http://www.faa.gov/about/office_org/headquarters_offices/avs/offices/afs/afs400/afs430/projects/media/AFS430ProjectContacts.pdf

[Tuner]

Wounderful you have got to design with the future in mind this gives me something to start with thanks!!!!

[Vindication]

I am skeptically about the vision system being as easy to implement as you say Tuner! Off the Shelf at 30 lb? Don't know anything about that system. However I haven't yet seen any fully competent systems on small platforms like we're talking about in these forums...

That said-- these are on going areas of research and improvement. I do believe we will have such abilities soon...

[space_case]

I worked on an image processing project that was shooting for just such a thing. At the time, we were just grabbing the horizon line (in lieu of pitch and roll ouputs from a gyro). The video was downlinked and then the image processing was done on a G4 Powerbook.

I saw some video from this project recently and they have come a long way in the terms of obstacle avoidance.

[space_case]

The plane had a 24" span BTW.

[kd7ost]

Just to throw another wrench into the gear train, I'm sure the FOV required by such a final system will have to be something like 180 to 210 degrees. Something that a pilot could see if turning his head from side to side. I think we can be certain 90 to 100 degree's won't be enough for the see and avoid.

Dan

[Tuner]

If you read my initail post I refered to an Imaging Chip that basicly has a microprocessor per Pixel interlinked with all other pixels so you can perform calculations that would require millions of processes in one proccess. It uses Cellular Neural Networks along with other techniques to accomplish this.

I have worked with systems that Use a PC and very powerful DSP cards to process the mass quantities of video and the problem is not the CODE but more the limitations of the hardware speed.

A Simple 120-180deg Fisheye lense was my intention. As the only proper system should "NOT" have any blind spots IMO.

I have done a fair amount of systems integration and I am always SHOCKED at what you can get with OFF-The-Shelf parts properly integrated.

The Sensors I am considering are being considered for use in the next Mars Rover to give you an idea of their potential.

Anyway I aggree its not as simple as I make it sound I have just seen this done for terrestrial based rovers so I know it can be done.

Scott

[shedao]

If you read my initail post I refered to an Imaging Chip that basicly has a microprocessor per Pixel interlinked with all other pixels so you can perform calculations that would require millions of processes in one proccess. It uses Cellular Neural Networks along with other techniques to accomplish this.

I have worked with systems that Use a PC and very powerful DSP cards to process the mass quantities of video and the problem is not the CODE but more the limitations of the hardware speed.

A Simple 120-180deg Fisheye lense was my intention. As the only proper system should "NOT" have any blind spots IMO.

I have done a fair amount of systems integration and I am always SHOCKED at what you can get with OFF-The-Shelf parts properly integrated.

The Sensors I am considering are being considered for use in the next Mars Rover to give you an idea of their potential.

Anyway I aggree its not as simple as I make it sound I have just seen this done for terrestrial based rovers so I know it can be done.

Scott

I may start another thread but this is related to see and avoid - have any of you dealt with or thought of using optic flow techniques for see and avoid? I found a great paper and some other info on using it as a very cheap, but not simple, technique of navigation. I intend to explore this on a ground based robot for simple navigation first then go from there, using simple sensors to help measure movement.


Mirrored versions of the papers:
http://shai.enemy.org/papers/Embedded/UAVS02.pdf
http://shai.enemy.org/papers/Embedded/Thesis_3194_Zufferey.pdf

I can provide the urls for the original owner urls if needed.

A company that will produce optic flow sensors:
http://www.centeye.com/pages/techres/opticflow.html

[Tuner]

I am very familiar withthe theory and In fact this is almost entirely what I am talking about. The sensors I am talking about create this information using a Cellular Neural Network.
The sensor is only 128x128 pixels yet can yeild some startling information at amazing speeds.
I mean this sensor is exactly what you are refering to.
It future encarnation will have greater resolution and greater processing power.

The company calls it Flow Processing Algorithms.

Im telling this is where it is at and anyone doing anything else will be either wasting their time or improving on nature as this mimics millions of years of evolution. The sensor is refered to as an Artificial Retina for a reason.

They are comming out with new chips shortly.

Scott

[Vindication]

I agree Optic Flow has great potential. I used it for measuring Height Above Ground on a UAV [in simulation only].

Vind

[goliathgrouper]

Tuner,

What sensor or company are you talking about?

[goliathgrouper]

Optic Flow is a hard nut to crack because of all of the variables involved. The Airforce has been working on the algorithms for years and have recently tested a system. It failed miserable, to many false alarms, the plane couldn't get from point a to point b without zigging and zagging all over the place. I think the first FAA Certified Sense and Avoid package will be a suite of a Millimeter Radar (Amphitec) coupled with an Optic flow EO Camera/IR camera and possibly an acoustic sensor thrown in.

Just my 2 cents...

[Tuner]

www.analogic-computers.com is the company website and the ace16k is the image sensor/processor soon to be the CTOL line of chips (greater resolution & processing power).

If their new line of chips are built the way they were talking 2 years back I beleive they will by SOC (system on a chip) so it may allow for a much more compact and efficient design it will be the 3rd revision of this chip so the technology is more mature than any other I have seen.

One comment on Optic Flow and the military. Dont let that be you weather vain on technology. The military gets it first I will agree but the private companies/universities always makes it useable in a mass market, witch is what some of us (me) are looking for.

I completely aggree that until I can run a detail 3D simulation on low powered CPU you are going to require things like GPS or Millimeter Radar to help avoid oscillations (ziggzagging around) and other anomilies that would happen with limited environmental/positional data. I just dont see most radar fitting in a 5lbs airplane anytime soon(and be practical).

I would be really curious to know if the Military Optic Flow experiments used traditional image sensors and processing techniques or if it used analog processing and Neural Networks???

[treehog]

Commercial UAV is most unlikely for the small enterprise if see and avoid rules are going to be enforced

The logical way was for a craft to have a Radar transponder and auto radio warning radio system combined with pre announced flight paths and a facility for ATC to force the craft to stop and parachute or alter course if they could see a potential future conflict issue arising

The mega rich companies that control the Aviation industry know fully well that with the gooks and 911 and the fact they have the mega bucks to get past the hurdles that they will design & form and decide for the FAA and other relevant aviation bodies so as to ensure the small back yard operators wont get a whiff of this market activity buy ensuring it has to have a state of the art see and avoid systems that they will license under franchise from the FAA

As a result the other world aviation authorities will follow the mold and so small back yard operators world wide will probably not be able to afford the expensive testing and license regimes

So I will only be interested in visual UAV or micro UAV less than 6LBS that might be able to sneak under the radar under balloon rules that apply to weather balloons that allow the equipment to be parachuted back

the only way that the multi nationals might be stopped from their drive to keep the small operators out is probably by ensuring that ATC has ultimate control over any UAV either by some form of radio control system that they would own or be in control of by subcontracting license or cell phone company etc
and the crafts carry some form of transponders and file flight plans

In that situation it might be possible to get less restriction for various types of UAV such as craft weights might be the deciding factors such as size under 20 lb. under 50 lb. and under 100lbs flight speeds less than 500-mph
height less than 20,000 feet could have different restrictions and requirements than a UAV 1000KG craft with 300-kg of cargo that flys more than 500-mph and height of more than 35000 feet

See and avoid will in time be possible at a cheap price but then the the aviation companies will change the goal posts so you have to have infra red night vision and radar and have a R2D2 unit on board as back up :eek:

The reality is most ATC is done in dark rooms with radar and transponders and flight plans voice communication change of course and height and speeds
and so it should be possible to conform to those rules and coupled with a transponder who's costs will get less every year in cost and size
and should be adequate that will be the norm for craft of 20lbs to 200lbs sizes with suitable unique High viz flashing strobe lights and no ATC override system
if the craft operates more than 30 miles from major airports
and simple cell phone contact-able stop Parachute or alter course possibilities for craft that would operate inside the 30 mile sector around the airport if the voice activated command from ATC were ignored by the ground controlling pilots systems

For larger size craft then its a different story and more complex but as I am only interested in models for fun anything bigger than a seven meter slope soarer or a 400-cc petrol craft dosnt really interest me and all I really want is the stupid fly away comes back or the dynamic soar can have self preservation ground avoidance when my thumbs get crossed

Ralf




:eek:

[space_case]

"gooks" :rolleyes:

Who is this clown?

[lvspark]

http://www.sony.net/Products/SC-HP/cx_news/vol34/featuring2.html

[poynting]

That sony concept has been around for years... the article referenced above was put out in November of 2003. Has anyone actually heard of a product that uses this system?

[lvspark]

http://www.ipix.com/downloads/ipix_cv_dome.pdf ???

[Tuner]

I have worked with just about every panoramic imaging concept around.

Ipix is an evil awful company. If they have changed it is news to me but they are by fare a very negative company and had they not gotten schooled in court they would have set back commercial panoramic imaging many years.

My favorite methode is the http://www.panoscan.com/

Unfortunately what people have not realized yet even the larger corporations is that now Joe Blow Corp has access to technology fresh off the press and with open source projects you have the resources of large groups of people collaborating. Treehog I have to say I see you point its a tuff world but if Burt Rutan crapped out a space craft for 20 mil and did it in no time Im a little more optimistic.

Full Spherical Panoramic Image can be obtained with just 2 cameras each with a fishey lense.
Scott

[Iraqigeek]

Tuner,

Make that four cams with fish eye lenses, set up as two pairs, where the cams of each pair are setup in a stereoscopic fashion (pointing to the same direction, but have some distance between them).

The hard part in such a setup would be the pattern recognition algorithms required to recognize an object location in the image produced by each camera of the pair. Once you can determine the 2D location of each object, extracting 3D positional information would be simple trigonometric math. Once you have 3D positional info of each object, WRT the UAV current position, the course of that object can be deduced by looking at its position change between two consequtive calculations, pretty much the same way a GPS calculates your heading, speed, and climb rate.

As long as the system doesnt have to recognize the kind of object (it doesnt care if the object at hand is a small cessna or a 747) the pattern recognition algorithms would be greatly simplified, but still have to be robust enough so as not to mix up different objects.

[Tuner]

Steroscopic Fishey is a Great Idea for larger models.
One of optic flows greatest advatages is that you would not need to calculate the position or do pattern recognition but you can react based off the optical flow.

Less cameras = less power & Cost & weight.

It could be done with one camera and a 360 lense instead of 2 camera with fisheys. The issue with the 360 lense is that it has a blind spot on the top and bottom. 2 180deg Fisheye have no blind spot but rather the resolution is greatest in the center.

The theme I keep seeing is imitate nature and Parralell processing.
Currently one of computings biggest hurdels is being able to parralell process information this is why I like the ace16k image sensor/processor so much.

[Iraqigeek]

I dont have much info about the ace16k, but I tend to take such "revolutionary" solutions with a big grain of salt.

As for the pattern recognition code, With PC104 or miniITX systems, soon to come low power dual/multicore processors, I believe that such a quad camera system could be crammed in one or two boards. Basic pattern recognition algorithms don't require that much processing power, and have a very high granularity when it comes to parallel processing.

As for the fish eye lens, its easy to solve this by correcting for the camera lens model.

All in all, if you can provide enough electric power to run the hardware, the whole system can be packed in under 10lbs (including the cameras).

I believe that the key point here is that you don't need an awful high amount updates, nor requires high levels of measuring accuracy. I think updating at 10hz, and an estimation accuracy of 50 meters (~150ft) is enough for the UAV to stay aware of all its surroundings, while avoiding any objects that may be flying around.

[treehog]

Iraqigeek
what you are saying might work in a clear bright sky devoid of clouds rain lightning mountains and flocks of birds
Then the real world sucks in that traffic can exist direcly above and below you and when its below you the confusing gound could include the highway traffic so how to discimate ground stuff from airbourne stuff

I have worked on small sailing yaughts and they have realitivly light weight radars that would be able to see and identify targets well in day or night and discrimate the size speeds and heading in useable electronic format that can allow for predicitive collision aviodance and can cost less than $1000

The radars each year get smaller and require less power so might be feasible to buy one with power supply that weighs less than 40 kilos and you would still have to have some form of tilting of radar so as to scan up and down as well as these are suface use radars

see and avoid using vision will probably never reach a satifacory low level of misidentication unless backed up with ATC update info from responders and or some other source such as onboard radar or similar

Even in ordinary flying vision is not considered a vary effective means of avoidance and so that explains the zillions of dollars that gets spent on ATC

For my money its better to invest the time and energy into systems that ATC will approve and the chances that it will be solly a vision based succeeding will be IMHO be very low

edit
Seems my toungue in cheek definition of a word
a bad word that some like to question
broke some rule
I hope this edit will be sufficient
edit


and worse they hide under your bed

Ralf

PS the austrial police use the 360 degree camera and it cost $63,000
how about a $20 radar reflector in this logic

[Tuner]

HUUUUGE Sighhhhh.

Both you guys need to look into the ACE16K.
IT IS NOT A FRINGE NEW TECHNOLOGY!

This has been in production use for several years and has been long in development.

IT USES NEURAL NETWORKS to process information not some cheesy Intel CPU or Ti DSP. I know it sounds scary and all StarTrek like but it is simply an Analog Processor. Infact it process all the information in Analog and then the results are then digitized and sent to the host CPU. It is in essence a PRE Processor that sends only the information you need Such as Edge detection vectors, Motion Vectors......

A Vision system like this is the ONLY Lightweight sollution for a Sub 10lb aircraft as Serial CPU's even in a Dual or Quad setup are to inneficient. This is of course just my, know it all done my homework hope to convert a few non believers, Opinion :)
I have done hundreds of hours (6months 10hours/day) of research into this area, but dont take my word for it do a little research yourself.

The whole point of this thread was to confirm that See and Avoid Technology would allow people to operate Commercial UAV's without a direct observer and for me to hopefully enlighten some people about this technology.

To give you an example I did R&D on a project that needed to track a milliwatt Laser (very low contrast) dot at millimeter accuracy and at EXTREMELY high speeds 1,000fps(slower for more complex scenes). We had to do it at a low cost under 10k (most highspeed High Res cameras start at 25,000) The design I came up with was basiclly the Bi-i(witch uses ace16k). They did not have the resources to develop this.
I lookedfor patents on this concept and found Analogic Computers. I presented my research and product recommendation to the company and I was dismissed from the project because everyone had the same attitude, Ill see it when I beleive it..... (Yes its bacwards :)

Incidently somone convinced them to see a demo or something because now they can boast having one of the fastest and most accurate system in their industry by a factor of 5-10 times :) The only other way to acheive this is at a cost of 10-20 times using traditional DSP equiptment and Cameras.

Transponders are a no brainer. For Larger UAV's or ones that fly in IFR conditions would need a radar. But For inspecting a Pipeline durring the day or replacing a News Helicopter a Vision system will work.

Scott
Sorry for the mild rant....
That said none of these solutions are what I call elegant In my opinion the Infra Red (thermal) Levelling systems are elegant and something like this only better would be Excellent but in the end you just can't beat a AUV that has a level of intelligence especialy since it is a Lethal object.

[Tuner]

TreeHog Would you enlighten me on some Manufacturers and Models of the Smaller Radars you speak of.

Or if anyone else has millimeter radar info I would love to see it posted hear or a link to a thread. THNX.

I would love to call them up and see what they have slated for development.

Thanks

TreeHog I got your Humor at first but later was unsure if that was what you intended it seemed a little out of place... as I look under my bed :)

[treehog]

if i climbed up some mast to fix a radar set i have climbed up a...

The radar units use standard radar frequencies presume 11ghz
they consume gobs of power 10 amps plus
they often have a facility to reduce range
they often have away to chose a section of screen usefull for entering port so you only see the front view and not the side clutter

The radar units themselfs each year get lighter and power demands reduce a bit but not so much

If you tilted a unit on its side it could doa forward looking sweep

There also exists passive radar reciever units which can tell you that you are being seen by radar and you can null point them to get a bearing on the target

Best is to go to a local marina and see the latest gear

My outlook on this is that although vision can be an interesting passive solution once you get hooked into the idea that it would be primary avoid system your giving in to the beuracratic types that invent potensial expensive solutions that could stop another legitiate sport interest industry dead in its tracks usualy under some catch all banner security etc

I prefer to push the cheaper now doable cheap radar reflectors realitivly cheap repoders

It would also be nice to have a vision back up system to complement other systems but I have no illusions about how difficult it will be compared to other systems which are cheaper more effective and more proven

Humour is what makes the planet tick

Every world sectors got buttons that need the odd push
I for one dont want the security issue card pulled in euroland just because of some potensial possibilty
that would let the bad guys win by spoiling the fun and give the big multinationals ownership of the skys

no humour no fun

Ralf

[Iraqigeek]

Tuner,

I don't know how you can be so sure that this is the "ONLY" way to go for such an endavour. Not that I like intel chips, but I wouldn't rank them as cheesy. As for DSP (TI or others), those were designed with a completely different set of applications in mind, than to being used for this kind of image processing.

One important thing to keep in mind here is that your platform (no matter what technology it is based around, and what silicon it includes) is only as good as your software/firmware and algorithms are.

As far as computer vision systems go, finding an object in a given image isn't hard, nor costy in terms of processing power. However, things will get exponentially more complicated when you want to identify what kind of object that is. For a UAV vision system, you only need to find objects within the image, and it doesnt have to be done at 100s of frames per second, nor does the system has to do any form of recognition on that object. Some will ask what about the ground and ground objects, and how will such a system deal with it? I think its easier not to deal with it at all at the design level, and let the system track any object it can fnd in the image. If the algorithms are optimized for the platform on which they operate, it will be more cost effective (in terms of processing power) to track a large number of objects using a simple and fast algorithm (pure and ugly brute force) than to try to recognize what each object is, and then decide whether it should be tracked or not. As long as the system can extract 3D coordinates information of that object, there shouldn't be any problems.

Another dilemma that any visual system will still have to deal with, no matter what technology its based on, is how will it extract 3D information about an object from a 2D image? Even more complex, how fast, and most importantly, how accurate will this system be able to match objects across multiple sets of images, in order to deduce motion information. Such a system needs to be able to work on a set of images, coming from more than one imaging sensor, concurrently, to be able to extract any form of 3D information.

Tracking a veichle or a laser dot with a camera is one task, finding how far that veichle is, in what direction and how fast is it moving in a 3D world is a totally different story. Just take a look at games. An old 20MHz 386 is more than capable of handling relatively complex geometry in a 2D world, but you will need a multi GHz processor (not counting the more effective architecture) processor to handle the same level of geometry complexity when things go 3D, and I am not talking about the graphics rendering here.

I don't think that such systems, no matter how good they may become, can reach the level of reliability of a radar, but as far as cost, size, weight, and power requirements are concerned, vision based systems are much more feasable.

[Tuner]

I agree I obviously have a bone to pick.

My only argument against alot of other solutions is they require signifigant hardware development.
Where vision using an artificial retina based vision system only requires signifigant and creative software development. In my opinion at least.
Being a creative Coder I see this as a much more obtainable solution for myself.
Now if I had a great RF engineer at my disposal that is another story.
Especialy because as you reduce the power of a Radar or Laser system its range and accuracy decrease (exponetialy I beleive) its something that is difficult to overcome as it is a physical limitation. The only way to improve this is with very very expensive components that have high-gain low noise and signal distortion(aka very expesive amplifiers).
I will admit see those Automobile backup warning system using radar-on-a-chip gives me hope but the range is obviously not in the hundreds of feet as would be needed.

I will stop B.S.ing and put my money where my mouth is hopefully by the end of the month I will have my AutoPilot either using Atair's new system or using Paparazzi with a Gumstix and the GX1 INU. I hope to use the Gumstix to process the Data from 2 Ace16K Image processors though I may need to go PC104 route though I hope I dont.

I am going to dissappear until I have my new Rotor hubs machined and a Flight Computer installed. From hear we can have some FUN!

Scott

P.S. Im selling my Futaba 14MZ if anyone is interested PM me. I need $1800+shipping for it.