Thread Tools
May 18, 2018, 04:39 PM
Registered User
barney-NG's Avatar
Thread OP
New Product

piCAMTracker F3F plane detection


Why do we need another turn detection after fxxtrack, MotCAM and FCD ?

Because it is:
  • small
  • Raspberry Pi based
  • free
  • working

This is an old project I started 10 years ago with C++ and opencv. I had some success with the help of Michael O. based on fxxtrack but I paused it because of too big leadtime caused by the USB cameras under Linux.
Then Alvaro came up with his MotCAM and I saw that it is possible to reduce the leadtime even with the Sony Eyecam. So I started from scratch with a new Python based approach on the Raspberry Pi. Soon it showed up that the Raspberry is too slow for this task. In parallel I recognized JŘrgen's FCD project who used a much faster Odroid computer. I had quite good results with this hardware using a high sophisticated tracking algorithm but the problem was still the poor performance in background separation. Getting more than 25 frames per second is very difficult and the even worse if there is much noise in the background. So the project stopped again...
One day at the slope a mate told me that the Raspberry should have a build in hardware motion detection. After digging some time I found the amazingly well written Python interface to the Raspberry Pi camera

...time for a complete restart of the project...

The motion detection of the actual solution is completely done in hardware on the Raspberry. This has a lot of advantages:
  1. low cost camera with mobile phone standard and size available
  2. direct interface between camera and GPU
  3. high framerates and high resolution possible with minimal leadtime
  4. low power consumption

With this approach the Raspberry provides:
  • a 1280x960 resolution with 42 frames per second on an old V1 camera (my preferred solution)
  • and even a 1280x720 resolution with 62 frames per second on a V2 camera
  • if very fast objects shall be tracked it should even be able to run 90 fps in a resolution 640x480 (not yet tested)
  • 10 hours runtime with a 3S 18650 battery

Actual status:
  • tracking is usable for training.
  • snapshots of the turns are provided via the build in web server.
  • rudimentary control of the device via build in web server. (Currently I don't adapt any parameters to run the device.)
  • The source code is open and you can find it on github (A reference to an image (V0.5) is included.
  • 3D printable cover box available
Last edited by barney-NG; Mar 03, 2019 at 11:55 AM. Reason: image updated to v0.5
Sign up now
to remove ads between posts
May 19, 2018, 05:47 AM
Registered User
concreteman's Avatar
Gotta love the smart guys.

Hey Barney
May 19, 2018, 12:25 PM
Registered User
barney-NG's Avatar
Thread OP

The cover box


I need to write the documentation...

What can this part do?
  • detect turns (in both directions)
  • detect cuts (currently switched off)
  • send snapshots
  • raise a signal on a pin when a turn is detected
What is not possible
  • manage a race
  • distinguish between fly in and fly out
  • detect far away objects (F3B may be difficult even if we have some testers)
  • send signal via wifi to a controller. (You need an extra device doing this.)

Olav Kallhovd (the inventor of the amazing CG measurement unit) created a cover box for the Raspberry.

All needed components should fit in the box which is more or less water resistant.
  • Computer
  • Camera
  • Battery
  • Wireless Switch

Today we are at version 0.3. (The orange box in the pictures is version 0.2 which is a bit too small and which has no openings for cooling)

Actually you just have to place the box, switch it on and direct it into the correct position.
May 19, 2018, 12:54 PM
Registered User
barney-NG's Avatar
Thread OP
Quote:
Originally Posted by concreteman
Hey Barney
Hey Warren!

How's life?

Hope to see you at Ruegen in autumn!

Barney
May 19, 2018, 04:40 PM
Team USA F3F Member
ttraver's Avatar
Barney!

Its great to have so many people making solutions now...

So, I've got a couple of questions!

1) What is the FOV of the camera? We have several slopes here in Southern California where we might fly almost directly over the camera, and cannot put the camera far back enough to be out of the way of people as well as point below the cliff. This is one reason why we are actually using 2 cameras with MotCam at each base to get at least 110 degrees of coverage...

2) Are the resolutions of the cameras getting better? Does the raspberry pi camera yet have a native HD camera?

3) might it be possible to have the unit send a simple signal over USB so we could connect other systems to it? We currently use Simon Watt's system to run our events, and although he uses a similar xbee system, it might be easier to just accept a signal from your box when a detection was made. We have worked with Simon and Alvaro to get detection directions as well as some smarts in the software to try and get rid of false positives (like timing and such).

4) You said you were using the web interfaces to see detections (and non detections). How are you accessing those?

You will definitely see me AND Warren in Germany in October for the worlds!

Thanks,

Tim
May 20, 2018, 09:08 PM
Registered User
g00bd0g's Avatar
This looks awesome! I might not be understanding, when you say "detects a turn" does this mean it sends a signal when the course marker has been passed or after the sailplane has actually made a turn?
May 21, 2018, 02:52 PM
Registered User
barney-NG's Avatar
Thread OP
Quote:
Originally Posted by ttraver
1) What is the FOV of the camera? We have several slopes here in Southern California where we might fly almost directly over the camera, and cannot put the camera far back enough to be out of the way of people as well as point below the cliff. This is one reason why we are actually using 2 cameras with MotCam at each base to get at least 110 degrees of coverage...
I am using a camera from Waveshare with a fisheye lens direct on the slope edge. The horizontal angle is 120░. The usable distance is about 5m - 50m (Finding the valid/invalid settings is the most important thing we have to solve in this practice phase)
I also used the new V2 camera with 73.8░ horizontal angle. (20m -- 200m ???)
In any case the camera is turned by 90░ so that the highest resolution is used vertically.
Olav tested today with a 130░ and a 160░ camera. He had the best results with the 130░

Quote:
2) Are the resolutions of the cameras getting better? Does the raspberry pi camera yet have a native HD camera?
Yes. The V1 camera supports 5 Megapixel and the V2 support 8 Megapixel. The camera is connected directly to the GPU of the Raspberry Pi by a 15 pin CSI connector. The theoretical bandwidth is something about 800 Megapixel. (Practically the Raspberry reaches 300 Megapixel on the CSI)

Quote:
3) might it be possible to have the unit send a simple signal over USB so we could connect other systems to it? We currently use Simon Watt's system to run our events, and although he uses a similar xbee system, it might be easier to just accept a signal from your box when a detection was made. We have worked with Simon and Alvaro to get detection directions as well as some smarts in the software to try and get rid of false positives (like timing and such).
I am quite sure that Simon just needs a raising or falling edge on a signal line. That can be delivered directly out of the box.
If you need something special on the USB port that needs 30 minutes or 35 to be programmed.
(I asked him directly by PM but he didn't answer my by now)

Quote:
4) You said you were using the web interfaces to see detections (and non detections). How are you accessing those?
I am accessing the web server which is running on the Raspberry with my Android or an iPhone.
The question is how to provide WiFi on the slope?

My first idea was: Each base creates its own WiFi access point.
Advantage:
You go to the base and connect you phone and access the web server.
You don't need any long range equipment.
Disadvantage:
You have two WLANs at the slope and you have to connect and reconnect every time you want to have a picture.

Currently I am using the following approach:
In the middle I placed a standardWiFi router. (dlink-605l)
I pimped
the Raspberry Pi with a long range WiFi antenna. So I can reach both web servers from the middle.
Advantage:
Both sides are reachable from the middle.
Disadvantage:
You lose the FCC for the Raspberrys
You have to connect t two web servers. (On for the left turns and one for the right ones)

Planned approach:
Both sides send their snapshots to a web server which is running on a third machine in the middle.
All users can see the pictures via one web interface.


Quote:
You will definitely see me AND Warren in Germany in October for the worlds!
Let's do a "real" barbecue for "real" men in that week --- please :-)
May 21, 2018, 03:15 PM
Registered User
barney-NG's Avatar
Thread OP
Quote:
Originally Posted by g00bd0g
This looks awesome! I might not be understanding, when you say "detects a turn" does this mean it sends a signal when the course marker has been passed or after the sailplane has actually made a turn?
You got it right.
It has to detect a turn because it shall detect a "cut". This means the plane has turned before the marker line.
In reality it is doing some simple tasks which are not really a "turn detection"

The easy task: (took me just a few months
The plane has passed the marker line -> beep!

The more complex task:
The plane stops progressing against the marker line. To detect this the algorithm has to measure the progress the plane makes in any of the coordinate system and it has to decide is this plane still able to do a correct pass or not. If not -> make a snapshot with a red box and send it to the web server.
When you take a closer look at the snapshot of the first message you will recognize a light grey box. This is the progress area the plane occupied during its object life time. Once the leading edge of the box is not progressing any more and the direction of the flight path starts pointing away from the marker line this must be a turn.

This algorithm is already there but it is currently disabled because it has an error.
May 25, 2018, 10:14 PM
Registered User
SimonWatts's Avatar
Hi Barney

I just got back from a holiday and saw your posts. Your system looks really interesting. I've sent you a pm with some details about connecting to my timing system.

Looking forward to seeing how well your system works.

I'll be at Ruegen for the worlds as well

Cheers
Simon
Jun 10, 2018, 04:02 PM
Registered User
barney-NG's Avatar
Thread OP

Tracker in action


Today we had a first successful training session.
The minimum distance to the edge should 5m.
The detection rate is still very impressive.

Comment from the actual world champion:
This system is too accurate. :-)

video 1
video 2
Jun 12, 2018, 03:04 PM
Registered User
barney-NG's Avatar
Thread OP

2 days in usage


Last weekend I met some friends in Denmark for a training and test session.

day 1: beach slope, 8 m/s, cloudy conditions with rain. (see short videos of the post before)

left turn: rounded edge, the camera was mounted ~8 meters behind the slope line.
- mostly 100% correct detection rate! The camera can be tricked with very near fly bys.
- the camera cannot be tricked with high passes -- all high passes where detected correctly. (thanks to the fishey)

right turn: full edge, camera ~5m behind the edge.
- we started with 3m distance. Low passes (not turns) were not detected.
- with 5m distance the low passes were still missed when the camera sees the silhouette against the shore break. (exactly from the side in front of the waves)
- "normal" turns were detected with high accuracy. (much higher than 90%; not 100% because the camera is not fixed perfectly)
- no problems with high illuminance colors

Currently the covers cannot stand a longer shower period. (I had some water coming through the front lens needs and some water coming trough the top openings. -- not much)

day 2: sea slope, 20 m/s, cloudy and sunny (mostly from the front) conditions with short showers.

left turn: rounded edge, camera ~15m behind the edge
- without fixing there are too many false positives because of camera movement.
- once we fixed the camera against the ground the false positives where acceptable. In the cloud of false positives the plane was detected always correctly.
- at the end we put the camera ~25m behind the slope in the calm parking area and there we were back to nearly 100% detection.

right turn: strong edge, camera ~8m behind the edge
- no possibility to fix the camera in a position where it had connection to the base in the middle.
- really a lot of false positives due to camera moving. (approximately every 2s a wrong signal)
- training was still possible because the turn was detected mostly.
- I tested the camera with nd filter against the sun but we had more problems with the camera movement. I will repeat the test in calmer conditions.

Unfortunately I have not much results of this windy day because our Austrian colleague hits the center base with full speed and I was not able to recover it before the end of the day.
I made just one video in the beginning. The beep sound is not well recorded because the wind is so loud.

@All: Do you have an idea to fix the camera 1 to 1,5 m above the ground. (Rocket proof in 25 m/s wind :-)

Summary:
- all people who flew with the system where impressed how accurate the signal is raised.
- actual setup is completely sufficient for training
- the setup can be handled without any technical background. (switch on and place the device, fly)
- everybody wants one :-)
Jun 12, 2018, 03:20 PM
Team USA F3F Member
ttraver's Avatar
Barney!

Great news that you are doing more testing! That is the only way to progress.

Yes, you have to stake down the tripods that the cameras are sitting on, because the detection is all about the movement, and if the camera moves, then everything doesn't work.

I have the MotCam cameras on tripods and those tripods have rope and stakes to attach them to the ground to make them stable. I also have some small tripods if I wanted to put them very close to the ground, although then you have to worry about grass waving and such.

I suppose if you used a regular large tripod, and did not extend the legs and staked it to the ground that it should be stable enough.

And yes, the planes tend to disappear at some angles and with certain backgrounds. That is part of what makes this hard...

I bought my raspberry Pi, now all I need is to get the pi2 camera and download your software and print a case and I should be able to help you test...

Tim
Jun 12, 2018, 08:08 PM
Registered User
SimonWatts's Avatar
Hi Barney

Great progress! How well does it handle moving grass and bushes in the field of view?

My pi2 camera is on its way, so hope to be able to test soon.

Simon
Jun 12, 2018, 08:38 PM
F3B and F3K
RetoF3X's Avatar
Hi Barney

Looks great! Do you think it would have a chance to capture an F3B speed run at Base B if it were pointed a bit more skywards (and the laps were flown low on the deck)?

I had used the FXX tracker years ago for F3B practice and it worked in some scenarios, but it was not practical (no Wireless, I did not have enough field of view with the camera, etc.) and too variable results.

Would love to give camera based detection a new try.

Best,
Reto

PS: still remember when we met in Laurac for F3F, fun times.
Latest blog entry: thoughts on contest flying
Jun 12, 2018, 10:08 PM
Team USA F3F Member
ttraver's Avatar
Barney,

Do you have a parts list of what you are using?

For instance, you say that you are using a fisheye lens on the Pi2 camera. Does that come with the camera? or was that an additional purchase?

Also, the lens cover for the outside case?

How do you see the video of where the camera is pointing when you turn it on? Do you connect to the Raspberry pi's wifi???

I've got some of the pieces, and may want to combine a case with one of Simon's xbee base units, so I need to get all of the parts...

Thanks,

Tim


Quick Reply
Message:

Thread Tools

Similar Threads
Category Thread Thread Starter Forum Replies Last Post
New Product Valenta Model Mini Twister 3m F3F/B Plane concreteman Slope 3 Jan 16, 2019 12:04 PM
Discussion iMax B6AC detecting 2 cells of 3. Buzzer detects all 3. FromHell Batteries and Chargers 21 Jul 04, 2017 07:56 AM
Discussion F3B/F3F planes timmig Multi Task F3X 0 Jul 07, 2016 10:35 PM
Alert Great Planes Triton Charger does not detect battery! guapoman2000 Batteries and Chargers 38 Feb 07, 2008 09:01 PM
Berg 5 DSP auto detect not detecting rickyblaze Radios 2 Oct 29, 2004 04:29 PM