SMALL - espritmodel.com SMALL - Telemetry SMALL - Radio
Reply
Thread Tools
Old May 01, 2014, 08:31 AM
Registered User
Joined May 2013
26 Posts
Build Log
A smartphone based telemetry system allowing IFR FPV

This build log covers a quite unique project: the development of smartphone software that allows to control a RC quadcopter using a primary flight display and a navigation display.

FPV not done visually but based on instruments. FPV by Instrument Flight Rules instead of FPV by of Visual Flight Rules. Or, if we take the terminology from manned aviation, IFR FPV instead of VFR FPV. You might remember this thread:
http://www.rcgroups.com/forums/showthread.php?t=1912808

Flying according to Instrument Flight Rules means that you control and navigate an air vehicle solely based on the feedback provided on instruments.

So how will the final solution, called FlightZoomer, look like?
  • One smartphone (FlightZoomer Sensorics) rests on the copter and transmits all kind of useful sensor data;
  • A second smartphone (FlightZoomer Groundstation) acts as display for the pilot. The information on the screen covers:
    ○ Primary flight information
    ○ Navigation map
    ○ Boeing Dreamliner style of the cockpit instruments;
  • Control is still done via existing RC transmitters;
  • 100% software solution, all hardware is COTS.
What requirements can be covered with such a setup?
  • IFR FPV as described above;
  • Premium telemetry features;
  • (Manually flown) flightplans;
  • Fly away safeguard. Flights can be tracked beyond the range of the RC transmitter or any other FPV equipment;
  • Presented information superior to any OSD, e.g.
    ○ Navaids
    ○ VOR simulation
    ○ Deviation from flight plan
    ○ Flight director (displays commands, how the copter has to be flown manually to follow the flightplan);
  • Technically not limited to be operated under visual flight conditions;
  • Technically only limited by the range of the RC transmitter;
  • Optionally the camera on the smartphone can be used to record videos or images and, in the future, transmit snapshots of what's going on up there to the groundstation.

Limitations observed so far:
  • The latency of the transmissions requires strictly that the copter principally must be stabilized by a flight controller like NAZA or SuperX. Any other gear would be suitable as well, as long as:
    ○ Position hold capability is similar
    ○ There is a flight mode, like NAZAs GPS mode, which basically allows to command flight vectors. The movement of the copter must be predictable enough, to bridge gaps up to one or more seconds without feedback;
  • The throttle should be spring loaded in order to get predictable altitude behavior (as long as I don't move the stick, the altitude should be kept);
  • From all the typical smartphone sensors the compass seems to provide be the most challenging output regarding accuracy. So getting an accurate direction while flying with Flightzoomer requires training.

Project background:
Years back, while flying with PC flight simulators I learned basic IFR skills. I don't know how many times I have repeated that silly flight simulator training session, where the FS98 737 had to be flown towards a glideslope, capture it and then follow it all the way to touchdown.

As a kind of impossible dream I have always wondered whether and how it would be possible to operate RC aircraft solely based on instruments. I could imagine a number of approaches that require lots of self-made electronics. But that would have required far too much work and research effort for my taste. So for many years the dream remained a dream. It was not before a year ago when before my inner eye I suddenly saw technologies and products aligning along a chain, that would make possible the impossible. These would be the ingredients:
  • A normal quadcopter with a flight controller that has NAZA-like capabilities. The NAZA would provide elementary features like stabilized flight, safety fallback (RTH).
  • Smartphones, which:
    ○ …essentially are supercomputers compared to anything else flying onboard RC aircraft;
    ○ …have a full suite of sensors, including GPS;
    ○ …have long endurance, redundant battery power;
    ○ …offer HF connectivity, with unparalleled capabilities (worldwide range for all practical reasons);
    ○ …allow state-off-the-art graphical user interfaces (touchscreen);
    ○ …cost much less than any other comparable hardware (after my first onboard device broke I got an excellent second-hand smartphone for 40€ in EBay);
    ○ …can be programmed with the most modern and capable tools and software engineering frameworks.

What has been achieved so far?
  • Flightzoomer Sensorics, Flightzoomer Groundstation work on my test devices;
  • Communication solution (called EartConnector) implemented and operability proven;
  • Numerous flights done, steadily got closer to meet the requirements above;
  • Various trials to achieve peer to peer communication between the two smartphones have been conducted, but none succeeded. So currently a relay server is running at home, to which both peers connect.


Still outstanding:
  • Debugging various cases
  • Refine different existing use cases
  • Add more features
  • Image transmission to the groundstation
  • App certification completely uncovered

Personal background
I am an electrical engineer who today is in the professional business for software development.

I will post some images and screenshots as soon as I have them ready...
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Sign up now
to remove ads between posts
Old May 02, 2014, 01:16 AM
Registered User
Joined May 2013
26 Posts
Updated, added images

Hello
Have a look at some images from my FlightZoomer setup:
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Old May 02, 2014, 02:46 AM
FPV Fan
Australia, QLD, Brisbane
Joined May 2011
9 Posts
Are You F******* Joking?

Are you aware of the aviation law and regulations that cover IFR flight in real aviation? Also you need to understand the difference between VMC/IMC/VFR/IFR. Technically all FPV flying is seen in the eye of authorities as IFR because you are not in the aircraft seeing the real picture.

I'm a full-time pilot who regularly flies IFR through IMC conditions (
not at 30,000ft either, but in helicopters at published safe altitudes, as shown on charts which can be as low as 1500ft), and the last thing we need is idiots like you trying to fly models IFR in bad weather without even the slightest idea or concern for aviaiton.

Any aircraft to fly IFR must meet a minimum equipment requirement, in nearly all cases be in radio contact, and have filed a flight plan, even in Class G open FIR.

Could you imagine what would happen if you are pissing about with your FPV in cloud, not knowing what else is in there?!!

Aviation works on the main rule of "See and avoid". When that is not possible, there are laws, rules and regulations to protect the aircraft. Being a software engineer with a passion for model aircraft does not give you the right to endanger public aircraft.

I myself fly FPV for fun, so I'm well aware of the equipment and telemetry available, but also I know the concerns of Aviation authorities, and their major concern is people like you.

I'm sure aviation authorities are aware of this forum (at least I know my Ops Inspector and the Authority's UAV department are).
mjb5 is offline Find More Posts by mjb5
Reply With Quote
Old May 02, 2014, 04:32 AM
Registered User
Joined May 2013
26 Posts
Quote:
Originally Posted by mjb5 View Post
Are you aware of the aviation law and regulations that cover IFR flight in real aviation?
Yes, and FlightZoomer does not violate any law. It is not intended for IFR more than your FPV aircraft. The only difference is, that the aircraft is controlled based on the instruments and not visual feedback. You can do that perfectly well (and should do it of course) at distances and altitudes that are safe.

It does not mean that I advocate to do something actually, when I talk about what is technically possible. No public aircraft was ever more endangered by my flights than by your FPV flights.

Of course it might be possible to misuse FlightZoomer in a way, that laws in some countries potentially would be broken (depends also on the country). But it is certainly also possible, to use it in full compliance with any law and regulation. And that is what I use it for.

FlightZoomer is in that regard not different than 99% of the other long range RC/FPV equipment out there. In fact I don't consider FlightZoomer as something for long range RC flying...

In the end it depends on the pilot. There is also tons of other FPV and RC stuff that has the potential to break laws or not. So is live. Cars, motorcycles, cameras, knifes, computers, smartphones, umbrellas, .... there is a tremendous amount of stuff which can be used to break a law ... or not.

Quote:
Originally Posted by mjb5 View Post
I myself fly FPV for fun, so I'm well aware of the equipment and telemetry available, but also I know the concerns of Aviation authorities, and their major concern is people like you.
I can't see the difference. I certainly did not go against any law and regulation with FlightZoomer.

I could remove FlightZoomer from my copter, take off and within 1 minute laws would be broken. But I don't do that. And I also don't do it with FlightZoomer onboard...
FlightZoomer is offline Find More Posts by FlightZoomer
Last edited by FlightZoomer; May 02, 2014 at 07:21 AM. Reason: For clarity reasons
Reply With Quote
Old May 20, 2014, 03:09 PM
Registered User
Joined May 2013
26 Posts
Update: flight simulation

Hi
Meanwhile I have added a simulation feature to the Relay Server. See this video to find out how it works:
FlightZoomer Demo I (5 min 31 sec)

Explanations:
- The video shows everything in realtime.
- The smartphone, which would be located with the pilot, is only represented by the developper emulator (on the right side).
- The display and user experience however is a full representation of the real device.
- The simulation feature is part of the FlightZoomer Relay Server (the program on the left side).
- In real world the Relay Server must run on a PC in order to connect the sensorics-device with the groundstation-device.
- The FlightZoomer Sensorics device is not shown in the video. The flight trajectory simulation takes its part.

- The displayed information on the FlightZoomer Groundstation shall be used to control the aircraft manually. Very similar, like pure OSD data would be used (but better IMHO).
- Primarily you have the countdown timers that tell the pilot, how long he shall fly straight or how long the turn will go on.

Question:
Do you think this solution is more a telemetry system or kind of a OSD system?
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Old May 20, 2014, 05:11 PM
Registered User
Joined Oct 2013
34 Posts
Wow mjb5 what a hostile person you are.

Someone spends their time developing something that looks really good and without another thought you rip in to him as if you are the law.

There was no need for it and it would be just as easy for someone to violate the law with a standard RC airframe.
multiplexer is offline Find More Posts by multiplexer
Reply With Quote
Old Jul 15, 2014, 07:35 AM
Goooo!!!! Get to da choppa!!!!
United States, NC, Albemarle
Joined Jan 2013
498 Posts
Quote:
There was no need for it and it would be just as easy for someone to violate the law with a standard RC airframe
No, not really. mjb5 came across a bit strong, but for good reason. I flew full-scale helicopters too, and can assure you that the FAA isn't the only group of people with no idea what the other group is doing. When you're flying along at 100 knots, a crow can come through the windshield onto you. You'd better believe a DJI450 will do much more damage. Until you've actually been forced to make an emergency landing after a bird strike, you can't appreciate mjb5's position. I can assure you that the fear of losing your life just one time makes an impression far deeper than simply an organization placing rules on your hobby.

I very much enjoy FPV, and am a fan of some of the inventions I've seen here. Unfortunately, I've seen simple electronics fail- meaning just because you CAN do something does not mean that you should do that thing. Some type of failsafe should be used with this system.

The "Flight Zoomer" system is very interesting, but I personally would never use it BLOS. The admitted latency alone is reason enough for that IMO. On the other hand, I hope Flight Zoomer continues to test and refine the system. The kind of potential it has is very interesting.
kpinnc is offline Find More Posts by kpinnc
Reply With Quote
Old Jul 17, 2014, 11:07 AM
Registered User
Joined Sep 2010
62 Posts
Control Your Temper

I too fly Enstrom helicopters and models of all types,, so am embarrassed at the intemperate language of the pilot mjb5 quoted above. Not only has he failed to understand the project, but failed to ask questions first before making his opinions clear.

Sadly we have to share airspace with these people. As he's in Australia perhaps it's fortunate that it's unlikely he'll cross the path of any reasonable person elsewhere in the world. People who cannot keep a steady temper in the cockpit need to be grounded permanently. We don't need hotheads flying full size or models.
michaelpowell is offline Find More Posts by michaelpowell
Reply With Quote
Old Jul 19, 2014, 09:26 AM
Registered User
United States, MD, Annapolis
Joined Jun 2013
2,384 Posts
Nice demo and development started. Excited to see how this will work.
Waterbound is online now Find More Posts by Waterbound
RCG Plus Member
Reply With Quote
Old Jul 19, 2014, 01:37 PM
Registered User
Joined May 2013
26 Posts
Quote:
Originally Posted by kpinnc View Post
I've seen simple electronics fail- meaning just because you CAN do something does not mean that you should do that thing. Some type of failsafe should be used with this system.

The "Flight Zoomer" system is very interesting, but I personally would never use it BLOS. The admitted latency alone is reason enough for that IMO. On the other hand, I hope Flight Zoomer continues to test and refine the system. The kind of potential it has is very interesting.
I would absolutely agree that there should be a failsafe layer. And you are also right, that the latency indeed requires a flight controller, which keeps the aircraft stable while flying.

Ideally FlightZoomer would be operated on a quadcopter with NAZA or SuperX (with GPS, altitude hold and RTH). The RTH of the flight controller would be mandatory IMO, to cope with a complete outage of FlightZoomer. And - of course - FlightZoomer should always be used under the local legal regulations. Which means that line-of-sight between pilot and copters must be given anyway in most of the countries.

When I flew my F450 with FlightZoomer my confidence in the failsafe modes (of the RC, NAZA) were so good, that I was not worried doing testflights with the phone battery so low, that outages could have happened at any time.

I would also argue that FlightZoomer would preferably be used in addition to an already existing FPV/OSD transmission link, so FlightZoomer itself would primarily be the backup...
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Old Jul 22, 2014, 03:02 AM
Registered User
Joined Oct 2004
1,252 Posts
Quote:
Originally Posted by multiplexer View Post
wow mjb5 what a hostile person you are.

Someone spends their time developing something that looks really good and without another thought you rip in to him as if you are the law.

There was no need for it and it would be just as easy for someone to violate the law with a standard rc airframe.
+1.
neavissa is online now Find More Posts by neavissa
Reply With Quote
Old Jul 22, 2014, 03:08 AM
Registered User
Joined Oct 2004
1,252 Posts
Unless I AM not getting it, what is all the fuzz about?!

People fly FPV, right?! They use a camera for navigation and in order to see where they are flying at, right?! Now instead of using a camera, FlightZoomer wants to use instruments, right?! What the heck is the difference then?! Sure, with a camera you are able to see any "oncoming" traffic, but who says you are flying in any Airspace?! Anyhow, just my 2 cents...

Thanks,

G

p.s. MJB5, stop calling people an Idiot, you Idiot.
neavissa is online now Find More Posts by neavissa
Reply With Quote
Old Aug 03, 2014, 08:21 AM
Registered User
Joined May 2013
26 Posts
High Level System Overview

Meanwhile I'd like to show you the system components and use the splashscreen of the Sensorics App for that purpose:

As you can see there is a closed control loop:
  1. The copter, on which a smartphone is mounted that runs the app FlightZoomer Sensorics. It transmits all sensor data like position, course, speed, altitude and attitude to the ground.
  2. A computer which runs the application FlightZoomer Relay Server. This PC can be located at home and does need supervision while it runs. The Relay Server forwards the information to the groundstation, stores the data of a flight into files, allows replaying such files, as well as simulating any other flight movement. Also the navigation aid database is stored on and accessed via Relay Server.
  3. A smartphone which runs the app FlightZoomer GroundStation, which is used by the pilot as display. This app resembles the cockpit of a real Boeing 787 Dreamliner and allows operating the copter lie a real jet would be operated. There is a Flight Management System, (simulated) Radio Navigation, a PFD (Primary Flight Display) and a moving map ND (Navigation display). The TERRAIN mode of the real 787 cockpit is emulated with a Google-map kind of display that shows aerial images wherever the copter goes.
  4. The pilot interprets the information on the display in order to ...
  5. control the copter via a regular RC transmitter.
Current state of the project:
I am awaiting better weather to make more test flights. Some parts around the FMS still appear a bit buggy. Feature completness for a first release is 98% I would say...
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Old Aug 05, 2014, 07:29 AM
Registered User
Joined May 2013
26 Posts
Radio Navigation

Hello

Today I'd like to present you another FlightZoomer feature: Radio Navigation!!

The radio navigation feature allows to approach radio bacons (called VORs in real world; VHF Omni Directional Radio Range). Check here for more informations:
http://en.wikipedia.org/wiki/VHF_omnidirectional_range

As in reality with FlightZoomer the radio range is not just directly approached, but on a predefined radial. The procedure is performed by keeping the current course until the predefined radial towards the VOR is crossed. Shortly before that happens a turn is initiated, in order to "capture" the radial.

The user interface and operating is closely modelled after the real Boeing 787 Dreamliner cockpit.

This kind of navigation (from VOR to VOR) is nowadays mostly obsolete for large civil aircraft. LNAV is the keyword in modern aviation (which is simulated by FlightZoomer as well, see the first YouTube Video).

Nevertheless Radio Navigation allows to learn a lot about IFR flying. Master Radio Navigation and you've got basic IFR skills!

FlightZoomer Radio Naviation Demo&Simulation (5 min 21 sec)


Here the transcript of the steps, that are shown in the FlightZoomer Radio Navigation video:

00:04 First there is a navigation aid database, which is stored on the PC where the Relay Server Application runs.
00:05 At the top the syntax for a navaid record is shown.
00:09 We will approach the VOR HOM in this demo, and then turn to ....
00:15 ...NS1.
00:17 The selection of the navaid in the GroundStation app will be done using either the ID or the radio frequency.
00:25 Open the Relay Server application.
00:27 and then show the GroundStation Emulator (this demo is fully run on the development PC).
00:29 We can spot HOM a the top and to the right on a radial of about 56° from our current location.
00:31 Straight to the South from HOM we can see NS1.
00:41 Instead of going straight towards a radio beacon, usually a designated radial is captured. For HOM we aim for the 75° radial.
00:59 Lets go the Flight Managment System (FMS) Panel. Radio Navigation settings are made on a subpage of the FMS on the Boeing Dreamliner (which is simulated by FlightZoomer). Earlier Boeing models had dedicated dials for those settings.
01:01 Open the NAV RAD page.
01:04 There are two radios available for navigation (VOR L and VOR R). Dial in the ID of the first navaid (HOM)....
01:09 ...and select it as VOR L by pressing the top left Line Select Key.
01:15 The same way select 75° as course towards VOR L. The system will guide the copter to approach the beacon on a 75° course. This means we will fly first on a course more to the north than the direct course and then, when the 75° radial is crossed, turn right towards the navaid.
01:18 The second navaid is dialed in using its frequency (110.7, as defined in the database file).
01:28 The course from HOM to NS1 is 180° (directly southwards).
01:35 Go back to the DISPLAY CONTROL panel.
01:42 Let´s see what keys we have at the top and the left side of the navigation display:
01:48 Selecting VOR L shows the direction pointer and the deviation indicator from the selected radial toward VOR L.
01:51 We see the 75° course...
01:53 ...and the deviation indicator, which is fully deflected to the left, which means that the current position is to the right from radial.
02:04 Selecting VOR R allows to display the direction pointer and the deviation indicator for the data, that has been entered for VOR R (NS1, 180°).
02:36 On the bottom we get some raw data of both activated radio navigation aids. The indications show the navaid ID and the DME value, which is the distance to the navaid (here in Meter; but can also be feet).
02:47 At the top right (in white letters) the currently selected navaid and some related data are shown (again DME, but also the direct course).
02:51 With two buttons on the instrument frame on the left the previously defined course can be altered. With +2°/-2°-steps a more suitable radial can be selected on the fly, if needed.
03:13 Go to the Relay Server, from where the actual flight simulation is triggered.
03:20 Apply forward speed, in order to start the initial leg.
03:39 Watch the deviation indicator slowly creeping to the center! This means, that we approach the radial which we want to follow to the navaid!
03:46 At the right time start turning right.
03:57 Look, how precisely we captured the radial towards HOM (this requires more skills controlling a real RC copter)!
04:01 As the first navaid is approached, switch to the second by pressing VOR R. This step prepares the right turn for the second leg.
04:21 The deviation indicator starts moving, soon the next right turn is needed.
04:25 Now start turning on a 180° course.
04:38 We ended up a bit late (= to the left of the 180° radial). Let's correct by quickly applying a 190° course.
04:42 It is an overcorrection, because we are already very close to the navaid. Close to the navaid, the deviation indicator starts to become over-sensitive and is hardly usable anymore.
05:12 At DME 7 (7 Meter distance from NS1) we stop our demo by applying zero speed.
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Old Aug 18, 2014, 02:38 PM
Registered User
Joined May 2013
26 Posts
FlightZoomer offers Bitching Betty? Or Barking Bob?

Meanwhile I have added a voice command feature at the groundstation...

As explained earlier, FlightZoomer does not automatically control the copter. You control it via RC transmitter and the presented information is just used to "navigate" and to a limited extend to "control" the copter.

So if you follow a built-in flightplan, the system basically tells you, how many seconds you have to fly straight. This includes a countdown indicator, telling you the remaining seconds of the leg as well as the remaining seconds of each turn.

In a very basic way, this represents the Flight Director of a real aircraft. In real aviation the Flight Director shows the pilot the exact manual input, he has to provide in order to follow a certain, predefined flight path.

And here comes the new voice command feature!!! As easy as it can possibly get, on the Smartphone I can run a TTS engine (Text-to-speech) in order to guide the pilot. So from the flightplan tracking engine in the groundstation I let the phone speak out the various commands audibly.

Stay tuned, I will soon post a video, where the feature is demonstrated on the emulator!

And what has it to do with "Bitching Betty" or "Barking Bob"?

These two slang names are used by pilots refering to voice systems in real cockpits. The voice of female warning systems is called "Bitching Betty" (e.g. in the F-16, Eurofighter) and male systems are called "Barking Bob" (e.g. aboard Boeing commercial airliners)!

http://en.wikipedia.org/wiki/Bitching_Betty

So, as FlightZoomer models the Boeing 787 Dreamliner cockpit, I call the feature "FlightZoomer Barking Bob"!
FlightZoomer is offline Find More Posts by FlightZoomer
Reply With Quote
Reply


Thread Tools

Similar Threads
Category Thread Thread Starter Forum Replies Last Post
Question FPV base system help flywmoi UAV - Unmanned Aerial Vehicles 1 Mar 08, 2013 09:14 AM
Discussion DIY Telemetry System base on TI CC110L RF BoosterPack spy004 DIY Electronics 2 Mar 06, 2013 07:58 AM
Discussion Land Based FPV System (what's best)? tsull FPV Talk 12 Feb 26, 2013 03:52 AM
Idea android based fpv/uav system baconflyer FPV Talk 2 Sep 22, 2012 03:16 PM
Discussion Radio telemetry system - what does it actually allow me to do moglos UAV - Unmanned Aerial Vehicles 5 Sep 09, 2012 11:07 AM