Thread Tools
Apr 01, 2008, 11:06 PM
Registered User
Thread OP

Alternate System Partitioning

One of the guys over at DIY drones hit on something that I think more should consider given the complexity we're facing: A smartphone in the cockpit as the autopilot. He had used an RS232 board to control the servos, but I've done a subtle tweak on that I thought I'd share. No, it hasn't flown yet, but hopefully I'll start some flights in the coming weeks as the snow melts.

The premise is that you put the servo control, 2 axis gyro, pressure sensor, small micro, current monitoring and a bluetooth on a small 1.5" x 2" board. Then you bluetooth the realtime status over to a larger smartphone or PDA and control everything from there. This small PCB goes in-line between the battery and the ESC.

Why partition this way? First is that a modern smartphone/PDA uses a chipset developed by 300-500 people. It is optimized for cost, size and power consumption while delivering an ARM11 at 500+ MHz and floating point support. They are standalone units complete with GPS, 3Mpix camera, cellular radio, SD cards. They run OSs that were developed by another 600+ people, and use an IDE that was developed by another 800 people and support modern languages with rich libraries. You can find all this in ~100 grams, with a self contained battery that is good for 3-4 hours of 100% computing. With lesser use it can of course run for days.

Yeah, you can roll your own. And if its fun for you, then great. But I want to fly and refine the higher level features, and a closed product doesn't let me play with the source, and the open source projects are too complicated to play with on the weekends because there are 10's of thousands of lines of critical code that you really need to understand before you dig into the system.

Smartphones run for weeks without a crash. Microsoft, for example, requires as part of certification that 30 devices run stress tests for more than 20+ hours, equivalent to months of normal use. Like with a PC, if you have good drivers and don't treat the system as an amusement park, then reliability just isn't an issue.

To me, the best approach is to offload the absolute minimum to custom hardware, and do the rest in off-the-shelf hardware.

Attached is a picture of a pcb. It has a an IDG300 dual axis gyro, SCP1000 pressure sensor, MSP430F2274 uP, a Roving Networks bluetooth module, and a DS2756 fuel gauge. The RC RX connects to the top of the board on the right, and the servos connect to the bottom of the board on the right. The bluetooth lets you connect to just about any cellphone or PDA device that you might find. Symbian and MS Windows are of course big candidate devices, but a host of Java phones might also be suitable.

Attached also is a screen shot of an app that receives 50 packets/sec from the board over bluetooth. Each packet contains current servo pulse width and gyro sample. The gyro filter has a first-order RC filter with corner at 7 Hz. On the screen shot you can see the servo data (all railed because the transmitter isn't on), the absolute pressure, the derived altitude, supply voltage, current and accumulated current, temp (from the uP), gyro counts from the ADC and the estimated bank angle based on gryo. In the plot, you see the roll axis gyro output. The leveling PID based on gyro will happen in the uP, but the GPS-based corrections will happen in the PDA. The app is written in c# an moves back and forth between the desktop and PDA with ease (minus the graph).

Curious how to get your current GPS location on a smartphone? In C#, you simply write two lines of setup code, then an event hander gets called when your location is updated. In this case, we'll time stamp the location and save it to an SD card:

void GPSLocationChangedUpdate(object sender, LocationChangedEventArgs args)
if (args.Position.LongitudeValid && args.Position.LatitudeValid)
sw.WriteLine("Time: " + DateTime.Now.ToString() + " Longitude: " + args.Position.dbLongitude.ToString("f5");

If you've done any embedded programming, the above can take days, or weeks even if you have existing libraries. On a PDA or smartphone, it's minutes, and you can readily use 32 or 64-bit floats if you want. Because of bluetooth, there are no wires between the smartphone and the rest of the setup. Some might be wondering about the logic of putting another radio link in the air. Bluetooth is a hopper and will happily co-exist with other hoppers or direct sequence schemes such as Spektrum. Everyone uses bluetooth everyday in their car and with headsets...reliability shouldnt' be an issue. But of course I'll learn this in the coming months...

All in all, I hope to keep things at about 1200 lines of C code total on the small micro, and build everything else on a PDA on top of that. No, I'm not looking to sell this. I'll share schematics and source once it has settled down a bit. I'm mostly just keen to get folks thinking more about this approach. It feels right, and the hardware is evolving so quickly right now. 3G operators are allowing video calls...imagine being in a video call with your airplane, seeing what it sees with 200 mS latency, all the while seeing live data streaming back and overlaid on terrain maps that are downloadign in real time while logging everything to an SD's all doable...and it all fits in your pocket...

PS. The IDG300 gyros are amazing. I'd done a fair bit of simulating looking at different ADC widths and sample frequencies. I was a bit worried about the 10-bit ADC, but after playing with these I'm feeling much better. The concern is that slow rolls aren't picked up (that is, those below 1 lsb). But after playing with this I think it'll be fine. The pressure sensor is also very cool.
Sign up now
to remove ads between posts
Apr 01, 2008, 11:29 PM
Chris Anderson
Sweet! I'm the guy who did the original Windows Mobile autopilot on DIYDrones, and my hat's off to you--this is way better.

We did another form of partitioning, in the sense that we let the smartphone do GPS navigation, communications and imaging, while a stand-alone FMA Co-Pilot did stabalization. The smartphone controlled the rudder, while the Co-Pilot controlled the ailerons and elevator. No communications or connection between them, so the plane kinda skidded through its waypoint turns as the rudder fought the auto-stabalized ailerons.

Your method is a lot more graceful. I found it difficult to keep up with all the smartphone OSs and phone SDKs, and eventually we stopped developing the code at Windows Mobile 4 on the HP iPaq 6515. If you can come up with a portable code base for Windows Mobile [X] and Symbian [any flavor] you will be doing us all a huge favor!
Last edited by zlite; Apr 02, 2008 at 09:42 AM.
Apr 02, 2008, 06:53 AM
FrankC's Avatar
You might want to look into the .Net environment, the compact framework is being ported to multiple platforms. I did see an announcement of compact framework for the symbian operating system not too long ago. The idea here would be one code base that would run on any device running compact framework. One other option would be programming in Java since it does run on a wide variety of systems.
Apr 02, 2008, 09:45 AM
Chris Anderson
We did use the compact .NET framework. But at the time we did it (a year ago, but an even older phone) we still needed phone-specific SDKs.
Apr 02, 2008, 10:27 AM
Registered User
Thread OP
Things have improved a lot in the last few years in that you don't need phone specific libs anymore for things like taking pictures or accessing the GPS. But where things are still a bit uncertain is, for example, what resolution image you'll get from the camera. I've also noted some early issues with the system bogging down a bit when dumping a 3Mpix image to SD.

But, things like sending a text message, making a data call, using wlan, connecting bluetooth, etc, are all the same no matter what smartphone or PPC you are on.

The .net compact framework is indeed being ported many places, and I looked hard at an ARM7 or 9 that supported it. But I didn't want to lay out and bring up a board like that. Something like the gumstix, with bluetooth (aleady there) and a cellular modem (not yet available) and GPS (not yet available) would be a real solid candidate for eliminating the PDA (and saving weight but probably not cost). I also looked really hard at the various 900 MHz dedicated radios and that's another can of worms I didn't want to open as cellular gives you all the range you'll need without resorting to increasingly exotic antennas.

The other down side side of the c# on smartphone today is that it requires visual studio standard edition, which is about $140. But historically microsoft has done what they can to make the tools nearly free, so perhaps that will change. Luckily, most probably have these tools anyway.
Apr 02, 2008, 11:58 AM
Registered User
Interesting approach. Seems to make a lot of sense.

Keep us posted as to how well it works.

What frequency range does bluetooth operate in? Same as the spread spectrum receivers?
Apr 03, 2008, 09:27 AM
Chris Anderson
Originally Posted by EpicAdventure
What frequency range does bluetooth operate in? Same as the spread spectrum receivers?
Yes. 2.4gHz
Apr 17, 2008, 01:06 AM
Registered User
Thread OP
Last weekend was our first nice weekend in a while, so I took the system up for a flight. Nothing was being controlled by the PCB or the PDA; it was more of a test to see how a bluetooth link worked alongside the spektrum link. No problems noted. The PDA was inhaling every frame of servo data over the bluetooth link and sampling both gyro channels at the same 50 Hz rate, as well as pressure, voltage, motor current, motor energy, gyro data, temp and a 500 mS rate.

The cool part is that the PDA can now speak (over the speaker) the parameters you wish to monitor, AND it can relay that info down to another PDA via a text message so that while you are flying you can hear the battery voltage, speed, heading, altitude, etc, being spoken to you every 15 seconds. So, while you are on the ground doing a range check, the PDA is chattering away how many satellites it sees, voltage, etc. Then when you are moving over 5 MPH, it starts to send a text message to the ground based PDA with the same parameters. Hearing your power and altitude being spoken to you while you are flying is very cool.

Code size on the MSP430 will remain around 1200 lines. All the heavy lifting is going into the PDA. MSP will run the fast inner control loops on gyro and pressure, and manage the servos. The PDA has the GPS and the smarts. The control is a small part of this for me. I'm more motivated by moving maps in teh hand and getting all the telemetry back in near real time in a setup that can be easily pulled out of a pocket.

Next exercise will be alt hold. That's a few weeks off due to work commits.

There arew two pictures to share. The first is the easystar with a PDA velcro'd to the top. Because the link is wireless (bluetooth) to the small PCB, and because the big PDA has its own battery, camera, GPS, etc, it can really go wherever you have room. Teh easystar is a pig in this config. This PDA is a 190g mobile office, complete with power point and excel (no joke). But it's fine for learning. AUW on the easystar is 884g. The Mega 16/15/5 isn't ideal for a plane this weight, but it fits nicely. Also, there are much smaller 90 g PDA and smartphone that will do the same thing. And looks like ATT is now offering 3G video calls.

Some data from the Sunday AM logs allows a crude plot to be made of ascent rate versus motor Pin after importing the logs into excel and averaging over 30+ seconds at a time. Roughly the easystar needs about 50W of motor Pin for level flight. That's about 25W/pound. I wish I could say it climbs nicely at 180W, but the motor is too hot on the prop too small for this weight.
Apr 17, 2008, 06:57 AM
Nemo in a blender
Connexxion's Avatar
Man this is awesome!

This is what I call "off-the-shelf"!!!

I finally got a reason to buy one of those nifty PDA's!

Looking forward to following updates.

May 07, 2008, 12:52 AM
Registered User
Thread OP
A few updates...

First, have a look at this PDA: It features a 3Mpix camera with mechanical focus, 4Gbyte of internal storage, GPS, 3G cellular, video telephony, and it weighs 110 grams. A really good candidate for on-board controller when it arrives.

This evening I flew the easystar with the elevator loop closed. When I flipped the gear switch on radio, it attempted to hold the current altitude. As you can see from teh graph, the gain was too high. But I'll tweak and try again later this week. The graph shows AGL altitude (barometric) in meters on the left axis, along with stick position (0 to 100%) on the left axis. On the right axis you see the correction applied to the elevator (+/-360 uS of servo adjust), heading (deg) and turn rate (deg/sec). I'll make the gain adjustable via the PDA so I can tweak right on the plane if needed for more testing.

I've done a fair bit of profiling of .net compact framework. Suffice to say I continue to be very enthused. Recall the small custom hardware board is shoveling over 4 channels of servo positions and gyro data 50 times per second. Voltage, current, pressure, etc, are coming over at slower rate (about 2 Hz). The PDA grinds on the data, and ships adjustments back to the custom hardware board 10 times per second. The custom hardware board uses the gyro info to close a very short term loop (about 160 mS total impulse length with 20 mS sample interval, or 50 times a second) for pitch and roll stability. I fly near 4000 foot mountains so techniques that others are using (such as the great work on Attopilot) would be difficult since the horizon can't be seen except for about 60 degrees of the 360 panorama.

In profiling the .net CF code, the main control loop, using doubles, is taking 1012 uS. Decodign bluetooth packets is taking 2011 uS. This is all without optimizations, so I'm pretty confident this is a viable way forward.

I've done some experiments taking photos on the PDA programatically while running the outer control loops and it seemed to work fine. Additionally, I'm logging just about everything I have at 50 Hz, so the system is being asked to do a lot and not visibly complaining yet.

In any case, graph of tonight's flight is attached, along with current schematic as jpg. All the GPS math routines are in and debugged on the PDA, along with the communication with the ground station. I'll hopefully get the elevator loop respectable this week, the roll loop respectable next week, and some simple "fly back home when I toggle the landing gear switch" the week after that.

If you look at the schematics and want to build a board, I'm happy to send you the current source. I had the boards made at Silver Circuits, and had a few assembled by Lil-Brother LLC here in the US. They know the design, and could probably assemble a few for you for under $100 if you provide parts. Part cost is about $50 for the gryo, $20 for the pressure sensor, $3 for the uP, $40 for the bluetooth module, and probably $20 for everything else. Lil-Brother folks are good guys. The PDA code might be a bit more of a challenge, as it uses source from around the net without a clear license, but we can probably work something out. No, this won't ever be turn key, but if you are looking to get deep into the inner workings, this is (yet another) way forward. But I like the think the PDA elmiinates an enormous amount of complexity that is present on every other auto pilot project. I think it is indeed feasible that you'd never have to muck around with the 8-bit micro code. Instead, you could develop directly on the PDA using world-class tools.
May 20, 2008, 01:39 AM
Registered User
Thread OP

Alt Hold working using gyro and baro

Two updates.

The first is that I'm at a reasonable point in the pitch and altitude hold problem that I can move onto the roll and direction loops.

In the first plot, you'll see a flight in which altitude was held reasonably (within 10 meters) in light winds but with steep turns AND with zero coupling between rudder and elevator. In other words, the plane had no idea it was turning. This is a challenge because as a plane banks to, for example, 35 degrees, the lift of the wing drops 20% and the nose will usually tip down. Even on the easy star is doesn't like to correct itself until it has accumulated a lot of speed over several seconds. When you know you are turning, some of the rudder or aileron can be coupled into the elevator to make this less of an issue, especially on steeper banks. So I'm looking forward to a bit of that too.

There are two loops currently running for the elevator, and I've been able to move the gyro loop off the microcontroller and onto the PDA. The PDA and C# continue to really impress me. I really hoped I could do the elevator with just altitude, but not such luck. The turns were the killer. But it was also possible to get into a nose up position and not realize it until it was too late. The subsequent corrections were not subtle. I've thought a lot about how I fly, and especially I've looked at data of my manual flights and it's amazing how little the sticks need to move to stay level. As you look at the plot of the data, keep in mind that 10 uS of change in elevator is about 1 click of trim. The pink trace is the elevator servo (min is 1 mS, max is 2 mS, althought the axis values aren't clear for this one trace), and generally it just wanders up and down one click over a second. During the two minutes shown, the total span of control is about 4 to 5 clicks of trim. My previous efforts were much more aggressive. Intuitively it seemed that having a computer almost going rail to rail on the elevator would be a good thing. It's not. I can tell you it makes the ESC really, really hot. Of course.

The pitch and roll gyros are sampled at 50 Hz, and sent to the PDA over bluetooth. The PDA extracts an average that eventually is comprised of several minutes of samples (to account for gyro drift) and then uses a leaky integrator to form an estimate of pitch. The assumption here is that over several minutes your average pitch will be 0, and that you won't have a "significant pitch departure" away from 0 for more than 10 or so seconds (if you do, it eventually "leaks" down to zero and then when you correct yourself you are in trouble). Based on all the log data collected so far, I think these are both reasonable.

In the graph, you'll see the current parameters of interest. Some signs are wrong on the display because I've opted to adjust everything so that when I see it going up, it means the plane is also going up. That doesn't hold across the board, but it helps in viewing this stuff.

So it's onto heading and bank hold, using the same approach.

About half way through this, I realized that even being 10 minutes from a flying field meant there was signficant turn around time to figure out a sign was wrong (I had my first wasn't serious but I tend to test low because I like to see what it's doing (50 to 100 feet) and tend to want to see as much data as possible, so I didn't flip back to manual control in time...PDA broke loose, but nothing broke).

Thinking more on this, I ended up (again) on the DiyDrones site, and read about a guy using FlightGear to test his AP. After downloading flight gear, and seeing the web-based interface, I was controlling altitude in about 20 minutes from a simple app. In another 20 minutes, I had my PID loops, GPS lib and math libs pulled over from the PDA. Another few hours had the graph and some clean up. So, I'm also including an early shot of a standalone autopilot that can control any of the full-sized models in the FlightGear simulation. This is indeed an excellent platform to workout the waypoint math and sequencing. My hat is off to those guys as the interface (http-based) is awesome, albeit a bit taxing if you want to pull every little bit over.

You can see a screen shot of the app below. You can set parameters on all 4 loops (baro alt + pitch gyro for elevator, bank angle + GPS error for rudder). In the screen shot, the graph shows altitude versus time. About halfway through I bumped the altitude up about 500', and you can see the step response to the change plotted in blue. In any case, incredibly useful--not so much for tuning, but for making sure the maths are correct. The possibilities here are really endless...quantizing sensor data to N bits, adding noise, changing sample freqs. It's is amazing how much I've learned with with this. And it saves you from having to go the flying field, say hello to everyone, put the plane up, flip the auto pilot on, realize something is wrong, bring it down, and head back home in 5 minutes. Embarassing isn't really the word...but it's getting close.

I am having a bit of trouble with bluetooth latency going back to the microcontroller board. Not sure if it's in the bluetooth module, or if it's in the PDA. I suspect it's in the PDA, and that the PDA is wanting to buffer chars for a bit before sending them over hte wireless link.

That's all for now.
May 20, 2008, 01:42 AM
Registered User
Thread OP
Quick correct: the total range of correction from teh autopilot is shown in green on the graph. It ranges from about 0 to -130, so that's roughly the same as about 13 clicks of trim.
May 21, 2008, 03:44 AM
Registered User
Thread OP
Weather was bad tonight. I worked some more on the desktop autopilot.

The screen shot below shows the waypoint sequencer added, along with live map in the lower left (thanks to USGS), and the FlightGear simulator window in the lower right (I just moved the window over, it's a standalone app).

I found one bug that I know would have taken a long time to find flying at the field, so I'm happy about that.

The airplane is still the J3, and it's flying out of an airport someplace near San Jose. I've been able to dial up thunderstorms and turbulence and insance winds and watch the autopilot cope (or not). Soon I'll start looking for model airplane parameters rather than full size. The waypoints are about a mile apart, so this isn't a small, tight course by any stretch.

Note that in the gains for the heading, it's all zeros. I went into this thinking aileron position would be a function of heading error. Instead, I ended up with bank being related to heading error.

Waypoint targets are 100 m in diameter. Max bank limit is 20 degrees.No particular reason for either at this point.

I'm about at the max sample rate using the easy (web based) interface to FlightGear. As time permits I'll move to the high speed interface, where I shoudl be able to reduce most loops down quite a bit if needed. Note that FlightGear appears to be running the internal physics simulations at 1200 Hz, so there's merit in in trying to get this to run faster.
May 21, 2008, 05:55 PM
Registered User
Hi Matttay,

It's good to hear that you were able to get up to speed with FlightGear and make it do useful things.

It's refreshing to hear that someone with some network understanding can get up to speed and talking to FlightGear in a pretty small amount of time. If you'd like some hints/tips for optimizing your connection and use of FlightGear, feel free to drop me a note.

For what it's worth, the built in flight dynamics engine of FlightGear runs at 120 hz.

I've been using FlightGear to visualize flight paths and actual control surface deflections commanded by my autopilot. In addition it's possible (with a bit of hackery) to drop virtual bread crumbs in the sky so you can see your route and if you are flying a circuit, see the repeatability of your autopilot.

FlightGear has a multiplayer interface and a google tracking map tied to that, so that's another way you can leverage FlightGear to do fun things ... especially if you bring a net connection to the field with you ... it would be possible for anyone in the world with a web browser to live track your UAV (if it's feeding it's live telemetry into FlightGear and if that copy of FlightGear is connected to the multiplayer system.)

If you are looking for smaller aircraft, take a look at the Rascal 110 that is modeled in FlightGear. It's not perfect (and is really way too squirrelly on the ground) but once airborne it's not bad. My Rascal 110 is powered with an OS 160 FX so it's way overpowered and airborne before the rpm's can even come up to full throttle, so some of that is probably reflected in the FlightGear dynamics model.

Rascal 110 20070220 Takeoff (0 min 18 sec)

For what it's worth, I used the Rascal flight dynamics to tune the gains for an initial stab for my real autopilot (MicroGear) which uses the same autopilot xml configuration file / and autopilot algorithm as FlightGear. The result wasn't perfect, but it actually flew right out of the box. I thought that was kind of cool, and yet another way to abuse FlightGear.


May 21, 2008, 07:51 PM
Registered User
Thread OP
Whoa! I didn't know you were the author, nor did I know the author liked RC. I've been super pleased with the http interface into FG. What a great way to let someone start tweaking in minutes. A lot of times you download something with high hopes, it fails to install, then it crashes a few times, then you can't make the simplest thing work, and on and on. This has been one of those rare occasions where everything has just worked, and worked as expected. Congrats.

As I was watching the autopilot zip over the USGS map, I saw a lake on the map and out the window of FG at the same time. Made me smile. Has anyone done any sort of synthetic vision system for remote flying? I have UDP latency around 500 mS from air to ground, and because it's cellular, reliability is pretty good. How good is your terrain database? I've been playing with a web service from the USGS that lets you send a lon/lat, and it replies back with an terrain height in about half a second.

Thanks for the offer for help on some more advanced interfacing help. I think tonight I'll play with UDP. I see how to get FG to send all the stuff I was getting over HTTP via UDP, and I see how to get the ports and intervals set. But it's not clear how to send stuff back (such as elevator position). Do I define another xml protocol and set another option via command line and then just start pumping the values over the right port and FG figures everything out from there?

What control update rates do you would be possible over UDP? Seems like 20-30 Hz shoudl be pretty doable on a snappy machine with FG running VGA with fancy stuff turned off, yes?

Again, really nice work.

Quick Reply

Thread Tools