HobbyKing.com New Products Flash Sale
Reply
Thread Tools
Old Jun 13, 2014, 03:16 PM
Registered User
Joined Jan 2013
203 Posts
Discussion
Flip of Death "Beautifully heart breaking" -Wookong the story

I have been flying the Wookong for about a year and a half now. Doing random AP jobs as they come up and many personal creative projects. I have shot aerial for Ford Motor co. , Local real-estate , Music video producers etc.

Being a DP/videographer and being able to put the camera in the air is a dream come true. I have been building and flying RC since the earl 1980's with a few Hiatuses. When this multi rotor wave formed I was all in.

With a focus on filming I jumped on the DJI bandwagon buying the Phantom when it was first released. But I wanted to put a better camera in the air then a go pro so I started looking at the S800. After seeing so many issues with that airframe I decided to go with a Carbon Core 950. Great frame, low vibration. For the flight controller I went with the Wooknong M, with V2 gps. I also bought the Z15 G so that I could fly my hacked gh2.

I also built seven other multi rotors to use as flight training, back up aerial rigs and just general practice at being as proficient as possible with these systems. I still have a lot to learn.

So close to 200 flights in with the Wookong M, 950, Zenmuse combo the hex starts flying a little odd. At first it was altitude would drop randomly. This is without any configuration changes to the set up. I also noticed the Gimbal wasn't keeping horizon and was twitching randomly. White flashes started to apear on the LED for the Wookong. I started doing a lot of research on this and obviously vibration is the first topic that people bring up. I went through my whole system, motors and props and verified that all was balanced and vibrations were in check.

I came to the conclusion that the v1 IMU must be getting tired after so many flights. I also knew that having a flying weight of close to 15lbs was pushing the boundaries of what the V1 IMU was designed for so it made sense to buy an upgraded V2 IMU.

I installed the new IMU and was preparing for a new job which was supposed to be filming some classic cars for a spot. I went to the location where the event was to be held a couple of days before. I wanted to retune the Wookong to fly nice with the new V2 IMU. The new IMU is more responsive so the gains are said to be a little lower then with the V1. I lowered all my gains to 150 from 180 and set up remote running knobs on my Futaba.

The moment of truth came for the maiden with the new set up. The field was empty when we arrived. By the time we set everything up we had 4 people walk up and start asking the drone questions. We let them know that we were testing a newly configured system and that they needed to be in a safe location if they were going to watch.

Lift off was perfect. I took her up to about 20 feet and started testing the gains for pitch/roll and yaw. It was very smooth. I tested the vertical gain and altitude hold during forward flight which was also 100% better then it was with the older v1 IMU. Things were looking good so I took her up higher and just did a few laps testing the forward flight and gimbal operation.

4 minutes into the flight as it is passing directly over head, about 150 feet up I notice a boom come into the camera farm on the FPV monitor. I look up and see the Hex flipping. I switch into atti for a split second, no control, I flip to manual, still nothing. As my camera operator and I side step the point of impact I cut the throttle about 20 feet from the ground. That does work although none of the other controls did.

Impact. This carbon fiber machine splinters into a pile of rubble. I was in shock. Watching a close to $7000 dollar crash is indescribable. I can hear the spectators laughing and calling their friends describing the Drone Crash they just saw. Trying to take picture for thier instagrams.

It was a traumatic experience to say the least and I know many of you have gone through this. I have always been very cautious about flying over people and realize the dangerous nature of having a flying, blade wielding machine up in the air. I am very that'll for having this happen during a safety test flight and not over some expensive classic cars.

So what happened, what caused this flip of death? That is the question. It wasn't the props, wasn't the ESC's all motors were spinning as it descended. Post crash inspection and testing of the power system shows all motors and sec's working properly. After crash Wookong and New IMU are operational. Still get full GPS lock.

Obviously I am researching this like crazy now and what I have come up with is that the version of the GPS I have, which is V2 gps was recalled. It was recalled because it can create a voltage spike that will disable or effect the MCU. This has probably been causing the erratic flight behavior for the last several months that I was trying to fix by upgrading to the new IMU. Everything else checks out. I have no way of testing the IMU or seeing if there were errors. At the Time I was not flying with the OSDmk2 as I was trying to eliminate possible sources for errors.

Damage. Frame is destroyed 80%. Electronics. All survived including the gh2. Zenmuses gimbal. Mangled.

I called DJI USA 6 times every time their system hung up on me after wait 10-15 minutes each time. I then emailed them and have not heard back from them. I am going to keep this post updated with how this process evolves. Dealing with DJI about their system failure. I know that in many cases that these crashes and flyways are user error. This was not the case here. I have plenty of experience with user error to know when I blow it or when things are out of my control due to a failed component.

I guess I am using this forum to process this experience and it is long winded and maybe typing this out is therapeutic. If you don't like it don't read it. I know there are tons of threads I could post it too also .

The Zenmuse did a wonderful job of stabilizing this flip of death, even kept the Horizon looking good

Video.
Crash (4 min 10 sec)
justadude is offline Find More Posts by justadude
Last edited by justadude; Jun 13, 2014 at 08:19 PM.
Reply With Quote
Sign up now
to remove ads between posts
Old Jun 13, 2014, 03:39 PM
Ars longa, vita brevis
ursa's Avatar
Joined Jan 2014
477 Posts
Really sorry about the crash. Isn't wonderful how kind spectators are? But then, people used to go to public hangings, beheadings, the Colosseum to watch people get ripped apart by lions; so we do seem to be prone to schadenfreude.

In reading your tale, the only thing that gives me pause is this:

"I started doing a lot of research on this and obviously vibration is the first topic that people bring up. I went through my whole system, motors and props and verified that all was balanced and vibrations were in check."

One of the gripes I have with the DJI systems is that they don't have a way to tell you whether vibrations actually ARE under control. You can balance everything, place your fingertips on an arm or leg with the thing tied down and motors running, but ultimately you're just guessing. It would be nice if DJI introduced what is available on APM, Pixhawk and SuperX FC's--a vibration log. As in:

http://copter.ardupilot.com/wiki/ac_measuringvibration/

So, your evaluation may be right as to what the culprit was. But, how do you know?

Good Luck with DJI. That should prove to be interesting. On the upside, if there is one, we're just a couple days from Father's Day. Maybe the FD Fairy will visit with something bright and shiny and new. Or some fat stacks of green.
ursa is offline Find More Posts by ursa
Reply With Quote
Old Jun 13, 2014, 03:55 PM
Registered User
Joined Jan 2013
203 Posts
Quote:
Originally Posted by ursa View Post
Really sorry about the crash. Isn't wonderful how kind spectators are? But then, people used to go to public hangings, beheadings, the Colosseum to watch people get ripped apart by lions; so we do seem to be prone to schadenfreude.

In reading your tale, the only thing that gives me pause is this:

"I started doing a lot of research on this and obviously vibration is the first topic that people bring up. I went through my whole system, motors and props and verified that all was balanced and vibrations were in check."

One of the gripes I have with the DJI systems is that they don't have a way to tell you whether vibrations actually ARE under control. You can balance everything, place your fingertips on an arm or leg with the thing tied down and motors running, but ultimately you're just guessing. It would be nice if DJI introduced what is available on APM, Pixhawk and SuperX FC's--a vibration log. As in:

http://copter.ardupilot.com/wiki/ac_measuringvibration/

So, your evaluation may be right as to what the culprit was. But, how do you know?

Good Luck with DJI. That should prove to be interesting. On the upside, if there is one, we're just a couple days from Father's Day. Maybe the FD Fairy will visit with something bright and shiny and new. Or some fat stacks of green.
Thanks for the Cheer up. My wife in trying to consul my offered to replace it until I told her how much I had invested in it and then to add insult to injury I was in the dog house for a hot minute. Hahaha.

As far as vibration, yeah that would be nice to have a graph present on the GUI for dii to see the raw sensor data. I have been using a iPhone app and the old school laser pointer technique to check vibrations.

As far as how I will know what exactly was the culprit, at this point deductive logic is my starting point. Observation and testing. I can't cheaply test the components so hopefully sending them back to DJI will reveal the problem. I read that when they finally discovered the GPS issue it took them 72hours of feathered flight before it occurred, GPS spiking voltage and blasting the MCU into a flip of death.

I will update this post as soon as I hear back from DJI. If I ever do.
justadude is offline Find More Posts by justadude
Reply With Quote
Old Jun 13, 2014, 05:41 PM
Registered User
Joined Sep 2013
4,066 Posts
Quote:
Originally Posted by justadude View Post
4 minutes into the flight as it is passing directly over head, about 150 feet up I notice a boom come into the camera farm on the FPV monitor. I look up and see the Hex flipping.
"Huh, what does that even mean? Surely the guy is exaggerating." Then I watched the video.

I seriously thought a 4' black spider just crawled down the top of my monitor!

Sorry to hear about the crash, that's definitely an expensive one. Hope you find the ultimate cause and get everything resolved!

And I agree, that gimbal deserves a promotion...

It doesn't seem like motor/esc/prop issue, but have you ever flown your hex in a motor-out config to ensure you'd still have control?

How is your IMU mounted to the frame? I assume from the post-crash picture that it was still firmly seated even after hitting the ground? From the video, it does look like a pure MCU/IMU freakout. You just don't see this type of failure often...
ElGuano is online now Find More Posts by ElGuano
RCG Plus Member
Old Jun 13, 2014, 05:49 PM
Ars longa, vita brevis
ursa's Avatar
Joined Jan 2014
477 Posts
I went back and watched your video a couple more times. 2 observations:

1. These crashes that involve a Z15 have such good advertising potential, in that even in the face of unfolding catastrophe and impending doom the gimbal does a spectacular job of keeping the camera horizontal and steady, allowing the camera to faithfully record the electronic or mechanical mayhem.

2. As I watch the video, there are several points where there is a momentary horizontal "twitch" or "bobble" in the image, best seen if you're keeping your eye on a horizontal line reference point somewhere in the frame. It isn't "jello" per se, but it's something similar. And it isn't related to you pitching the camera up or down, or changing the throttle. I could be hallucinating, but based on what I'm seeing I think vibrations may have been more of a presence than you think they were. Unfortunately there's no way I can think of to prove or disprove it.

Out of the dog house yet?
ursa is offline Find More Posts by ursa
Reply With Quote
Old Jun 13, 2014, 05:59 PM
DJI SUPPORT / TESTING
blade strike's Avatar
United States, CA, Big Bear
Joined Jan 2006
10,864 Posts
Do you have a log file?
blade strike is online now Find More Posts by blade strike
Site Sponsor
Old Jun 13, 2014, 06:00 PM
Registered User
CG Photo's Avatar
United States, CA, Fresno
Joined Feb 2013
1,661 Posts
I have always said that most crashes are a result of user error, but this looks like a case of the flight controller just freaking out. Sorry for the loss. The Zen really did hang in there till the end.
CG Photo is offline Find More Posts by CG Photo
Reply With Quote
Old Jun 13, 2014, 06:02 PM
DJI SUPPORT / TESTING
blade strike's Avatar
United States, CA, Big Bear
Joined Jan 2006
10,864 Posts
Quote:
Originally Posted by CG Photo View Post
I have always said that most crashes are a result of user error, but this looks like a case of the flight controller just freaking out. Sorry for the loss. The Zen really did hang in there till the end.
I would agree but in my mind if the MC freaked out then how an hell did the Zen hold so stable? Remember it's receiving flight and compass data from the MC and Main IMU.
blade strike is online now Find More Posts by blade strike
Site Sponsor
Old Jun 13, 2014, 06:19 PM
Registered User
Made in Austria's Avatar
United States, NV, Las Vegas
Joined Dec 2013
837 Posts
Sorry to hear about your crash. This shows once again how unreliable our flying machines are. Cheap, expensive, hobby grade, or professional grade, it doesn't really matter much. They are all unreliable and can fall out of the sky at any time.
Made in Austria is online now Find More Posts by Made in Austria
Reply With Quote
Old Jun 13, 2014, 06:55 PM
Registered User
United States, CT
Joined Oct 2012
646 Posts
Quote:
Originally Posted by blade strike View Post
I would agree but in my mind if the MC freaked out then how an hell did the Zen hold so stable? Remember it's receiving flight and compass data from the MC and Main IMU.
1. He said he wasn't using iosd so he does not have a log file unfortunately.

2. I don't believe the Zenmuse receives data from the compass, just the MC and IMU.


I agree with justadude, this seems to be a case of the classic flip of death most likely caused by a GPS that was supposed to be recalled.
TimeTrave1er is online now Find More Posts by TimeTrave1er
Reply With Quote
Old Jun 13, 2014, 07:17 PM
DJI SUPPORT / TESTING
blade strike's Avatar
United States, CA, Big Bear
Joined Jan 2006
10,864 Posts
Quote:
Originally Posted by TimeTrave1er View Post
1. He said he wasn't using iosd so he does not have a log file unfortunately.

2. I don't believe the Zenmuse receives data from the compass, just the MC and IMU.


I agree with justadude, this seems to be a case of the classic flip of death most likely caused by a GPS that was supposed to be recalled.
I must have missed that he didn't have a markII.

It does get compass and main mc data, unplug it from the canbus and see what happens.
blade strike is online now Find More Posts by blade strike
Site Sponsor
Old Jun 13, 2014, 07:35 PM
Registered User
Joined Jan 2013
203 Posts
Quote:
Originally Posted by blade strike View Post
I must have missed that he didn't have a markII.

It does get compass and main mc data, unplug it from the canbus and see what happens.
true zen does get data from the IMU and and gps in order to use some of it's functions such as heading hold if you rotate the craft.

After the crash I tested all systems and they were all functioning, except the Gimbal. But motors spun up and changed speed as expected under pitch, roll and yaw. Gps found all satellites, modes switch between gps, atti, manual worked as expected.
justadude is offline Find More Posts by justadude
Reply With Quote
Old Jun 13, 2014, 07:36 PM
Registered User
CG Photo's Avatar
United States, CA, Fresno
Joined Feb 2013
1,661 Posts
Quote:
Originally Posted by blade strike View Post
I would agree but in my mind if the MC freaked out then how an hell did the Zen hold so stable? Remember it's receiving flight and compass data from the MC and Main IMU.

I have seen this in a good number of videos, most of the videos are of older Wookong controller software. The aircraft is tumbling to the ground end over end but the Zenmuse is still able to hold image well. I understand that the Zen gets position data from the controller.

When a computer OS crashes, parts of the OS sometimes continue to work while others don't. It is common for a computer program to just close if it crashes and the computer continues to run it's other processes.

I don't pretend to understand how the controllers are built or function but I can see how figuring out where you are (GPS) is a different task than figuring out what it needs to do to get where it should be. I could totally see one process crashing while another process is still correctly running in the controller. It may be possible that the controller knows where it is and what orientation it is in but the routine that sends the data to the motors could be crashed.

It is also possible that the controller is sending bad data to the Zen once the controller crashed. The Zen could continue to function using data from other sensors as best it can while it awaits correct data from the controller. This would help explain why the Zen is not holding the horizon perfectly once the aircraft starts looping out of control.

Looking at the video, all the props that I can see seem to still be spinning and the Zen has power so I doubt this was a loss of power.

We know that older versions of the Wookong software suffered from random flips of death.
CG Photo is offline Find More Posts by CG Photo
Reply With Quote
Old Jun 13, 2014, 07:40 PM
Registered User
Joined Jan 2013
203 Posts
Quote:
Originally Posted by blade strike View Post
I would agree but in my mind if the MC freaked out then how an hell did the Zen hold so stable? Remember it's receiving flight and compass data from the MC and Main IMU.
I believe that if you unplug the Zen from the can bus it will still auto level but will drift and not know it's position as relative to the craft. I tried this a while ago but my memory of it's behavior is a little fuzzy.
At any rate if it was the faulty gps as I suspect it's voltage spike screws up the MC, not the imi. Basically the spike momentarily disables or confuses the control inputs in the MC. The unit which sends the signals to the sec's.

I don't think the new IMU is the issue.
justadude is offline Find More Posts by justadude
Reply With Quote
Old Jun 13, 2014, 07:44 PM
Registered User
Joined Jan 2013
203 Posts
It is also possible that the controller is sending bad data to the Zen once the controller crashed. The Zen could continue to function using data from other sensors as best it can while it awaits correct data from the controller. This would help explain why the Zen is not holding the horizon perfectly once the aircraft starts looping out of control.

Looking at the video, all the props that I can see seem to still be spinning and the Zen has power so I doubt this was a loss of power.

We know that older versions of the Wookong software suffered from random flips of death.[/QUOTE]
Agreed, pretty much the logic I am using as well. As far as power, Both packs still had 4V per cell after the crash. I had to put the packs out of their misery with a long bath in salt water. I was afraid they were too damaged and catch fire.
justadude is offline Find More Posts by justadude
Reply With Quote
Reply


Thread Tools

Similar Threads
Category Thread Thread Starter Forum Replies Last Post
Discussion DJI F550 flip of death??? bbarb Multirotor Talk 6 Feb 08, 2014 03:34 PM
Discussion Dji Wookong M Prop Break and Crash. WHY? mustafa Multirotor Talk 9 Oct 28, 2013 05:09 PM
Discussion The tell tail heart (ESC beap of death) Nethole Electric Plane Talk 0 May 19, 2012 06:56 PM
Joke A beautiful and heart warming story... Berkie Humor 2 Nov 19, 2011 02:02 AM