I've pretty much got the thing working. Still need to add some drift correction stuff, but otherwise it works great!
It's too cold to fly, so I've been testing it in FSX. Currently there is quite a bit of drift in the panning, I've not yet added any button to re-zero the angle, I've simply programmed it to reset the angle when you lower your head more than 60 dgrs. You can see me do that in the video. Because the roll and pitch axes are accelerometer based, there are no drift in those. Here's a video of me doing the helicopter oil rig inspection mission:
The first couple of times I tried flying with goggles, I was looking around a lot, but I've since stopped doing that because it really gives me motion sickness!