No it won't drift because it senses the gravity vector, which should always point downward unless you are sitting in a centrifuge, or something fundamental in the universe goes wrong
I've tested mounting it on my hat, works amazingly well
! The annoying thing is you must tilt you head down by 20-30 degrees if you want it to work like a regular head tracker. Other wise it wont sense any change in panning.
It's too windy for flight tests today, I made a video from my stupid robot thing:
Right now it's a bit twitchy because I limited the head turning angle so when I turn my head I can still see the screen. Should be even better if I had goggles!
Accelerometers are much cheaper than gyros, no drift and require less processing, perhaps this can be made into a much cheaper HT solution!