Just my 2¢ -- as somebody who flies purely LOS and isn't too concerned with endurance -- I would never risk flying my GoPro without the case. I've had at least 1 crash that I know would've destroyed the camera had it not been for the case. I figured if I'm flying something as aerodynamically un-friendly as the GoPro, there's really no harm in adding a little extra weight and drag with the added benefit of arguably the best protection.
Now of course I understand the importance of saving weight and trying to be as efficient as possible for endurance and distance flying, and if that was my goal than I'd certainly rethink using the case. But then if I could afford all the FPV gear, top of the line batteries and electronics, etc., then I could probably afford to destroy a GoPro or two as well