I hear you on using adhesive to secure the adaptor. Wish it wasn't necessary but my own experience is pointing in the same direction as what you are already doing. Gotta bite that bullet it seems.
Originally Posted by PAPE
This makes total sense, I have seen it happen on other fans, and this is why I mount all my adapters the following way. I don't like flat spots on motor shafts and if there is one I usually fill is with jb weld. When I mount an adapter I drop 3 or 4 drops of red loctite in the adapter hole. Then I press it against shaft until it bottoms out and quickly start going at the grub screws to get the best run out, then let it dry 24hrs.
This way my adapters never shift, but if it has to be removed it will have to be heated.