One thing I have noticed about the CO5 wire is that it is very brittle. Once you solder on your connector you need to put heatshrink over that and overlap it over the heatshrink of the output wires to hold it all together. Too much wobbling and the wires break.
The fact that all your motors are getting so hot sounds like you are over propping them. Use smaller props or bigger motors.