Sounds to me as if you have a two pole three phase, one tooth per phase motor. This combo can be found in the chart at post #4 of this thread. It is the least number of poles and teeth possible for a three phase motor.
I believe this combination runs a little rough because the two "on" phases are not symetrical to any other phase as would be the case of a winding around say six teeth.
As a result, I would try a sensored controller with it. It should eliminate the start up stutter typical of a sensorless control.