Mofos
To be able to control where the horse is going, you need a saddle.
You'll have to crouch to place it on the horse, after you've tamed it.