Turn off his AOE - you have to expand his hot bar and right click on #8 (well in my game it's number 8).
It resets EVERYTIME you use your speeder, send him to sell items, make items, if you die, etc
It's a major pain to remember to turn it off all the time. He should remember the setting - well at least until he is dismissed or you die.