I have this same problem. My mouse will randomly, very quickly change the look position every so often. I use right-click to look and move with WASD and am currently playing in Full Screen Windowed mode--not sure if this affected me as much in normal Windowed mode, though. My mouse is a Razer DeathAdder.