So I have fixed the mouse look issues I were having with the camera. It basically came down to that as I got further away from the origin the unit vector I was using to make my target vector never really changed because the changes were so small.
That really doesn't make much sense out of context.