Don't worry about it. You do it naturally when you are day dreaming and just never noticed before. Now you're using this skill in a special way and you're noticing it.
Just do something that brings it back to the appropriate focus for what you are doing.
Alex