I wouldn't worry about it. Paul does a check at the beginning of many of the Paraliminals. That's a good opportunity to switch them to the correct ear. If you miss it because your mind is already wandering, let it go. It gives a different experience and the information is still processed by the brain.
Alex