I think these power dips are exactly the problem, the switching of the connected devices causes spikes which influence the communication.

Could you connect the AC/DC-adaptor to your notebook? In most cases the grounding of the notebook eliminates this problem.