If the controller port is connected to the same +5V rail as the CPU, wouldn’t the NES crash if it only got 4.6V or less?
If only the BlueRetro is affected, maybe something behaves like a resistor in series with it, for example a broken solder joint in the adapter or at the connector on the NES
You could also downvote on the desktop site by using the RES keyboard shortcut