This is what the plug looked like before I took the pump out
When I took the fuel pump out, the pin you can see loose fell out
So it appears that at some time the pin has got hot ( I assume due to a poor electrical contact) and damaged the plastic holding the pin, leading to it becoming loose. So I've put shrink Insulation on the damaged part of the cable, polished the pin, and potted the cable pins on the inside of the pump lid with Araldite.
So once it is back together we'll see if that has sorted it.