Calls for Wales to have one ferry port in Pembrokeshire instead of two due to declining trade following Brexit would be a “disaster” for the county, a Senedd candidate has said.