Could take ‘decades’ to shift away from reliance on overseas staff, say food sector leaders attending Downing Street summit.