Scottish rural economy minister Fergus Ewing said the sector was being decimated by the pandemic, Brexit and tariffs.