The dangers of pooling data?

Andy Read takes a look at a well-written scientific report that has recently been quoted, and potentially misquoted, by many concerned fishermen and environmentalists, in many different places. Reading the small print, he suggests, is important before jumping to conclusions

“86% of assessed areas in the Greater North Sea and Celtic Seas experienced some physical disturbance from fishing between 2010 and 2015. 58% of this was highly disturbed!”

That was a tweet from a group called the Coastal Communities Network (CCN). The same figures were quoted last month in a presentation to MSPs who were considering a petition for a mobile-gear ban of some type in Scottish inshore waters. The phrase has been repeated elsewhere, and with less care than by CCN: ‘86% of the seabed damaged by fishermen’ is the simplified message many have repeated.

There are maps showing this data, too, that are highly professionally put together, conveying what would seem to be an alarming message. The fishing industry is apparently laying waste to vast areas of seabed. Politicians seeing the maps, and hearing the message, would rightly be concerned that they are allowing environmental destruction on a vast scale, and see the need to do something urgent to curtail this.

This is the map that has been doing the rounds on social media. The software used to generate the map has filled in huge areas of estimated impact and sensitivity, grading vast areas of seabed as either ‘damaged’ or ‘badly damaged’.

And we as an industry are obviously highly efficient! Somehow we managed to get mobile fishing gear on nearly nine-tenths of the entire UK seabed in just five years! That is some going.

And then I looked closely at the map. Hang on a minute. Ramsey Bay, in the Isle of Man, is shown on the map as mainly orange. That means it is ‘disturbed’ or ‘highly disturbed’ by mobile fishing gear.

Ramsey Bay, as FN readers with good memories will recall, has been featured in our pages several times in the past 18 months. It was designated as a Marine Nature Reserve over a decade ago. For three years of the 2010-2015 period used to produce these maps, it was subject to a complete closure to all forms of mobile gear!

Small areas within it are now open to mobile gears, but only for a handful of days a year. It is one of the most intensely monitored, and intensely studied, fishing grounds in the world! Data from the Manx Fish Producer’s Organisation suggests that just 3% of the ‘fishing zone’ sees mobile gear each year. That equates to about 1% of the bay in total.

If 1% of the bay is actually fished with mobile gear, how can the entire nature reserve be classed as ‘disturbed’ or ‘highly disturbed’? And if this classification is wrong, which it patently is, what other data or classifications on the map are also incorrect?

Time to go back to the original scientific paper where the maps, and claims, were first made.

The 86% figure being widely bandied about comes from a paper produced by the OSPAR Commission. They are scientific heavy-hitters. This is not an NGO with preconceived ideas, which might pool or gather data in a way that would support its message. So on the face of it this is bad news for the fishing industry. It is much harder to refute a well-respected, international body with a reputation for unbiased research.

You can see the original paper on which the claims being made are based here.

It is quite a hard read, if you are not used to some of the scientific language being used. I myself felt I had to check with real academics before writing my thoughts about the paper to make sure I had got it right.

But the bottom line is that the scientists estimated these levels of damage within the small areas they have divided the seabed into, to colour in the maps. Damage within an area is just that: damage within it. Not ‘the entire area has been damaged by fishing gear’. That is a huge difference.

It is like saying that a muddy footpath trampled across a nature reserve has ‘damaged’ the entire wildflower meadow that it has passed through.

Add in an impressive-sounding measurement, and the results, to the untrained eye, look to be even more conclusive!

The figures become even more questionable when you examine some of the statistical methodology used to fill in the map. It is similar to the 3D charts that many plotting systems now use to visualise the seabed. If you haven’t actually sailed over a patch of seabed, the clever algorithms inside the plotter simply join the dots between areas you have sailed over, and make you a pretty visualisation of what is in between the two areas.

It can look good, and very authoritative, but actually it is a guess. Many readers of FN will have seen such plots, and then have reacted in real time when a fastener appeared on the sounder when the plotter is showing a clean tow.

The OSPAR scientists likewise estimated damage in areas they had no data for, based on the nearest available data, and filled in the map accordingly. Then, on top of that, they chose to round up, rather than round down, their estimates about habitat sensitivity, and therefore the score about ‘damage’ that they coloured the maps with.

This is all there in the small print. The scientists are not trying to pull the wool over anyone’s eyes. They acknowledge in the report their incomplete data, and explain how they have filled in the map using estimates – that they have chosen, when in doubt, to class habitats as sensitive, when in fact they may not be.

They could, and possibly should, have done a cross-check. It would be easy enough to estimate the actual areas of seabed swept by each fleet segment they looked at, and compare that with the percentage coverage of seabed that their maps suggest as ‘damaged’, after also adjusting for areas that are swept once, twice or more times a year.

That was possibly an honest mistake.

What is less honest is how this 86% figure is being used by some NGOs – not all – as a tool with which to bash fishing. Perhaps they have been too lazy to actually read the small print. I suspect not. Certain NGOs with huge funding and plenty of well-paid scientists on their books – Greenpeace is a prime example – long ago walked away from any genuine science. It is much easier for such NGOs to misquote other articles, reprint alarming-looking maps, and attract more donations through headlines about ‘disasters’ in the marine environment, than to produce genuine science of their own.

That moves us on to the really well-intentioned, well-meaning people who are worried about the marine environment and take on trust the maps and headlines produced by these NGOs. Concerned citizens like the members of the local community who demonstrated about the shellfish die-off in Whitby last weekend, or the surfers posting films online of illegal sewage discharge spewing into bays and beaches across the UK.

Presented with maps that suggest that nine-tenths (let’s follow the report’s example and round up the 86% figure) of the seabed is being damaged by mobile fishing gear, they will be concerned. If the figure was actually correct, so should all fishermen be! We have no sustainable future with that kind of pressure on the marine environment.

We’ve banged on in FN in recent weeks about the value of good science and good fisheries data, to protect our industry against misinformation. We’ve highlighted the work being driven by fishermen across the British Isles to support accurate stock assessments.

In a world increasingly being driven by soundbites, where very few people have the time or inclination to check the small print, it is easy to criticise comments such as those made by CCN and see these genuinely concerned volunteers as opponents. That is likely to be counterproductive. With correctly interpreted science, many of these campaigners would side with sustainable fishing and the need for protection of the marine environment that will prevent more shellfish die-offs, and ensure a sustainable industry for generations to come.

This story was taken from the latest issue of Fishing News. For more up-to-date and in-depth reports on the UK and Irish commercial fishing sector, subscribe to Fishing News here or buy the latest single issue for just £3.30 here.

Sign up to Fishing News’ FREE e-newsletter here.