Survivorship bias and false conclusions.
I saw an interesting You Tube programme last night on American B-17 "Flying Fortress" bombers. These suffered greatly in daylight bombing with many lost to enemy fire and others only just managing to return to their home airfields but badly shot up and some then having to be scrapped as being beyond repair to be able to ever safely fly again.
So the US Army Air Force made a study of these returned badly shot up 'planes. Marking the damage on the outline of a drawing of the 'plane and decided that the solution would be to add armour to the areas that this diagram showed as getting the most hits from enemy fire.
Until...
Until a more intelligent mind said that they were using the wrong logic. That their survey result gave a false conclusion. That in fact the areas that needed to be reinforced with added armour were those areas on the badly damaged and returned aircraft were those areas THAT SHOWED NO HITS.
As the actuality was that as none that came back had any hits in the same undamaged areas this meant that by simple deduction that any hit in what on a surviving plane was an undamaged area was actually fatal to a 'plane that was hit in that area. Which is why those that come back showed no hits in those areas.
So by extension to what we do the ONLY research some are relying on is that they have found lead shot in the gizzards of wild dead ducks. There may be considerably more wild ducks that also have lead shot in their gizzards that aren't picked to the be dissected because they haven't died from it. So therefore the survey by only using wild dead birds is based on flawed evidence.
Again the reply many have given as to "where then are all these dead wild duck"?
And that surveys based on captive bids dosed with lead are also to be discounted as the dosing of them may far exceed what would be, if any, the amount of lead in the gizzard of a wild bird. So basing a survey on what you have to hand (be that survived 'planes) or dead wild birds is incorrect as what you actually should be surveying is the very things you can NEVER survey (shot down 'planes or live wild duck).
See 2:50 onwards in the video. It is called "survivorship bias". In the Wikipedia article what is relatable to wild ducks is the paragraph about cats falling from buildings.
In this case the vice-versa in that wild duck that have ingested some lead shot and survive are never subject to testing. As still being alive they are therefore never sampled. In simple terms some would therefore condemn such conclusions based on dead wild birds as useless as being no more that the equivalent of "cherry picking" dressed up as supposed science.
en.wikipedia.org