Our bodies and in particular our guts are pretty finely tuned to respond to unusual proteins. It’s how our immune system works - unusual protein - may be a virus or a bug etc and lets get rid of it.
In the western world we have a very simple diet with not a huge variation of different proteins going in. When we were hunter gatherers we ate pretty much anything and hence we had greater tolerance to things. We also died that much younger.
Venison for many is not a common protein so hardly surprising that it can upset a stomach. Its also why when go over seas and eat differently or indeed try many new foods are stomachs and guts take a bit of time to settle down. And some, who haven’t grown up eating venison may well struggle with it as an adult. Many Chinese and Japanese, who grow up in a culture with limited exposure to dairy products really struggle when exposed to cheeses etc.
With some people and some particular proteins you simply cannot tolerate them for whatever reason and trying to build up or reestablish tolerance is very difficult and pretty miserable. Things like antibiotics and cheap processed foods don’t help either.
And the different species of deer in the UK are very different genetically so hardly surprising that people react differently to different types of venison. The exception of course are Sika and Red which are really races of the same species given their ability to interbreed and produce fertile offspring. Roe, Fallow, Muntjac and CWD are all far apart.