As with all things food this really comes down to personal preference. No right or wrong just what works for you (yourself)
As a butcher when we make sausages or burgers its usually about pleasing as many people as you can most of the time ( we want to sell them!) With venison we would recommend adding diced pork fat or if unavailable shoulder or belly. The fat delivers the moisture that pure venison mince will lack but as pork fat is a relatively neutral flavour doesn’t mask or cover the flavour of your hard aren’t spoils. When you cook sausages or burgers it’s a pretty vigourous cooking process & the meat really benefits from the additional fat. Goose fat or duck fat also works very nicely too & can be found canned in most Supermarkets.
Beef fat & lamb fat are both strong flavours. I am sure there are people out there with some great recipes that use them & would swear by it. Who’s to say they are wrong?
If you want to add moisture & richness to your venison though without overpowering its natural flavour I would avoid them.