Think carefully about the type of rifles you want rather than the calibre / cartridge per se.
I think an armoury should look something along the lines of
A nice light 22 lr - practice, walkabout shooting rabbits etc.
A varmint / target type rifle. The sort of rifle that can reach out and touch small targets. Needs to be fast, flat shooting, small bullet and able to fire good long shot strings. Double duty as a rifle shooting for on ranges. Cost of ammunition is key. 223 with a faster twist so you can shoot longer ballistically better bullets is probably the way to go. Plenty of good arguments for 22-250, 243, 6.5, 25-06.
A light do it all rifle. This should be an accurate, low recoil easy to shoot easy to carry rifle. It should be in a deer legal cartridge - certainly for the smaller deer. It will probably have a lightish weight barrel shooting a 50 to 100 ish grain bullet, needs to be accurate though for first round hits out to 250, but doesn’t need to be shoot long shot strings. In Scotland a 222 or 223 would fill this for Roe, down south ideal for Muntjac and CWD. 243 has for years filled this role, and now the min bullet weight has been reduced to 80gn it still does for all deer. One of the 6.5mm can also do this admirably.
The stalking rifle. This should a rifle capable of shooting any deer, boar, antelope sized game. It’s the hunting rifle that can go anywhere. Probably has a medium / sporter weight barrel, but shootable from any position from a quick standing shot out a carefully aimed longer range shot in mountains. It’s cartridge will shoot a 130 to 180 ish, perhaps 200 grain bullet, at 2,700 to 3,000 fps. 6.5 at the lower end, 30-06 or the PRCs, belted magnums at the top end.
The above is a core quiver of rifles. You can then add others as needs require, but they will tend to be more specialist. Choice of optics dictates hugely what the end use actually is.
Cartridge and bullet type does need good consideration. Also very much depends on your shooting style, technique etc. If you are old school, like to keep things simple and use maximum point blank range, then velocity really helps.
If you like lots of gear and range finders, apps, etc then long high BC bullets zeroed at 100 and dialled for drops is the way to go.
Optics should match the rifle. I see little point in putting a massive scope on a lightweight rifle. But if you putting together a long range rig on an Accuracy International base then additional weight and bulk of a PM2 makes sense.
