Assuming a bullet is competently designed and also assuming a viable spin rate is achieved through the barrel's rifling twist pitch to stabilise the bullet fully, there are three factors that affect bullet performance so far as groups sizes (dispersion) are concerned:
1) Nose / ogive type. Traditionally, Sierra, Speer, Lapua, Nosler et al used relatively short, blunt and tangent ogive noses on their bullets, whilst Hornady used a 'mild' secant form on most of its bullets. Tangent ogive designs are easier to 'tune' (largely in terms of COAL / 'jump' to the rifling) than secant types albeit Hornady's secant bullets were so mild that this wasn't a great issue. A jump-tolerant design usually simply shoots well at various settings / speeds assuming the bullet to barrel match is OK. Early Berger long-range designs were all 'aggressive secant' long-nose VLD designs. These are much harder to tune but reduce in-flight drag. While they were all for match shooters that didn't affect stalkers, but the rise in very long-range live quarry shooting in some places and circumstances has seen a marked shift from high-drag easy to use designs to low drag 'finicky' models with Berger redesignating some its range as 'Hunting VLDs' and everybody else following with super long-nose high-BC designs options.
2) Bullet to barrel compatibility. No matter how well designed or made, if the bullet and barrel dimensions don't suit, the results won't be great. An extreme example is GB and Commonwealth 'Target Rifle' from 1968 to the Millennium where the use of military-spec 7.62mm NATO ball was the norm. Not only is the 7.62 not quite the same as 308 Win needing a modified chamber, but the bullets were markedly undersize compared to 308 Win - as small as 0.3073" (compared to the nominal 0.3080" in SAAMI 30-calibres, and often actual 0.3082-0.3083" for commercial American or European bullets). As a result, TR rifles used 'tight' barrels, some very 'tight' indeed such as those Number 4s rebarrelled with late 60s and 70s RSAF Enfield Lock hammer forged heavy barrels designed for the British forces L39 and L42 target / sniper variants of the Number 4 rifle. Bores aropund 0.294 or 0.295" (v the nominal 0.300") and smaller groove diameters were the only way to make these rifles shoot acceptably with these bullets. It's also one reason why many commercial 308 Win sporting or match rifles gave so-so or worse precision with cheap surplus milspec 7.62 ammo.
There is variability between makes in some calibres, especially 6mm for some reason. Two sets of 6mm barrel dimensions are available from the US custom barrelmakers and people may choose one or other on the basis of the make / model of bullet to be used. More an issue for the extreme precision match shooter than the sporting rifleman though.
3) Manufacturing quality and consistency. It is very easy to make poor bullets and difficult to make good ones. Components - the gilding metal jackets and the lead alloy cores have to be very well dimensioned and consistent to get small groups. The bits have to be put together by people who really know what they're about and do it the same way bullet after bullet. That's why even today, top short-distance BR shooters use handmade bullets from small output producers. The only such UK producer is G&C Custom Bullets in Hampshire
http://ukvarminting.com/forums/topic/21107-g-and-c-bullets/
Even so, custom bullet makers buy in the unformed jackets and if you cannot get high enough quality, you can't make the best. Berger Bullets bought the best mass maker of BR quality jackets in the world, the makers of J4 products and it is a key part of Berger bullets quality. If the lead core and the jacket are not made to very high consistency, the bullet will have a marginally thicker jacket on one side than another and that makes it heavier on that side resulting in the bullet's physical axial centre and centre of gravity not being the same. Under spin, the bullet will always veer towards an imbalanced C of G. That is why to go back to GB TR and rubbish military bullets, the slowest possible rifling twist rate (1 turn in 14") was used to reduce bullet spin speeds and hence imbalance caused dispersions. J4s and suchlike are consistent to less that 0.0003", often smaller than that.
This was also how the 'myth' of flat-base designs being 'more accurate' than boat-tails arose. It is much harder to make a super accurate / consistent product with a BT. For years the latter were inferior in their consistency / balance, but people have learned now to make very, very good BT designs these days and that applies to the mass producers too, so Sierra, Hornady, Nosler and Berger etc's products are streets ahead of those of 10 or 15 years ago. Short-range BR doesn't need BT designs and they're too difficult for the smallscale garage workshop producers to make so they remain FB types, but not because FB is inherently superior to BT anymore.
Obviously, other interacting factors / components have to be 'right' too. The best bullet in the world won't shoot well from a poorly made or worn out barrel.