The average group size thing is complicated. A long time back we had a bit of fun and put a one inch black dot on a bit of paper and some folks on this forum shot at it from 100 yards. I think about 1 in 10 of the shots fired hit the black dot, and there was a very small number of people willing to give it a try which says something in itself.
We've all shot occasional "good" groups and my experience is that there is a tendency to claim these as our "average" group. Equally I've seen people on here claim to have shot groups off the bonnet of their pickup that, when I looked them up, would have been among the top 10 bench rest record groups for the distance/calibre combination. I've also seen a lot of people who shoot groups where they discount the shots they don't like for various reasons - a flyer, wind, some made up reloading balderdash, you name it... So some half inch 3 shot groups started life as 5 inch 10 shot groups.
Statistically speaking people shooting a goodly number of rounds simply must occasionally shoot a very small group, and a very large one, and the size of their groups will follow a normal distribution. There are, of course, people who are good enough that they've moved their normal distribution such that it favours smaller groups than someone with very limited skill, such as myself, might shoot. It is also the case that a calm day, shooting from a comfortable position, must give you some advantage over a wild and gusty day shooting from a ditch. However, there was a very experienced person on this forum (now sadly dead) who was involved in scoring stalkers shoots and over his years in the game he'd seen a lot of rounds fired at a target, he maintained that the stalker shoots a 3 inch group on average and my observations lead me to conclude something similar. Yes I've seen people shoot very small groups, but I've also seen the same people shoot much larger groups as well.
What it is worth doing is putting a backing board behind all of your targets - attach your target in the same position every time - and after firing hundreds or thousands of rounds at a target you will have a backing board that shows you the area in which it is likely your next shot will fall. Unless you have a very special talent indeed then my money says that this area will be much larger than the "average" group size being claimed by people on the internet, this doesn't mean you are any worse than them, or that your barrel is shot out, or that you need the latest cartridge because it will show 1/4 inch less drop at 600 yards, or that you need to engage in some weird reloading practise that no one can explain but everyone says you need to do, but what it does mean is that you have an honest idea of how you normally shoot and that would put your shooting skills miles ahead of some people.