As Dama says fish taling can also be genetic so it was the right buck to take. Dama's points were spot on about ageing (with body condition, gait etc being the best tool) but as the question was about going back specifically the reduction in palm size point (with fish tailing in the herd I used to manage) is a guide, so if it looks mature (as opposed to pre prime) and has a narrow or split palm you could assume it is going back.
You can only very roughly age by looking at tooth wear, again it depends on the soil type and what they have been feeding on, the deer in the picture does not look very old - you have to section (cut) and polish a molar and count the layers like a tree stump to get a more accurate idea.
ATB
Nick