Their night-vision is quite sharp. From what you describe I imagine they were stirks/bullocks or maybe young heifers. These type of cattle are generally more 'flighty' than dairy, even during daytime. When I am shooting at night where cattle are present it is best to be sensible and skirt around them and keep a fence line close by if possible. I have been in one or two unnerving situations over the years when a herd of stirks became spooked while I had crossed their field. Twenty plus bullocks thundering helter-skelter around you.....in the dark, is not a 'good place to be' ! That said, best to stay calm and still and not run, 'make yourself big' and roar/shout if it appears they are about to run into you. To run would only exacerbate the situation and encourage them, you would then be in greater danger of being 'accidentally' run down !
You did right as you describe. The farmer would not be too impressed if you had pressed-on and said cattle then went on to take out the fence in their panic. Worse still if they ran you down ! A few rabbits aren't worth that !