I don't recall ever culling a deer using an IR, despite having several scopes that have one.
If the light gets so low that you think the IR will help, and you switch it on, even on a very low setting, the remainder of the sight picture gets even worse.
Therefore my advice would be that if you can still see the crosshairs and silhouette of the animal you have identified and are trying to cull so you can place your shot correctly - go ahead.
A silhouette (outline) is fine - you don't need to see the outline of the front legs and shoulder. With some experience you can still place the crosshairs correctly in the killing zone. In fact with time and experience it becomes instinctive.
If your crosshairs merge with the background and you have to say a little prayer' Oh my God lets hope this is about right...': Call it a day, and don't take the shot.