I just about never bother with varnish on anything, except if it's to change the glossiness of the model — like to make it gloss if it's matt but shouldn't be. Other than that, I generally consider varnish to be just one more thing that can go wrong.