I'm not that familar with varnish, but I do have a few questions...
What kind of Varnish are you using? For example... Tried & True Varnish has a much different application and drying technique from regular varnish. (This varnish I have experience with.)
How long are you letting it sit? Again... Little varnish experince here, but with stains or gel stains if it doesn't set up long enough you will get a pulling or removal effect that looks like wrinkles.
Generally wax on varnish is the best topcoat. I think anything else (Urethane, PolyUrethane, etc...) would soften it and pull the finish off.