I do not believe, given that you have the proper swivel pre load, that a Bilstein damper will cause a push.
I just did this exact same job; rebuilt both swivels, new pins and bushes and bearings, new seals, new tie rod ends, and a Bilstein damper. Adjusted the cross rod so that the steering wheel is centered and the tie rod for zero toe in. The steering is dead perfect, I can drive hands off and brake hands off with no pull.
There is only one way to determine your swivel pre load. Until you come back and write that you have x number of pounds pull on a particular side you're just whistling in the wind. Once you know what your numbers are, it can be determined whether this is part of your steering issues or not.
1966 109 5 door wagon 300Tdi "spermaceti fueled"
1994 RRC LeWiB "ruining the air behind me"
1968 2A 88
All my troubles are Rover