Taper and undo taper creates barrel distortion

What I want to do: taper geometry, do modifications, taper back

(In the end I want to create an offset with varying thickness, fading from top to bottom.)

Now, the problem is that I get a barrel distortion when tapering back, and infinite is toggled on. Without infinite, the distortion is even worse.

In the attached example, how do I undo the taper?

Alternatively: How do I create an offset that fades from top to bottom, but with more complex geometry than a simple rectangle?