I think you've already pointed out the cause. You see the tapering of the line down thru the leader acts like a brake taking the energy turning over thru the line and dampening it, or rather reducing energy to a whisper if you go fine enough.
Thus you need to first add an appropriate leader, then see if it still hooks or doglegs to one side. Also add a small puff of yarn to the end so there is a small bit of wind resistance on the very end to prevent cracking like a whip.
Give that a try and tell us what happens.
IFFF Certified THCI @ 2005
Capeflyfisher Guide Service
Island Hopper, Guitarist, Incurable Dreamer
and Founder, Worldwide Flyfishing Forum