Here's a true statement : if you can see the fish, they can see you. LM bass do not spook easily; I've had 24" bass swim past me while wading. Trout on the other hand, can be easily spooked.
I don't like to get too close while casting to fish. Sometimes I will cast 40-50 ft from them; I like the challenge of casting to them.
Fish can also see your profile. I have spooked fish from over 100 feet away; I was walking along a ridge overlooking the lake, and the bass near the shore made ripples as they swam toward deeper water.