Jubbin......I have news for you.....anyone who puts that much time and effort (as you did), researching rods and reels.....WILL get hooked big time on flyfishing. So do yourself a favor and get a quality rod and reel....in the long run, you will save money because you won't wind up buying twice.
It doesn't take that much to go from a starter combo to something that will last you a lifetime....and if you now have time....go for it.
Follow the advice that you have already received. I second the Temple Fork/Albright rods, I don't own one but I have casted one and for the buck, they are great rods. My opinion is that they perform up to 90% of the $600 rods. The five weight will give you much more flexibility, esp. if you take that 1 1/2 hr trip and go for larger fish. It also gives you some distance for the wider streams you are likely to go to. By the way.....panfish on a flyrod is nothing but fun, and you'll never get skunked!