logy,
If you do a search of the site for "fly tying kits", or "beginning fly tying", you will find quite a bit of info on this topic.
That said, here is a very brief summary of what is best for anyone just starting to learn fly tying:
1) Buy a good vise. This cannot be emphasized enough or the hook slippage will drive you nuts. You don't have to spend a lot for a good vise because there are good ones on the market for around $50.00. The Thompson Model A and the Griffin Model 1A and 2A are both excellent, inexpensive vises with quality jaws that hold hooks very well. Avoid the cheapo imports, they are false economy because they have poor jaws, don't hold hooks well, and don't hold up. You don't need a rotary vise to get started tying, or a more expensive one (meaning $90.00 or more), all you need is a good solid, serviceable, hold hooks without slippage, good quality vise.
2) Buy good scissors. Again, you don't have to spend a lot to get good ones. Griffin and Gudebrod both have good scissors for around $10.00. But even the best scissors like those from Anvil and Dr. Slick are only about $20.00.
3) Get a good bobbin (or 2). You don't need a ceramic tipped one, a good wire bobbin will work just as well as the ceramic ones. One of my favorite bobbins, which is also one of the cheapest, is the S&M Bobbin. They sell for about $7.00 and I like them so much, I own 15 of them the oldest of which I got back in 1973.
4) Get hackle pliers. Although I have several different types, all you need is one good pair of the tear-drop style. Again, these don't cost much and even the cheaper imported ones will work fine. Expect to pay between $3.00 and $6.00 for a decent hackle plier (unless you go for broke and get one of the rotating hackle pliers, which I don't recommend to a beginner).
5) Get a bodkin, which is nothing more than a large needle in a handle. Again, the cheap imported ones work just as well as the more expensive ones. You can find them for as little as $2.00.
6) Get yourself a whip finisher, it will make tying off the thread much easier and produce a very durable fly. Get one of the Materelli or Materelli-style ones. Although I prefer the real McCoy Materelli's (which sell for around $16.00), the cheap import copies of the Materelli work fine (they are about $6.00).
You can get all of the above (good vise, scissors, hackle pliers, bobbin, bodkin, and whip finisher) in a tying tools kit made by Griffin for around $65.00 (which is a very good buy).
The best way to get materials is to buy only the materials needed to tie a single fly pattern (or maybe two different patterns). The best flies to begin tying with are simply tail, body, hackle ones like the GREY HACKLE, BROWN HACKLE, WOOLLY BUGGER, WOOLLY WORM, etc. And when you tie a fly, don't stop with one, tie at least 6 of the same size and pattern before you move to another fly, then do the same tying 6 of them, etc.
And it is always best to start tying through taking a fly tying class at a local fly shop. Taking a class will save you huge amounts of time and frustration and prevent you from developing bad habits. Fly tying classes are money well spent.