I can attest that the X-factor is indeed very stable. I use it with a poling platform and I flyfish the flats from it. Not everyone can achieve what I do from it, but everyone can easily stand on the lower deck.
A kayak can be very uncomfortable, especially if it is undersized, has a wet ride and is not stable. It is better if you can try one before you buy. As a personal choice, I remove everything I do not need on a yak that could snag my fly line or prevent me to move around easily: this includes the back rest.
Here is a list of kayaks that are very versatile (all SOT). Not the fastest, but speed is not all you need.
Ocean kayak drifter, prowler
Malibu X-factor and X-treme
Wilderness system ride and tarpon 14
Cobra kayaks fish & Dive
I prize a lot the MArquesa. It is fast for a boat that wide and short, tracks really well, and is very stable. It is very dry too and the hull is very hard (= very responsive hull and tough).
I like also the WS ride (but it does have some hull slap: nver prevented me to catch bones in shallow water though). The X factor is big and is best to be used with someone heavy (I weigh 155 pounds).