I've always been a canoe guy, but I've have taken some sea kayaks for a spin a few times. I used to be a member of the Pointe Claire Canoe Club, and went for an hour or two paddle on Lac St. Louis most every day in the summer after work. It was a lot of fun, but the waves were never big enough.
We used to have a place in PEI and our neighbor took me out on the ocean a few times, which was a lot more fun!
Sometimes the waves were a little too big
As for fishing, that's why I prefer a canoe. You can pack a lot more stuff, stand up to cast, and there's just generally a lot more room and stability to get on with catching some supper.
I've never been in whitewater in a kayak, just in a canoe, but obviously a kayak is a lot more versatile in rapids than a canoe.
What kind of kayak have you got?