Kayaks are actually very stable, and the only time when I ever kind of went over was when I was trying to get out. It is a great upper body workout and since you stabilize yourself with your legs and core, it is a very good exercise.
Our area is a great place to kayak and canoe. There are many public boat launches where we can put in and explore the shore line and small creeks. We have seen gorgeous swamps, plants, and birds. At various times we floated next to rays and turtles, absolutely delightful. The photos below is one I took during our trip and one that my wife took of me that same day.