Whatever browser you use, it is still necessary to have a good firewall and antivirus program, plus programs to get rid of spyware. I have ZoneAlarm Pro, Norton Antivirus, AdAware 6, and Spybot and so far have avoided major problems except for a few tracking cookies that get by when I use IE.
Agreed, a lot of problems come from the user just "clicking away". I yell at my mother all the time for that. I have a hardware firewall to my network and my mothers. This seriously helps me maintain their systems. Even so they seem to have been pretty safe so far.
I tried zonealarm (the free version) and it was more trouble than it was worth. Even after setting up certain rules and IP filters I still had issues with it. I control/monitor my other PC's with UltraVNC (so I can run those desktops from my main computer often times eliminating a need for any periferals for them) and zonealarm was such a pain about it. MS's new firewall is just as annoying with VNC too.
I wouldn't touch or recommend norton products to any one. I have had nothing but trouble with them. In fact 4 out of 4 times my system got all hosed just by installing their junk and only after uninstalling it and some fixing did my system come back. I refuse to use their products. To think people pay money for them makes me sad.

I use AVG it works better isn't a resource hog, and best of all it's free to home users. I use adaware and SB S&D and have been for a while now.
It is still necessary to have IE, both to update Windows and to log in to certain websites, like my bank, which are so poorly written they only work with IE.
To have windows updated yes and no. If you have SP2 on XP you can just set it up for auotupdate and then you don't need any browser at all. The other updates most people want will bug you enough i.e. WMP and MS messenger. Of course you can't really get rid of IE anyhow since the browser is just an extention of the window manager also called explorer. (you should notice the process named explorer.exe in the task manager if you use win 2K or XP). I only state all this for anyone who doesn't know. You may know all of this.
You should write and complain to your bank. As you have said it is poorly written and hence can not be used with other browsers. Luckily my banks webpage works with firefox/thunderbird/firebird what ever iteration you are using.
As far as stopping cookies even in IE you can disable them to "ask for every one". I block any cookie I don't want or know where it comes from. OF course you need to be sort of savvy to use this option as I have had to fix several accidental cookie blocks for my mother. I haven't had a tracking cookie I didn't want since I started blocking cookies 2 years ago or so. It's a pain at first but once you get your list setup you almost don't notice it, unless of course you visit new sites all the time. It's veryy telling though of just how much junk a website trys to dump on your PC.
Though I like the RSS feeds in Firefox, it is not a very useful browser as it has a bug that makes it impossible to log in to most sites or to fill in forms - for sites I have to log in to, I have to revert to Firebird or IE.
I think you just don't have certain settings coorectly setup. I have been using Firefox since the .6 version of then named thunderbird over two years ago. I only use IE when I have no choice which is almost never. Firefox has and still keeps me logged into every website I personally use. This includes obviously CU. As far as forms I believe that's an option you need to set. I did it so long ago though it just keeps following me with every new version. A great thing about firefox is it keeps your settings and bookmarks separate so even if you uninstall and reinstall your bookmarks are still there.
If you go to the menu and use tools --> options and then click on the privacy tab you will see a section for "saved form information" and another for "saved passwords". You can then tick the box and it saves any form and search info you have typed in. It can also be cleared by using the clear button.
It works all the time for me. In fact, when I fill out certain forms, I just hit the down arrow and it takes but a few seconds to fill out forms. You don't even have to type any letters. It's a beautiful thing.

You guys should give me some examples of what doesn't work. Maybe I can help.