I've been using Firefox almost exclusively over the last year+ (I only use IE for Windows and Office Update). It's small (approx 4-5MB download). It's fast (especially with the adblock plugin, over dial-up). And it's a whole lot more secure than IE.
The plugins allow you to add tons of functionality that you want, without requiring you to have stuff you don't need. Personally, I use the Adblock (blocks ads based on keywords in the url), Web developer (allows me to view and edit the source for pages I work on in realtime), Diggler (for more advanced navigation abilities), Tabbbrowser Prefs (extends the already incredible tabbed browsing abilities), Bugmenot (avoids compulsary web registration, like for new sites), Down Them All (advanced downloading options), and Sage (RSS feed aggregator). And if I find I don't need one of those plugins, I just remove it and it's gone. No unneeded functionality, no bloat.
It may take a bit to set up just to your liking, and it is still a pre-release version, so there could be issues. I've had very few issues, but I know of some people that have had more. However, I highly recommend this browser to anybody.