Did you know Wordpress is one major memory hog? No? Neither did I until I started getting "500 Internal Server" errors after having my hosting account at Joyent moved from FreeBSD to OpenSolaris, with it's resource management functionality being used to it's full extent.
You'll be surprised to hear this appears to be a topic that isn't really discussed much out in the open. The closest indication of anyone else encountering memory issues with Wordpress is this this post on the Wordpress support forum.
This is a great post and provides a good example that illustrates how much memory Wordpress really uses, and an good explanation as to why. In short, it seems Wordpress loads pretty much the entire options table into $GLOBALS when it initialises. This causes the memory usage to jump significantly.
I tested the suggestions in the forum post on a relatively clean installation and sure enough, the memory usage goes from a mere 50 Kb before Wordpress initialises to a massive 5.79 Mb after Wordpress has initialised, and then finally a maximum of 5.81 Mb once it's all done.
What a jump. OVER 5 Mb of memory attributed to Wordpress alone, and most of this is consumed by the vast amount of crap loaded into $GLOBALS.
Things are further compunded by plugin writers not knowing about the autoload functionality, which defaults to "yes", when adding their plugin options to the options table, and then not removing their options when the plugin is disabled.
The lack of knowledge of the autoload functionality and it's consequences can probably be put down to some bad documentation and Wordpress has always had an issue with options left over by plugins that are no longer installed, but does Wordpress really need all that information loaded into $GLOBALS on initalisation?
I don't know, but I plan to find out.