Mike I saw your site (IIP) and it shows as you said:
Generated in 3.527 seconds in which 3.482 seconds were spent on a total of 36 queries. Zlib compression enabled.
I would first start reducing the number of IIP information there is on the index screen. Also another way is for you to increase the cache time for IIP (it can be done through the control panel of IIP). Also when users log in get them to go to the main index and not the entrance (if you have installed that modification)
Finally a suggestion is to reduce the information on the sidebar. It is my view that people don't really need to see the short stats all the time - but then again it is just me talking here.
Try shuffling around the information displayed and monitor it with your host.
In the past I had seen one of my forums nearly bringing down the host's server and Jeremy was very helpful in trying to find out the cause with me. It appeared that by installing a modification for the who's online on all pages (below the nav bar) I was killing the database.
I hope this information helps a bit.