Well, like you said, load times will be faster for users. But.. the drawback is the CPU overhead with turning on compression. So you should monitor the actual server loads, as that is what compression will affect most. So it is a a give or take, and will depend on your server hardware. If your server is seeing high loads with gzip on.. you may want to turn it off, to sacrifice slightly longer user load times to bring the load of your server down.
THere are no set rules to this, as it all depends on so many individual factors and prefrences
