I know what you mean, my site is vehicle oriented... but now, the off-topic area is way more busy than all of the other forums combined. It was bothersome at first, but the more I thought about it I realized that the people came to the site for one reason.. for the information. Then while there, they make friends who share similar interests, and then spend time chatting.
I think the biggest thing to be concerned about is to make sure that ALL off-topic chat is contained in the forum setup for it. Be very strict with that so that all of the technical forums and such are just that. I found that without a place for people to chit-chat and just talk leisurely, then the off-topic dribble gets brought into other forums, and causes more problems.
I supposed you could add a small hack or something that limited the number of posts in a certain forum per day or something, but I think like you said, you will start to upset people and may cause more problems.
Sorry I can't be of much more help, just some of my own observations from running a specific vehicle forum for 3 years
