UBB.Developers Forums UBB.threads 7 Development How Do I? Tons of Ram Needed for Hi-Traffic Sites w/UBBTH?

My site gets 1000 postings/day, and about 250K page views (mostly on boards).

A DevWiz has just warned me via PM that I may have some fairly serious problems with ram; large sites sometimes even have to split things among two servers.

We can't afford to pay beaucoup bucks for some hyper gnarly server or server array. Help?


JIM

My site is a bit different than most forums go. It's not a heavily posted forum site. I do get anywhere from 250,000 to 800,000 unique visitors a month though and everything runs through the ubb.threads software for the header and footer. This is on a shared hosting account. The server is a dual PIII 1ghz with 2gig ram using the Zeus webserver. Not sure if this helps any but thought I would mention it.

every bit of info helps!

Our visitor levels are similar, so this is heartening. How many page views/day do you get?

Kind of surprised you're on a shared server...I thought conventional wisdom was you can't do heavy mySQL stuff on a shared server....

For the last month of May I had over 300,000 unique visitors. The highest daily number was 19,327 and the lowest was 4,108.

I'm not sure about what you can and can not do on a shared server but at this time I'm doing it.

"I'm not sure about what you can and can not do on a shared server but at this time I'm doing it"


ha. well put!

what's your typical daily page view count?

I'm running between 20-60k pageviews per day on a shared server and its holding up fine so far

--woops, I thought I was replying to Dave, didn't realize a new respondent had entered the thread! Thanks, msula. I won't delete my reply, because there's other stuff, below---


ah, there's the issue. We have similar traffic levels, but ten times the actual clicking action.

I'm hoping to get a response to an email I sent Jason Lester; his ford diesel site is probably the busiest UBBthreads implementation.

I've heard that sites with half my site's action can need to split to two servers to handle the extreme memory requirements. I just can't afford to take those sorts of measures (we've got to do managed server).

InfoPop insists that any memory issues at high load levels are mySQL issues, rather than UBBTHreads issues. But I guess that answer's to be expected : )

I picked a day in May where I had just under 9,000 unique visitors. On this day my .cgi scripts had 3,300+ hits. My .php scripts had 22,000+ hits. There was probably close to 1,000 hits on plain old .html files. No actual page view numbers but each hit on these files ment it was loaded. I didn't count hits on graphic files.

I haven't counted them but I would guess my main page has about 20 queries involved with it and this is the page that gets hit up the most when people visit. This page alone had 6,000+ hits.

dave, per my reply to msula, we get 250K page views/day with similar user numbers (guess they're just more jittery -- too much coffee!).

I think the significant number regarding memory issues is page views rather than # users. My understanding is that as you move through the low six digits in daily page views, there's a steep, steep memory curve.

Month totals:

code:

Hits:        11459209
Files:        4371724
Pageviews:    1924873
Sessions:      184963
Kbytes sent: 50791136
[/code]

I have no clue if that is the type of information you need, it's all I could find.    We were on a regular old shared server at FutureQuest, and we were looking for a dedicated server (then we crashed).  FutureQuest ended up setting up a "Hi-Capacity Server" that is now only shared by us and one other website.

I am almost positive Jason is running FordDiesel on 2 servers. He talked about moving to that a while back. That said, he's still on 5.5, v6 should be easier on the server due to the greatly reduced queries...

Tweaking mysql setup can be the difference tho in whether or not you need to go to 2 servers. Do you mind sharing your setup stats?

thanks, Lisa

how much memory does your hi-capacity server use?

and how fast/slow does ubbthreads run on it?

also, how long has it been since your last server crash (in other words, since you made the server switch)?

and have you made any fundamental tweaks to UBBT that could affect its speed/memory overhead/etc?

I really appreciate everyone's help on this!


jim

Allen, thanks for piping in.

We're presently on a shared server. Part of our migration to UBBT (or another program, if I can't get assurance that UBBT will run decently within reasonable server resource parameters) will involve moving off a shared server onto dedicated (but managed...we're not geeks).

We'll want something running Apache/FreeBSD-ish. We're likely to go with Verio, FWIW. We're pretty open, other than that, except we can't afford to double up. And we need to have SOME idea of what memory/server requirements to expect before we get the server, before we migrate, before we even buy UBBT. But we will gladly choose a server to suit UBBT.

But InfoPop doesn't have any data on server resource requirements on high-traffic sites. And since I can't know till I've migrated and gone live, I'm extremely reluctant to be a guinea pig.

I am, however, willing to wait for v.6

We currently get about 250K page views/day, and 1000 postings/day, but we tend to double every six months, so we need to be sure this will easily scale to at least triple that traffic.

thanks,
JIM

I will have to check on the memory. This isn't a package that they offer, so it isn't listed (I'm sure we have it in an email somewhere though.) I think it runs quick enough (I've got a dial-up, and there are times when things are slow, but it's usually my connection.) We crashed the third week of March -- but we'd been going over our allotted bandwidth for several months. We were down for 2 days while the host got the new server set up for us, then we were down for a couple more days because we had to switch from the perl version to the php version of threads (host wouldn't bring us back up running the perl version. ) It was a heck of a lot of work but we haven't had a problem with the server since.

The only tweaks we've made have been cosmetic type ones (PM email and limits, little things like that.) Oh, I did change the defaults for "show threads active _______" to anywhere from 2 weeks to a month, and that seemed to speed things up a little (they had all been set to show all threads.) We did end up moving about 1/2 of the threads to "invisible" forums, basically like an archive. We'd like to figure out a way to archive them so they are still available, but that hasn't happened yet.

You can go check out the speed if you want. http://www.fishingfordeals.com/forum/ubbthreads.php

When I said stats, I meant your mysql .conf file and the connections/memory settings within Those can be adjusted (if you are on a dedicated server) to maximize the throughput and memory allocation.

woops, ok.

We're currently on a shared server, and our boards are in a non-database bbs program (wwwboard). We intend to migrate to a dedicated server at the same time as we migrate to UbbT. Hopefully, soon.

We've not yet done the importation or even arranged for the server, because I'm trying to determine:

1. whether UbbT is scalable to our traffic level using a reasonable amount of server resources

and

2. the memory/processor resources needed so I can get a server that will fit the bill.

I'm trying, quite understandably, to avoid going live with a new setup to discover that it won't work.

thanks yet again!


JIM

true, the software is a little new to be able to predict that as of yet...

I've been intimately involved with this software for about 3-4 years now, so if any very large site is inclined to "go first" and be a guinea pig, it'd be mine. But since I won't know if it will work (or what it will require) until I've actually gone live, I'm extremely reluctant. There's a vacuum of information re: what resources are needed at high levels.

Other very large sites will be at LEAST as reluctant as I. Which creates a vicious circle: we won't implement because of lack of data/experience, and that means we won't be generating data/experience.

InfoPop needs to test the scalability of this program, because bigger sites will be loathe to do public, live testing for them.


JIM

Here's another one running on 2 servers, tho they have 500-600 online users most of the day:

http://boards.stratics.com/php-bin/stratics/ubbthreads.php

hmmm... they have ~330 visible users right now.

One thing to keep in mind is that I would bet that the majority of the better php forums out there, such as ubb.threads, will be using pretty much the same amount of resources (yes it will vary some but I bet as far as the cream of the crop goes they will be close). This is because they have MySQL in common. If your site is growing at that rate to where it doubles every 6 months I have to wonder if you wouldn't be better off getting Infopop's Open Topic? This would be scalable and then some as your needs grow.

Just my 2 cents.

thanks, Dave

Problem is I really like UbbT. I found a fairly similar program, which can handle postgres (which would make more sense for me): phpBB (http://[censored]/phpBB). But it's not really as good as UbbT.

One thing I could do: severely limit searches, to save resources. Maybe charge users to search further back than one month. That would limit the number of searches and also bring in money to pay for the heavy server. The problem is that there's no easy solution for microcharging yet, so it'd have to be a monthly or annual subscription.

Also, I found that rackspace offers this server for just $743/month, which might be up to the task (though, again, I have noidea how much RAM I'll need...a gig would probably make more sense)

1000 Mhz Processor
256MB RAM
SCSI RAID Controller
dual 9.1GB SCSI Drives
20/40GB SCSI DAT Tape Drive
30GB (30GB incld.) Burstable Bandwidth

Groan. Idunno.

I haven't heard anyone mention that they use postgres but as far as I know ubbt can be set to use it in the config.inc.php file.

Have a look at http://www.tera-byte.com for servers. I'm thinking of moving there myself for the virtual server offering. $743 a month? Ack! Way beyond my budget. Have a look at communitech.net's offerings at http://www.communitech.net/order/dedicated/packages/intel/ to compare.

(that dual 1000hz machine with 1024meg ram sounds pretty good at $599)

I've made queries both here and to Infopop sales about postgres compatibility, but had no luck at all, only one vague warning against it from Rick a couple years ago.

I'll check your server links, thanks for them. Bear in mind that Rackspace has the advantage of being huge and staking it's rep on tech support and customer service. That's worth a premium.

When I was hosted with communitech.net their customer services was top notch. I left because they were slow to get PHP4 into the webservers as a module. They tested and tested and tested till I couldn't wait no longer. LoL but hey they were just being causious I guess.

If you don't go with Infopop web hosting, I'd recommend:

http://www.digitalprinceton.net/

An email I recently rec'd from their sales:

==================================================

Here is what we have in-stock and ready to roll.

You have to keep in mind this is going to be more expensive because you have SCSI HD, and it requires a Control Panel that is included in the price.

1000Mhz PIII
512 MB RAM
18 GB SCISI 15k RPM
60 GB Monthly Transfer
PLESK Control Panel


Setup: $250 Monthly: $389
Additional Drive $190 one time fee

If you would like to call me I can try to work out a better deal for you. 609.252.0713 x13"

==============================================

They are getting rave reviews in the support area... tell'm Allen Ayres sent you by Some friends of mine are putting a busy/growing ubb.classic on one just like the above today.

Infopop hosting has a dedicated server with a gig of RAM and 30 megs of bandwidth for $499

That was my first and last post advertising. I't probably in bad taste and if so I apologize.

I didn't think it was in bad taste.

Actually, where better to host the software than with the company that stands behind it? IMHO

I don't think it's in bad taste to let someone know of a good deal

As with JustDave, you couldn't go wrong with hosting with IP, they'll stand behind their work and make sure you aren't left in the lurch regarding server load/bandwidth issues

$499 for 30 megs of bandwidth? You got ripped

did he mention it was fast?

Writes JustDave:

---

One thing to keep in mind is that I would bet that the majority of the better php forums out there, such as ubb.threads, will be using pretty much the same amount of resources (yes it will vary some but I bet as far as the cream of the crop goes they will be close). This is because they have MySQL in common. If your site is growing at that rate to where it doubles every 6 months I have to wonder if you wouldn't be better off getting Infopop's Open Topic? This would be scalable and then some as your needs grow.

Just my 2 cents.

---

Yeah, but I want the features I want, I like UBBT, and I'm stubborn !

So I've been cobbling together solutions:

1. use gzip compression
2. get a server with dual processors
3. limit people's ability to search all forums all the way back (we'll charge 'em for that, actually...THAT'LL limit 'em!)
4. PHP Accelerator (http://www.php-accelerator.co.uk).
5. prayer
6. have backup S/W ready to install if our server crashes Mike Tyson-like under the load.


Best would be if UBBT could hook into postgres. Doesn't look like that's currently being worked on, though it's been bandied about over the years. Postgres seems to be getting very "hot" in many circles exactly because of its scaling robustness.

Also, Infopop promises to try and work up some benchmarks re: what scaling is possible with UBBT. If they don't, larger sites are going to remain terrified to install (which, in turn, will perpetuate the dearth of real world scalability data). This should be of concern to InfoPop (they're limiting their market) and to all of you. Good communities grow quickly. And UBBT communities tend to be good communities. Scaling issues are really important.


JIM