UBB.Developers Forums UBB.threads 7 Development How Do I? Why is my forum not being indexed?

I upgraded to 6.7 Beta 1 the day it was released practically.

I enabled the short spider friendly URL's..

still though.. it seems it's not indexed at all..

forums that I've posted on elsewhere are already indexed even with posts that I made after I upgraded...

yet my forum.. no.. ??

what's wrong with it? What else must I add/do?

http://www.musicianchat.com <- my forum

did you submit to google.com ?

There are results on Google, from as new as a week or two ago.

You're not in FAST/alltheweb though, which really surprises me. FAST easily handles UBBs, even without spider friendly URLs.

That sorta tells me that the spiders just aren't interested for now... suggest submitting to both sites, and getting more people to link in, then pointing spiders that the linking sites...

complete indexing will never happen, as they update url's for changes and follow paths next check. It can take a month to see a considerable change.

There are hundreds of places to index your site, just search goodle ... Heck, 70% of my traffic to my site comes through google...

well.. there's a few entries in google.. however they've been there always...

I mean topics.. posts..etc.. those are't indexed.

I've found posting as many times here as possible it a good way to get indexed

Anyhoo, just submitted your url to google for ya, wait a week and see what happens

Mark, it's not something that'll happen in a day... Heck, with my board it took 2 months to really see anything that could be satisfactory... Check:

UGN Security search on Google: here
Musicianchat search on Google: here

So long as you have the setting triggered on it should begin working. Note that UGN is using a differant spiderable URL program than the default one on the UBB now, but it's a good example...

true... I don't quite remember when exactly 6.7 beta 1 came out...

I thought it would have been long enough by now.. but maybe not. Usually I've had all of my sites indexed pretty quickly in the past... but I'm thinking maybe this is different because it's already been indexed before without the spider friendly URLs... I dunno how long they have it set between indexes... so the bot very well may be scheduled for another looksy soon.

Thanks for submitting the site Weird Al. Maybe that'll do it.

One thing I know that helps (at least with placement) is by putting the site URL in your sig.. that way whenever you post anywhere, there's a link from another site. However I wonder if the engines have pattern searches for stuff like that.

I probably should add meta-data as that might help.

Sorry to bug you guys about this. I was just wondering if I had done anything wrong. The only reason this is even starting to become an issue is because my board has somewhat died lately. I need some fresh interest from the outside to help spark things. My board was pretty active at the start.. but the lack of users this time (my last board had 100 users a week) seems to be a turn-off to the current members (and to be honest.. I can't blame them).

I just hope that when it gets indexed.. that'll help bring people in. In the meantime.. I'll work on getting it looking attractive for when it finally is indexed.

Oh well.. thanks for the advice and help guys.. I appreciate it as always.

also... just out of curiosity.. are there any benefits to the short vs regular spider friendly URL's?

Is one "better" than the other for certain purposes? I'm just wondering why we have the choice between the two... not that giving us options is bad.. I'm just wondering if there is any particular purpose why there are two versions.

I'm currently using the short version... is there some limitation to using that (IE: large board with many posts?.. etc.)?

Thanks! This thread just reminded me... I set my forum to be indexed when I upgraded to 6.7 B, but what I didn't do is remove the .txt file I had telling spiders to "take a hike". :rolleyes:

robots.txt ?

that'll do it.. lol.

no kidding...

Good thing this is the "Newbie" forum. :rolleyes:

I went through my referrer logs a few moments ago... the nearly 7,000 Flare topics indexed by Google have pulled in over 3,500 hits from google.com alone, and 1,500 from Yahoo (which uses Google). The remaining search engines in the top 30 referrers pulled in nearly 2,500 hits. This figure includes the international Google sites (.ca, .co.uk, .jp, etc), as well as MSN and AOL searches.

In the top 30 referrers, only 7 aren't search engines.

Wow.

I think it's safe to say that the spider friendly work... well, assuming that you don't tell them NOT to work.

So... which is better to use?

Spider-friendly URLs using Paths

or

Short spider-friendly URLs

Thanks

that's my question too...

Thanks also from this version of Holmer.

Guess that line "Disallow: /cgi-bin/" wasn't a good thing to leave in there.

lol definately not... You'll need to take out all referances to your cgi path and noncgi path...

I think that should be added into the documentation...

to leave in what? robots.txt?

Looks like he was having the same issue only it was because he was dissallowing search engine spiders form indexing his board via his robots.txt having a dissallow line of his cgi path...

ahh ok..

still, anyone care to comment on the difference between short and regular spider friendly URL's?

There's none, other than the short URLs being nicer for humans to read.

so there is a difference... the URLs are shorter.

Thanks CC!

Well...

The spiders sure have found THIS board.

I was doing a Google Search of my domain name and number nine on the list was my profile here on this board.

The only "disallow" line in my robots.txt file now is for my web stats directory. It's password protected anyway, but what the hey. Funny how those little details get overlooked.

http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=utf-8&q=site%3Aalforadmin.com+afa&btnG=Google+Search
it's working, I've got 450 pages indexed, I didn't even know I had that many pages

was wondering... the "Use Spider-friendly URL Compatibility Mode?" option... I initially had that turned on. I don't remember turning it on... I just turned it off and it all still works (no 404 errors). Should I leave it off if I can, or is it better to leave it on?

also is it better to use the fake .html extensions or not?

As long as you have no errors with it off, I see no reason to leave it on.


Some bots require a .html extension, so I'd leave it on

ok... all I could see as the difference between the two was the placement of the "?".

I noticed also that removing the "?" and just leaving the .html seemed to have the same results. What does the "?" mean?

some servers require that ? to function (like mine)

just out of curiosity, what is the difference between our servers? Is it a configuration issue, or a platform issue?