Welcome to Geeklog, Anonymous Thursday, March 28 2024 @ 10:00 am EDT

Geeklog Forums

Bad Behavior2 blocking Search Bots


Status: offline

richard.bkk

Forum User
Junior
Registered: 09/27/08
Posts: 21
curious
As I expected, after some days of testing the latest Bad Behaviour2 I found that BB2 is blocking several search engine bots on our website.

Some, I can understand, they are Chinese, some Japanese, and a few regular lesser known search bots. The latest I found was http://www.exalead.com (193.47.80.51 2008-10-11 04:39:39 Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) Required header 'Accept' missing)

Also, be careful using BB2 if you did some extensive work to get your website registered in web directories which check regularly if the URL is still valid, or if you still have a link back...
 Quote

Status: offline

Dirk

Site Admin
Admin
Registered: 01/12/02
Posts: 13073
Location:Stuttgart, Germany
But then again, do you really want to be indexed by such a clueless company? What exactly is the benefit of being indexed by a small search engine that nobody every heard of (or uses)? If they can't even get their bot to speak HTTP properly (as indicated by BB's reason for the blocking) - what does that tell us about their competence?

bye, Dirk

P.S. The above is my personal opinion only, of course.
 Quote

Status: offline

richard.bkk

Forum User
Junior
Registered: 09/27/08
Posts: 21
confused
I personally belief that BB2 is a good solution if you have a website which depends on mainstream visitors, but if you are running a website in a niche market, say like diving or Asian culture, BB2 becomes less a good idea.

exalead.com indexes about 8,029,402,560 web pages, with enough visitors to generate a serious visitor rate. The hit rate is higher then Altavista... or Ask....
 Quote

Status: offline

richard.bkk

Forum User
Junior
Registered: 09/27/08
Posts: 21
We still testing the Bad Behavior2 plug-in and found that most pings we do to blog directories are being blocked by BB2. I have to say that not all is bad, I can see usefulness in the plug-in I only and to see if there is a possibility to have a white list?

For example, http://www.feedsky.com indexes our website regular, and it is now blocked by BB2.
(Okay, Feedsky is Chinese, but still in China alone are more active Internet users then in whole Western Europe..., we run a few travel websites, and more and more Chinese travelers us our websites)
 Quote

Status: offline

Dirk

Site Admin
Admin
Registered: 01/12/02
Posts: 13073
Location:Stuttgart, Germany
Bad Behavior does have a whitelist in the file public_html/bad-behavior/whitelist.inc.php

bye, Dirk
 Quote

All times are EDT. The time is now 10:00 am.

  • Normal Topic
  • Sticky Topic
  • Locked Topic
  • New Post
  • Sticky Topic W/ New Post
  • Locked Topic W/ New Post
  •  View Anonymous Posts
  •  Able to post
  •  Filtered HTML Allowed
  •  Censored Content