View Full Version : robots cursor
pinbrook
07-Apr-2007, 05:13 PM
A client has asked me how to block a spider called "cursor" from spidering their site as it appears to be eating bandwidth.
I can find very little about "cursor" except that it refers me to a Hungarian site called adcenter dot hu
anyone seen or heard of this ?
I just want to add an exclusion in a robots file, I can't find anything useful on robotstxt dot org or any where else on www
Bruce
11-Apr-2007, 10:59 AM
The only other reference I have found of this is a search for 'Spider Cursor' that brings up a document on a data analysis tool. Does any one else know anything about this one?
Kind regards,
gabrielcrowe
11-Apr-2007, 12:18 PM
since its in the logs, can i have the ip address and other information thats the spider wanted? what about the files it was interested in? 404s it made? searching for mp3/mpg/swf/html?
edit ==>
look here too, however, you'll not find 'cursor' sounds new.
http://www.user-agents.org/index.shtml
gabrielcrowe
11-Apr-2007, 12:30 PM
is this the string:
Kurzor/1.0 (Kurzor; http://adcenter.hu/docs/en/bot.html; cursor@easymail.hu)
if so, the this:
http://kurzor.hu/
is the search engine spidering your client.
its blockable. its only for hungarian people.
/me spits on hungarian search engine.
pinbrook
11-Apr-2007, 12:43 PM
that looks like the pesky thing...
I've got no more info in the AWstats other than under Robots/Spiders visitors, "Cursor" is the top visitor at 1106 hits this month.
again in awstats you can click on "cursor" which takes you to adcentre dot hu
So to disallow this one i'd use
User-agent: Kurzor
Disallow: /
User-agent: *
Disallow:
or even
User-agent: Cursor
Disallow: /
User-agent: Kurzor
Disallow: /
User-agent: *
Disallow:
vBulletin® v3.8.4, Copyright ©2000-2012, Jelsoft Enterprises Ltd.