PDA

View Full Version : robots cursor


pinbrook
07-Apr-2007, 05:13 PM
A client has asked me how to block a spider called "cursor" from spidering their site as it appears to be eating bandwidth.

I can find very little about "cursor" except that it refers me to a Hungarian site called adcenter dot hu

anyone seen or heard of this ?

I just want to add an exclusion in a robots file, I can't find anything useful on robotstxt dot org or any where else on www

Bruce
11-Apr-2007, 10:59 AM
The only other reference I have found of this is a search for 'Spider Cursor' that brings up a document on a data analysis tool. Does any one else know anything about this one?

Kind regards,

gabrielcrowe
11-Apr-2007, 12:18 PM
since its in the logs, can i have the ip address and other information thats the spider wanted? what about the files it was interested in? 404s it made? searching for mp3/mpg/swf/html?


edit ==>
look here too, however, you'll not find 'cursor' sounds new.
http://www.user-agents.org/index.shtml

gabrielcrowe
11-Apr-2007, 12:30 PM
is this the string:
Kurzor/1.0 (Kurzor; http://adcenter.hu/docs/en/bot.html; cursor@easymail.hu)


if so, the this:
http://kurzor.hu/

is the search engine spidering your client.

its blockable. its only for hungarian people.

/me spits on hungarian search engine.

pinbrook
11-Apr-2007, 12:43 PM
that looks like the pesky thing...

I've got no more info in the AWstats other than under Robots/Spiders visitors, "Cursor" is the top visitor at 1106 hits this month.

again in awstats you can click on "cursor" which takes you to adcentre dot hu

So to disallow this one i'd use

User-agent: Kurzor
Disallow: /
User-agent: *
Disallow:

or even

User-agent: Cursor
Disallow: /
User-agent: Kurzor
Disallow: /
User-agent: *
Disallow: