PDA

View Full Version : URL restricted by robot.txt...?


Paul S
22-May-2007, 02:59 PM
I have just been trying to submit my site map to Google using the url of my site map page. An error message came back 'URL restricted by robot.txt'.

Does anyone know what this means and how to get round it?

Cheers
Paul

pinbrook
22-May-2007, 03:08 PM
There'a a useful website about robots at http://www.robotstxt.org/

FTP to your webspace download your robots.txt file and use the website to rewrite it and upload.

Paul S
22-May-2007, 05:57 PM
Thanks Jo,

But this is Express, so I don't have access to the robots.txt file.

Paul

pinbrook
22-May-2007, 06:07 PM
Sorry, sorry, sorry...

i did it again....

RuralWeb
22-May-2007, 06:38 PM
It may be because Express seems to use a redirect to a url like http://shop1.actinicexpress.co.uk/shops/partsforaircraft/index.php .

Bruce
24-May-2007, 04:46 PM
Malcolm is correct here. The problem is due to the way our Domain name is configured. There is a redirect setup for the domain provided by Malcolm which I will use as an example, www.partsforaircraft.co.uk which redirects to http://shop1.actinicexpress.co.uk/shops/partsforaircraft/ this isn't very search engine friendly so we would recommend that you look at setting your domain up in actinic express by following the instructions at http://www.actinicexpress.co.uk/overview/domain-name-registration.htm (see Using an existing Registered Domain Name).

This will allow much better search engine placement.

The sitemap page in actinic express is not currently compatible with the google sitemaps system, this is something that will be done in a future maintenance release.

Kind regards,

Paul S
25-May-2007, 12:20 AM
Thanks Bruce

Cheers,
Paul

Paul S
25-May-2007, 10:14 AM
Bruce,

I changed everything as suggested and went into my site to check functionality. The home page comes up but that is it. None of the links to other pages within the site work. I have now reverted back to the original settings.

Any ideas?

Thanks
Paul

Bruce
26-May-2007, 08:22 PM
Paul,

You will have to take this up with support so that they can then check to see what is happening.

Kind regards,