Page Not Found on Google Webmaster Tools - Advice Please!

[GAViN™©]

Within google webmaster tools under the Web Crawl section and Not Found
I have experiencing on a daily basis more and more pages listed. At the moment I have 126 (!) pages. :unsure:

The problem is these pages that are continually being added are pages in this format (notice the uppercase folder ACATALOG)


http://www.dutchwest.co.uk/ACATALOG/phones.html


We are on a linux VPS hosting so I realise the page URL's are case sensitive.

However, the correct page is in fact:


http://www.dutchwest.co.uk/acatalog/Phones.html


So I have no idea where Google is getting this incorrect URL from?

Have checked the Sitemap and all pages are correct.
We do not have a ACATALOG folder in uppercase on the server.
All links on the site are point to the /acatalog/ in lowercase.

Can anyone explain to me why this is happening??

Also, what if in my robots.txt file i added the line


Disallow: /ACATALOG/


Would this block any crawling to that folder in capitals but not block any crawling to /acatalog/

[Mike Hughes]

Did you submit a sitemap to Google? If so I'd check that.

You also seem to have a load of pages in Yahoo that have gone and now produce 404 errors. Is this deliberate?

Mike

[GAViN™©]

Yes have submitted a Sitemap to Google in XML format.
And the sitemap is fine.
Am using the free Google Sitemap Generator from Mole-End to create it.

Also am uploading the urllist.txt file and have that submitted to Yahoo.

We have just moved over from II6 shared hosting to Linux VPS hosting at the weekend, and up until now was recommended by our host to use the meta-refresh method for re-directing pages with old page urls...

Since Monday when we moved to linux hosting, i have been busy creating a .htaccess file for correct 301 re-directs for these pages...Am still in the process of doing this.

Can you post a link please to show the yahoo 404 errors?

Thanks.

[Mike Hughes]

See if this link works:

https://siteexplorer.search.yahoo.com/search?p=http%3A%2F%2Fwww.dutchwest.co.uk/acatalog/phones.html&bwm=i&bwmf=u&bwms=p&fr2=seo-rd-se

It has links like:

http://www.dutchwest.co.uk/acatalog/HD7840.html
http://www.dutchwest.co.uk/acatalog/CDE90.html
http://www.dutchwest.co.uk/acatalog/RT790.html

That give 404 errors.

Mike

PS. Some hosts (such as 1and1) will automatically redirect urls with 'ACATALOG' in them (for example) to 'acatalog'. I just tried this on my own website and there's no problem. It might be worth discussing this with your host.

[GAViN™©]

Hi Mike
Thanks for the URLs
Yeah this is all to do with the 301 redirects I am working on at the moment.
Which is why these indexed pages are appearing as error404 pages.

However those three examples you gave me when you do a search on google for these pages these URLs are not indexed but the new URLs aaaargggh!!

So its going to be a bit of a pain to fix all of this quickly!

Thanks about the heads up for ACATALOG, think I will drop my host an email today about this.

[GAViN™©]

PS. Some hosts (such as 1and1) will automatically redirect urls with 'ACATALOG' in them (for example) to 'acatalog'. I just tried this on my own website and there's no problem. It might be worth discussing this with your host.


Been thinking.. This maybe a tweak to the .htaccess file

Found this:


301 Redirect a Folder to a New Location, But Leave its Contents In-Tact
Let's say you have an entire folder filled with sub-pages on your site, http://www.mysite.com/cars, and you've either renamed the URL structure of that folder to "/newcars" or wish to move it to "/newcars" (so that it looks like http://www.mysite.com/newcars). You'd also want all pages within the folder to remain the same, but just be located in the new folder named "/newcars" - so, the HTML page "cars/toyota.html" will still exist, but will just be located at "newcars/toyota.html". This is the statement you'd put in your .htaccess file, under the line "RewriteEngine on":

redirect 301 /cars/ http://www.mysite.com/newcars/


So I could do a redirect 301 from ACATALOG folder to the acatalog

redirect 301 /ACATALOG/ http://www.****.co.uk/acatalog/

This might work.

[Rich Brady]

Gavin,

I've just created a XML sitemap for your site at the following site:

http://www.xml-sitemaps.com

Having looked at the file I can't see any "ACATALOG", just "acatalog" through out.

The sitemap generator is free you to 500 pages, after that you have to buy there Unlimited version, which I find really useful.

Give the free version a try and submit it to google and see if you have the same issues.

[Darren B]

your existing sitemap is fine, all lower case, you can make a change to you htaccess but you may need to edit the http.conf you probably wont have access to it, but speak to your host they should be able to sort it out for you

there are other ways but using perl & php but this is not something i can say i have done

[GAViN™©]

your existing sitemap is fine, all lower case, you can make a change to you htaccess but you may need to edit the http.conf you probably wont have access to it, but speak to your host they should be able to sort it out for you

there are other ways but using perl & php but this is not something i can say i have done

Cheers.
Have got this sorted now :)
Spoke to my host.

I added the above line of code into my .htaccess file and appears to have done the trick.

Hopefully in a few days when google have indexed my sitemap again these pages will disappear - Where? I dont care lol.

Am going to start using Gsite Crawler (http://gsitecrawler.com/) for generating my sitemaps, see how that goes.

[GAViN™©]

See if this link works:

https://siteexplorer.search.yahoo.com/search?p=http%3A%2F%2Fwww.dutchwest.co.uk/acatalog/phones.html&bwm=i&bwmf=u&bwms=p&fr2=seo-rd-se

It has links like:

http://www.dutchwest.co.uk/acatalog/HD7840.html
http://www.dutchwest.co.uk/acatalog/CDE90.html
http://www.dutchwest.co.uk/acatalog/RT790.html

That give 404 errors.

Mike

PS. Some hosts (such as 1and1) will automatically redirect urls with 'ACATALOG' in them (for example) to 'acatalog'. I just tried this on my own website and there's no problem. It might be worth discussing this with your host.

Realised that these are old links as well, and last submission of my urllist.txt sitemap to yahoo was stamped May,2008 so have this morning resubmitted our newest sitemap file.

[Rachel01]

Hi again, I have been reading this post on Crawl Errors. According to my Webmaster tools I have 4 'Not Found' - they quote my /acatalog/ all in lowercase. I have 2 others that do not mention /acatalog/ but these have 102 links between them - I need to know the following:

What does it mean?
What are the implications?
How can I correct this?

As you may have gathered I am a total newbie to all of this but really need to get to grips with it. We are in a very competitive market but just don't seem to get many visits at all to the site so we must be doing something very wrong.:o

If there is some kind person out there who can offer help and advice, it would be truly appreciated.

Also, another question ... what is the difference between '_' & '-' on my html pages? My site seems to have both and I am not clear if this is a problem or not.

Thanks again for any help.

Rachel

[leehack]

Actinic replaces a space or an invalid character with a _. The best character to use for this as per Google guidelines is -. In v11 this is planned to be changed, for the time being, make sure you manually adjust the html page name of each section to its optimal name.

Rachel-Learns-Actinic.html

is better than

Rachel_Learns_Actinic.html

As for your missing pages, what are the names and do they mean anything to you? Probably just pages that you have changed or deleted and they will get dropped from the index.

[Chunkford]

Be careful when changing URL's
If you were to change the _ to - then the search engines will treat this as a new page and you will lose any value the page may have had before.
If it is an old site then I would just leave them, but if you insisted on changing them make sure you do a 301 redirect in the .htaccess file to pass any value to the new page.