Duplicate Content

[leehack]

I have just had an enquiry off a possible client who provides 3 ways in which to shop for his products, with the master product duplicated into 2 other sections. This duplication is to counteract the lack of 'shop by brand', 'by price', 'by color' etc. you get from a database driven site. (he does wish to add a 4th shopping method also) Although a little time consuming when he creates products, duplicating things all over the place, it has worked well.

Where it falls down is with regards to duplicate content, Google has not been happy with the site for a long time and we have just had a discussion on possible ways to improve things in this area. The obvious thing seems to be stopping Google from accessing certain pages leaving the products seen only once each. I imagine that using the 'nofollow' META TAG (where applicable) limiting just Google would be a good way to stop pages being indexed, you could make this easily switched on or off at section level.

I can't help thinking that I am missing a trick here though, it may be with regards to how his SPP is setup, it currently uses one product per section, no extended info pages. Maybe this kind of setup should use products and extended info pages and all duplicates could all then just point to the one extended info page? I personally hate using extended info pages, but i can see it might be a way round this, you could effectively have 3 links on 3 different pages, all pointing to the one extended info page. I dislike that extended info pages do not have all the standard SEO hooks as standard though.

Anyone else doing something similar and can possibly pass on some advice in this area? With many sites now showing multiple ways of shopping for products, I think this an area where some good solid sound advice would be useful for others in this situation. I am not sure what is best to be honest and I told him that, so i'd be interested in seeing what more experienced SEO folk think on the matter.

[Orchid]

Were thinking of introducing a similar thing but didn't want to use extended info pages either.

Our solution - though we haven't implemented it yet is to:
have a product with a single paragraph of description.
Build the rest of the single-product-page with fragments.
duplicate the product to multi-product-pages spread through the site so that customers can shop in whatever way suits them
create a user variable that contains the url for a read more link to be displayed in all the duplicate products that goes through to the single product page.
pass through a fake breadcrumb trail to be displayed using clientside scripting.This means that you can allow Google to index the whole site because you won't have duplicate content because:
The multi-product pages will have other products on them too (and lots of sites just have picture and name showing, which could be used to avoid problem entirely).
We can alter paragraph descriptions in the duplicates to go after different keywords at a more general level.

[Darren B]

Hmmmm Time i pulled up my chair and watch. Sorry i dont have anything to add but it has me interested

[Buzby]

We operate this system on a "as close as you can get" to one product per page system without actually fully going that way.

What we do is have duplicates, but as mentioned above they link to an extended info page so there is only one page with that content on.

Our extended info page was completely stripped out and a brochure page pasted in with the product layout to the centre.

For seo we use Full Description variable placed in to the Meta description and use a bit of code that turns the short description in to keywords so that "Red Ford Escort Turbo Diesel" reads as Red,Ford,Escort,Turbo,Diesel.

For an example have a look at the source code of this page that has been added today. And all SEO was automatically done within Actinic. Tianguis Jackson Leaf Set (http://www.qualitysilver.co.uk/acatalog/Tianguis-Jackson-Silver-Leaf-Set-CN554SET.html)

Works quite well on a site with 3000 products which is just too large for us to properly turn into a one page par product layout.

Its not a perfect set up but it does work quite well.

[gabrielcrowe]

well, however the solution, i think the problem wont go away.

even if you were to have a 'products can appear in multiple sections' notion, then you'd still have duplicate content.

take a look at some other sites with shop by brand. dont they also have a similr problem? regardless of fatutures, if you have clever mecnanisms or not, you're still making duplicates, of the same data on different pages.

my suggestion?

use a different template on brand duped products. and re-write the description, or at the very least, shorten it with some php. permute it in such a way that its different enough

[Darren B]

OK So gabe being a man who understands all this stuff, is then possible to create various layouts all containing the master product with its full description. Then using something, php maybe i dont know enough, to use various parts from this description, thus only showing these bits

I dont know if its possible to have code to select various paragraphs, or wether you can put markers in so it can select the sections of text you want it to.

As i said i dont fully understand this side of coding but this would then surely reduce the possibility of duplicate content. But then what do i know :D

[gabrielcrowe]

..let me have a think about it. :)

[Darren B]

You done thinking yet :D

[gabrielcrowe]

yes, i am, and my solution works well, in my test cases.

What i propose is this:
Instead of a section with a duplicate linking to a page with a duplicate on it, why don't you turn the links in the section, to links to the original.

This would mean that:
- ONLY single product per page people could use this idea
- Duplicates never get bought.
- Sections with duplicates are orphaned, since they no longer have links to them.

You'd have to find a way to remove content in those sections, to avoid the duplicate content from being spidered when your sitemap is crawled, etc. i'd suggest putting the sections in your no fly zone for google sitemaps too.

enough talk, here is the code:

<a href="
<actinic:block if="%3cactinic%3avariable%20name%3d%22ContainsSingleDuplicateProduct%22%20%2f%3e%20%3d%3d%20true" >
<actinic:block php="true" >
$SectRef = "<actinic:variable name="SectionID" encoding="perl" selectable="false" />";
//echo $SectRef." - ";
$text = "";
$ProductReference = "";
$connect = odbc_pconnect("ActinicCatalog8","","", SQL_CUR_USE_ODBC);
$query = "SELECT TOP 1 [Product Reference],[Short description] FROM [product] WHERE [nParentSectionID]=".$SectRef;
$result = odbc_exec($connect, $query);
while(odbc_fetch_row($result)){
$ProductReference = odbc_result($result, 1);
$text = odbc_result($result, 2);
}
odbc_free_result($result);
odbc_close_all();

if (isset($ProductReference)) {
$pos = strrpos($ProductReference, "!");
if ($pos === false) { // note: three equal signs
//we dont actually need anything here
} else {
//echo "has duplicates: ";
$originalproduct = explode("!",$ProductReference);
//echo $originalproduct[1];

//this part gets the parent section Id of the page where the original is held
$connect = odbc_pconnect("ActinicCatalog8","","", SQL_CUR_USE_ODBC);
$query = "SELECT [nParentSectionID] FROM [product] WHERE [Product Reference]='".$originalproduct[1]."'";
$result = odbc_exec($connect, $query);
while(odbc_fetch_row($result)){
$nParentSectionID = odbc_result($result, 1);
}
odbc_free_result($result);
odbc_close_all();
//echo " - ".$nParentSectionID;

$sPageName="";
//this part gets the url of the parent section, of that product
$connect = odbc_pconnect("ActinicCatalog8","","", SQL_CUR_USE_ODBC);
$query = "SELECT [sPageName], [nSectionID] FROM [Catalog section] WHERE [nSectionID]={$nParentSectionID}";
$result = odbc_exec($connect, $query);
while(odbc_fetch_row($result)){
$sPageName = odbc_result($result, 1);
}
odbc_free_result($result);
odbc_close_all();
echo $sPageName;
}
}
</actinic:block>

</actinic:block>

<actinic:block if="%3cactinic%3avariable%20name%3d%22ContainsSingleDuplicateProduct%22%20%2f%3e%20%3d%3d%20false" >
<actinic:block if="%3cactinic%3avariable%20name%3d%22IsLoginPageSuppressed%22%20%2f%3e%20AND%0d%3cactinic%3avariable%20name%3d%22UnregCustomersAreNotAllowed%22%20%2f%3e" ><actinic:variable name="SectionPageName" /></actinic:block><actinic:block if="%28%3cactinic%3avariable%20name%3d%22IsLoginPageSuppressed%22%20%2f%3e%20%3d%3d%20false%29%20OR%0d%28%3cactinic%3avariable%20name%3d%22UnregCustomersAreNotAllowed%22%20%2f%3e%20%3d%3d%20false%29" ><actinic:variable name="SectionURL" /></actinic:block>
</actinic:block>
" target="_self"><actinic:variable name="SectionName" /></a>

in this example, i created a variable called: 'ContainsSingleDuplicateProduct' settable at the section level, that turns the feature on and off.

this code block makes a link, so you should put it where yout link would be. if ContainsSingleDuplicateProduct is turned off, then the normal link takes place.

Here's the catch. This will only work for single product per section catalogs, and it'll only work if there is a single duplicate product inside the section you use the code in. If there is, this script will hijack the link and send it to the original product.

sadly, this script is without support, since i just knocked it up to illustrate a point. :D

have fun though!

[gabrielcrowe]

here it is working:
http://www.interact-studio.co.uk/acatalog/
notice the section: 'test'

in reality, it should point here (a duplicate):
http://www.interact-studio.co.uk/acatalog/test.html

this is a duplicate of this:
http://www.interact-studio.co.uk/acatalog/actinic-order-customer-login-php-script.html

in the example, i have made it so that the header takes you to the hijacked page, and the 'more info' takes you to the original page with a duplicate.

[Darren B]

What took you so long :rolleyes:

OK still trying to get my head around it though. Perhaps sunday afternoon when im full of food and beer is not the time to be looking at.

Nice work Gabe - Take this big gold star and wear it proudly, you have earned it

[Rich Brady]

Genuis Gabe... I've been wondering recently whether this sort of thing would be possible, but didn't know where to begin with regards implementing it... :confused:

I've spent most of the afternoon trying to incorporate it on a site of mine, but have hit a few stumbling blocks. Will keep you guys posted on my progress...

Many thanks again Gabe, you're ACE :D

[RuralWeb]

Just finished adding image linking to my short listings pages eg http://www.the-old-smokehouse.co.uk/acatalog/Cameron_s_Smokers_.html . Normal more... link but images also link to SPP.

[Duncan Rounding]

My stomach's grumbling now since seeing those hampers again.

[RuralWeb]

since seeing those hampers again
Yep its Christmas AGAIN and Santas got his sack out. :D:D

[Rich Brady]

I have Gabe's code implemented URL removed (#)

If you click on, either, the Title or More Info... link of the 4 piece set at the top of the page, you will get taken to the original product.

I struggled yesterday and had coding errors poping up around the block ifs associated with the variables, but having started a fresh to day that didn't happen. So I assume it was something to do with my variable yesterday, but can't say I did anything different today:confused:

The only thing I definately changed was the bit of code associate with the Actinic Version, which I hadn't noticed yesterday:

$connect = odbc_pconnect("ActinicCatalog8","","", SQL_CUR_USE_ODBC);

to

$connect = odbc_pconnect("ActinicCatalog9","","", SQL_CUR_USE_ODBC);

[Jethro]

Lee, just to let you know we are operating as you suggest with SPP. We have separate pages for duplicate items so it's easy for the customers to get back to the section they came from.

We use the variable 'ProductID' to name the extended info pages to make sure they all get different page names, otherwise they get over written. (The productID includes the duplicate number.

We then use the Meta Tag nofollow as you suggest to stop Google indexing the duplicate extended info page.

This seams to work OK for us. Although I do admit I would be much happier having just one page for all the duplicates if I was able to populate a list to every section which contained the product. on the extended info page.

[Rich Brady]

You'd have to find a way to remove content in those sections, to avoid the duplicate content from being spidered when your sitemap is crawled, etc. i'd suggest putting the sections in your no fly zone for google sitemaps too.


I've put the nofollow meta tag in the <head> and wrapped a block if around it referencing Gabes Variable that seems to do the job:

<actinic:block if="%3cactinic%3avariable%20name%3d%22ContainsSingleDuplicateProduct%22%20%2f%3e%20%3d%3d%20true" >
<meta name="robots" content="noindex,nofollow" />
</actinic:block>

[Mike Hughes]

Lee, I haven't really been following this thread but if your client has problems with Google then it's unlikely to be because of this.

When Google finds duplicate content then it will remove what it sees as duplicates from the index and leave what it thinks is the original. As far as I know there isn't penalty for this so there's not really much point in worrying about it. Particularly if we're talking about SPP as the customer only needs one of the pages to be in the index. Just let Google do it's job.

Mike

[Jethro]

That basicly what we use, but we use the variable 'DuplicateIndex' which a default an actinic variable, as below;

<actinic:block if="%3cactinic%3avariable%20name%3d%22DuplicateIndex%22%20%2f%3e%20%3e%3d%201" >

[Duncan Rounding]

Presumably all this is only relevant to non SPP. I can't understand the problem with using cut down duplicates all linking to the SPP.

Or am I missing the point here?

[RuralWeb]

Duncan. It's a complex way of making a simple job complex. How many times have we been here over the years?

[leehack]

The problem lies in the fact that neither solution prevents a perfect answer IMO, but like I said if I am missing a trick, then great I can learn something, hence the discussion and raising the point. I've summarised things below as i see them.

CUT DOWN DUPLICATES & EXTENDED INFO PAGES

Pros

Quick & Easy to setup
One overall page that all products link to
No duplicate content
Standard items at product layout level available for layouts

Cons

Incorrect breadcrumb trail presented a lot of the time
Performance woeful on a page with a lot of products
SEO fields not all available as standard
ONE PRODUCT PER SECTION

Pros

Vast improvement of speed and page previews
SEO fields all available as standard
Correct breadcrumb trail

Cons

Have to create a new section for every product duplicate
Multiple pages all doing the same or very similar things

I saw the performance hit mentioned above with my own eyes, a page with 40 products using extended info pages, took almost 4 minutes to preview, yet when converted to SPP via sub-sections (exactly same products), page preview completed in 4 seconds. Nothing wrong with the site and on my high spec desktop. The same performance hit seen by others too as it's been mentioned in the forum a number of times. Factor in that and the missing SEO fields and for me, i don't see it as a done deal decision, BUT like i said i am happy to be shown the light, hence the thread on what i feel is at least a worthy discussion.

[RuralWeb]

IMO - anything that involves the extinfo is a complete waste of time.

using duplicates is more time consuming yes BUT you will spend far more time trying to get the extinfo methods to work correctly and if you do it correctly there will be no duplicate content

[leehack]

I don't like extinfo pages either, but i can at least see a possible argument for them where you may want products provided in 3-4 different ways of shopping, purely because there is only one ultimate page that all duplicates link to, even though there are issues as detailed.

You seem to intimating that you also hate extinfo pages and favour the cut down duplicates route, yet you don't seem to be creating subsections for each duplicate. That got me thinking as to how you do the 'more info' link on the duplicates, being products there is nowhere for them to link to unless of course you manually maintain the more info links and link them direct to the master product section. I can only assume you do yours that way, UNLESS of course I am missing something else, which is what made me start the thread in the first place.

There seems to be at least 3-4 methods being used by people, I guess with your SEO expertise, your method should be one of the very first ones to look at using.

[RuralWeb]

Each "listing" on a multi listing page can be made unique and can be on multiple pages eg brand, colour etc. Each one also links to a true single product page (NOT extinfo) with link text that can be "tuned" and by clicking the listing image.

[Jethro]

I'm with Lee, the way we do it certaintly isn't ideal, but having tried both options we have found it the best way for us.

We only duplicate pages to keep the breadcrumb trail correct.

[leehack]

I think i have it worked out now what you are saying and i can certainly see the advantages of that, especially the tuning of link text, my only initial concern would be how well clients do it, but then again we have that concern on any method we try to pass on. Thanks for sharing.

[RuralWeb]

Clients have to cope IMO - they all want the easy option but there is no gain without pain.