What Is Unintentional Duplicate Content?

What Is Unintentional Duplicate Content?

Q & A Video

Many of us understand what duplicate content is, but may have missed the fact that 95% of all duplicate content is unintentional, and possibly not even identified by webmasters. Unintentional duplicate content can cause many problems, and can cannibalize your targeted keywords. None of us want this, so watch on to hear more about how to find and fix your duplicate content problems.


Hi, I’m Art Enke, Director of SEO services at Vertical Measures, here to talk about duplicate content. So, today, I want to talk about unintentional duplicate content, because recently, Matt Cutts said that certain kinds of duplicate content, like a terms of service page, is okay as long as it’s not spammy or keyword stuffing. Well today, I want to talk about how unintentional duplicate content actually is a problem. I want to talk about some ways to identify it and how to get rid of it.

Ninety-five percent of all duplicate content is unintentional. Out of the hundreds of websites that I’ve worked on, only a few of them have had intentional duplicate content, like a franchise opening up a new geographic region and replicating the neighboring franchise site. However, the vast majority of duplicate content issues are quite innocent and unintentional. Unfortunately, Google does not give you the benefit of the doubt in most cases when well-intentioned webmasters and business owners are simply trying to do their job and publish their content on the Web.

Unintentional duplicate content can come in many forms, like secure HTTPS pages, development or staging server sites, URL parameters, heavy CMS templates. If you have a WordPress blog, tag pages are really a problem. What all these things do is end up cannibalizing keywords. If you have pages that are duplicate, each one of these titles are competing for each other’s keywords, and that’s why duplicate content is a problem.

So how do you get rid of all this duplicate content that’s unintentional? There are a number of tools that you can use. First, you have to identify the duplicate content, and you can do that through a number of tools like the Moz Crawler, Screaming Frog, Link Sleuth. And once you identify duplicate URLs, duplicate titles, then you can use a number of things. I really like the robots meta tag. It’s a header level directive. It’s a very strong signal to Google that the bots follow every time. That’s the Noindex, Nofollow tag.

There’s rel=canonical. Although it’s not a header level directive, it’s still a signal for search engines including Bing and Yahoo. URL parameters in Webmaster Tools, it’s more of an advanced feature, but you can go in there and you can tell Google bots which parameters you don’t want them to count in their index. You can also return 404 errors or 301 redirects. That’s a great way to clean up duplicate content.

So, to sum all this up, it’s important to recognize that most forms of unintentional duplicate content do negatively affect rankings and traffic. Although duplicate content comes in many forms, a thorough site audit can help identify it, and specific actions can be taken to remove duplicate content and URLs from Google’s index through the use of a number of methods and tools available to well-intentioned webmasters.

That’s all I have today. Thanks for watching.

Worried you can’t find all your unintentional duplicate content on your site? We’re experts at combining the right tools to get you in the clear with search engines. Want to learn more? Contact Us.

Art Enke

As the Director of SEO Services, Art enjoys helping clients earn top rankings through the development of high quality content and stellar backlinks. He’s a technical SEO who enjoys conducting in-depth site analysis, hunting for duplicate content and resolving site-wide issues. On the creative side, he appreciates functional and visually-rich content that engages visitors and moves them through the conversion funnel. +Art Enke