Using Keyword Clustering To Identify Keyword Cannibalization – Case Study No.1

We recently increased a client’s website by 110% by reducing keyword cannibalisation 🚀. Want to know how? Keep reading 👇.

Firstly, what is ‘Keyword Cannibalisation’? It refers to a situation where your web pages inadvertently compete, reducing their impact. Think of it as your web pages playing a game of musical chairs, sometimes ending up in each other’s spots and causing a bit of a mix-up. Despite its significance, many overlook this issue, which can greatly undermine a website’s performance. This phenomenon is particularly prevalent on large e-commerce, travel, or property platforms where multiple similar category pages emerge, either by design or oversight.

For example, say you’re running an e-commerce store and you want to rank for both the keywords “Bluetooth headphones” and “Bluetooth earphones”. You create separate category pages, but oops! They end up stepping on each other’s toes. And guess what? If you’re not careful, this playful tango could happen across many of your categories! On platforms like Etsy, such scenarios are quite evident. Take the screenshot below as an example. It showcases pages titled “Kitchen and Dining Tables” and “Dining Tables.” These differences probably stem from an unintended technical quirk since the pages have distinct types. It seems one might be automatically generated through a search query. Still, this should potentially be investigated and fixed.

We recently used our keyword clustering tool to solve a similar problem for a large travel client of ours. Once our solution was implemented organic traffic increased a staggering 110%!

Please note: This approach is especially apt for detecting keyword overlap on expansive e-commerce, travel, or real estate platforms. For challenges related to cannibalisation on editorial domains or blog sections, there’s a distinct guide crafted specifically for those, available here. I suggest reviewing both guides to determine the best approach for your needs.

You can dive into the video ahead or continue with the text – your choice. However, the video skips the steps to take once you’ve found cannibalising keywords. For those insights, kindly head to this guide’s concluding section.

The Background

Zerodown is redefining the U.S. real estate game with its next-level website tech. While most platforms stick to standard search filters, Zerodown integrates data and AI, allowing users to fine-tune their home hunt. Whether it’s a preference for sky darkness, greenery intensity, or even local voter trends, their platform offers an array of unique criteria. With Zerodown, the home search is transformed into a tailored experience.

As they eagerly pursued further growth in a fiercely competitive market, Zerodown approached us for assistance.

They came to us with a simple brief: to amplify the organic visibility of their expansive website, which had a staggering 50 million pages. Their primary focus was on our expertise to improve the technical elements, enhancing both crawlability and indexation, particularly for those key folders that were missing out on organic traffic.

The Challenge of Improving Indexation of a Several-Million-Page-Site

Examples of the sorts of queries Zerodown wanted to rank for include:

• Homes for sale in California
• Houses for sale in California
• Properties for sale in California
• Homes for sale in Las Vegas
• 2-bedroom homes for sale in Florida
• 3-bedroom houses for sale in Florida
• Log cabins for sale in Boulder
• Wood cabins for sale in Boulder
• Mansions for sale in Los Angeles
• 9 bedroom houses for sale in Los Angeles
• Colonial homes for sale in Orange County
• Ranches for sale in Texas
• Beachfront condos for sale in Miami
• Apartments for sale in New York City
• Townhouses for sale in Seattle

And so on.

There were a number of issues. Firstly, many of their town-based queries were stuck on the second page of search results or beyond. Secondly, a significant number of URLs, in the hundreds of thousands, were labeled as “crawled but not indexed” in Search Console. On top of that, frequently the wrong page altogether.

To get to the root of the problem, we decided to crawl the site. However, our crawler ran into a roadblock. After two weeks, it crashed, having only completed half of the intended crawl – stalling after about 25 million URLs.

It became evident that the website’s massive size was due to a strategy where they created a “property type” landing page for every keyword they identified. Their research led to a list of over 400 different property types, which included:

• Homes for sale in …
• Houses for sale in…
• Properties for sale in…
• Log cabins for sale in…
• Wood cabins for sale in…
• Lakeside houses for sale in…
• Waterfront houses for sale in….
• 9 bedroom houses for sale in…
• Mansions for sale in…

+ another 400-odd more.

These keyword, property type pages would be created for every state, town, neighbourhood and postcode in the USA leading to MILLIONS of pages.

Our Hypothesis

Given the sheer number of pages, the “crawl budget” was likely an issue. For the uninitiated, Google allocates a specific time to crawl a website before it “moves on”. With so many pages, Zerodown’s “crawl budget” was likely strained.

Furthermore, they had multiple pages that seemed to target very similar terms which might have resulted in keyword cannibalisation. For instance:

• Properties for sale in…
• Homes for sale in…
• Houses for sale in…

Or:

• Colonial houses for sale in…
• Spanish colonial homes for sale in…

Or even potentially something like:

• 9-bedroom houses for sale in…
• Mansions for sale in…

Consequently, we spot-checked some of these in our SERP similarity tool. In the screenshot below we can see the keywords “houses for sale in…”, “homes for sale in…” and “properties for sale in…” had a huge 70% SERP similarity.

In layman’s terms? Most of the results across these terms were almost identical. They didn’t (and probably shouldn’t) have 3 separate keyword pages for these terms.

The BIG Problem

We were going to have to sift through over 400 “property categories”, checking each one against each other as well as analysing the search engine results pages. Of course, it wasn’t practical to do this manually, or even using our SERP similarity tool, given the sheer volume and tediousness of the task.

How We Used Keyword Clusters to Reduce Cannibalisation and Improve Indexation

We acquired a list of all property types from our client and appended the phrase “for sale in California” to each. This step was crucial. Without adding a location, a simple search like “farmhouses” often led us to blog pieces describing them, rather than listings showcasing them for sale – the exact info we were after.

Here’s a snapshot of how our list shaped up:

• Homes for sale in California
• Houses for sale in California
• Properties for sale in California
• 2 bedroom homes for sale in California
• 3 bedroom homes for sale in California
• 4 bedroom homes for sale in California
• Castles for sale in California
• Mansions for sale in California
• 9 bedroom houses for sale in California
• Log cabins for sale in California
• Colonial homes for sale in California

+400 odd more

To beef it up further, we sprinkled in keywords from our own research, ensuring we left no stone unturned in potential opportunities.

We were left with 524 “keywords” to test.

After we got the search volume, we uploaded this information into our keyword clustering tool, Keyword Insights.

What our tool does is pretty neat. It dives deep into search results for each query and groups them, determining if they should be targeted on a single page. So, if our report bunched “properties for sale in California,” “houses for sale in California,” and “homes for sale in California” together, it’s a nod that these queries should all be tackled on one page, rather than spread across three.

On top of that, the tool pulled out ranking data and the specific URLs that held those rankings. This made it a breeze to spot if several URLs were vying for the limelight in the same cluster, a telltale sign of self-cannibalization.

So… What Did It Show?

Keyword Insights will send you a report once it’s finished running. You can either open it up and visualise it within the platform itself, or download the Google sheet/Excel version. For this example, it was easier to view the results in Google Sheets as Keyword Insights already makes the pivot table for you.

As expected, we found way too many categories, and many were stepping on each other’s toes, causing cannibalization.

Some were obvious culprits, like the trio of ‘homes’, ‘houses’, and ‘property’ pages. But we also tripped over sneaky overlaps, like the ones among ‘cabins’, ‘chalets’, and ‘log home’ pages.

Below is a screenshot showing the first few property-type pages. On the right, you’ll see property types that can be paired with those on the left:

Here’s a screenshot demonstrating the first 30 or so property-type pages. The right column indicates property types that could be merged with the corresponding property type in the left column:

In the end, we managed to slim down their property-type pages from 413 to just 85 by redirecting all the URLs that could be combined, together.

When you consider all their states, postcodes, towns, and cities, this translates to a whopping cut of about 15 million URLs.

The Results

An incredible 110% rise in organic traffic:

Where Else Would This Work?

As hinted earlier in this piece, this strategy is equally effective for expansive e-commerce platforms and travel websites. For e-commerce businesses, extract a list of all indexable categories, focus on the primary keyword (typically found in the H1 tag), and cluster them to assess potential mergers.

In the case of travel websites, apply a similar method. Determine if there’s a necessity for distinct categories like “chalets in…” versus “cabins in…”. Or ponder if there’s a need to differentiate between “activities in Hawaii” and “top sights in Hawaii”. You get the idea… 😉

What To Do With Cannibalising Content

In our case, we just redirected all the URLs to a piece of “dominant content”. We established the dominant content by pulling in Google Analytics and Search Console data into our pivot table using the Screaming Frog API. It’s out of scope how to do that in this article, but rest assured it is pretty easy to do. See below a list of key steps once you’ve identified your cannibalised content:

Determine the Dominant Content: Start by pinpointing which piece of content holds more weight in terms of performance or value. Look at factors like organic reach, audience interaction, and the number of backlinks to guide your decision.

Combine Overlapping Articles: If several articles address the same topic, think about amalgamating them into a single, in-depth article. This not only centralizes the information but is often more beneficial for the audience.

Redirect Using 301: Once you’ve merged the content, direct the obsolete URL to the enhanced article with a 301 redirect to maintain SEO benefits and direct users appropriately.

Refine the Content: Refresh the headline, meta details, and the body of the content to better align with the most pertinent keywords and ensure its relevance.

Revise Internal Linking: Make certain that all interlinking within your website that previously led to the cannibalizing content now directs to the refined or selected article.

Review External Links: If notable external backlinks lead to the content you’ve addressed, it’s a good idea to get in touch with those websites and ask them to redirect to the updated URL.

Prevent Recurring Cannibalization: Establish a well-structured content strategy and undertake consistent content reviews. Always ensure new content doesn’t duplicate existing topics on your platform.

Track the Outcome: Once you’ve executed these modifications, routinely monitor the revamped content’s performance. Tools like Google Analytics and Google Search Console can help gauge metrics like traffic, position, and user interactions.

In Conclusion

Keyword Insights provides a powerful solution for navigating keyword cannibalisation issues. While our SERP similarity tool examines up to 6 terms, our keyword clustering capability scales to millions. Just assemble your keyword list, upload it, and uncover which terms can be amalgamated under one primary term.

Although showcased in the context of real estate, this methodology is equally potent for sectors like e-commerce and travel, underscoring the widespread applicability of an intelligent, data-centric approach in shaping landing page strategies.

Read our second case study here to see another way of using keyword clustering to identify keyword cannibalisation. The method may be better suited to you.