Ouch: Hidden Treasures From A Low-Rent CMS
The below crawl data is five hours after initiation, and 6% complete, having discovered over 700,000 URLs. The three largest competitors to this proprietary CMS – a new client and a first for our agency – will max out around 50,000 URLs with the same crawl configuration:
An exact match site search operator for their bizarre XML Sitemap URL shows this company has at least 240 clients. After an initial look at this crawl and the sitemap, it’s safe to assume the company’s sales deck reads something like:
- 20 Doorway Landing Pages – for every product you sell!
- 100,000s of Dynamically Generated, Self-Canonicalized URLs
Crawling their XML Sitemap index reveals another hidden treasure: 75% of the 28,500 URLs are blog-like content pages within a specific directory. These pages are only linked from the XML Sitemap, represent less than 2% of organic entrances, are duplicated 20 times for geo-targeted “optimization”, and, sampling this data, have no inbound referring domains.
Finally, the site is HTTPS, but looking at Index Coverage within the HTTP version shows an exceedingly slow rate of de-indexing and crawling of HTTP URLs:
I’m sure using relative URLs for the canonical link element isn’t helping.
The company also sells SEO, which is the weirdest part.
Vitamin (C)ontent Deficiency
One constant across automotive retail websites is a glaring lack in user experience differentiation. Nearly each site has thin, copied content, overwhelming above-the-fold carousal images, and unused widgets on their Home page.
Home page content can be copy / pasted, with only the geographic location and the dealership name changed, and could be used by any dealership in the country:
Welcome to [dealership name], our New & Used [brand] dealership has been providing drivers from around the [geographic keyword] area for years with a comprehensive collection of vehicles, in-house auto financing and reliable auto repair and service. If you’re interested in purchasing a new car, truck, SUV or van, we’re conveniently located on [address] in [location], and we’re ready to serve you with the best car buying experience.
It’s as if marrying offsite branding to a User’s landing experience has anything to do with selling cars. It’s unfortunate.
Panda concerns aside, the images dealers have elected to display above-the-fold are overwhelmingly slides advertising the latest deals from original equipment manufacturers, replete with industry jargon and small text detailing financial lease rates probably few consumers understand.
Of the industry, and of the potential to improve upon a User’s experience, below is my favorite quote:
“If I looked at your website would I see that you are community minded or just another dealer wanting to sell me a car? Better put your “Relationship Building” and “Trust” messages right on your landing page else you will look just like everyone else and then it becomes all about best price.”
– Chuck Barker, Dealer Magazine, Jan. 2018, “Shifting From Crisis To Opportunity”
This could be as simple as a photograph of the principals of the business, or of the entire staff, or highlighting the aesthetics of a vehicle showroom, with text-based content detailing perhaps history, community involvement, or whatever else it is that makes the business unique.
Or get creative – could you imagine landing on a local, or even semi-distant, auto dealership website, and among the sameness of your search results, seeing a video of these guys going crazy?
“Our Prices Are NUTS!”, they could say.
And as busy consumers progress the purchasing funnel, this and these user experiences will have certainly influenced their awareness and interest.
Of XML Sitemaps & Unscrupulous Digital Vendors
Thanks to Search Console’s new Index coverage data, one idea was to crawl the indexed URLs of our auto dealer clients, bundling the URLs of sold vehicles into a sitemap.xml file for submission to Search Console (and a sitemap.txt file, for comparison). The additional Sitemap Declaration would be added to robots.txt, and this procedure would be included in monthly or quarterly webmasters maintenance.
The web vendors that dominate the automotive retail space employ logic to populate a website’s XML sitemap with new URLs for available vehicles, as they are made available. Once this vehicle is sold, however, the URLs returns 404 (unfortunately) or 410, and is immediately removed from the sitemap. Googlebot doesn’t get another easy opportunity to crawl this sold vehicle page and drop it from its index. The 4xx URLs linger…
Unfortunately, two of the largest digital vendors for automotive, the majority of which have developed our client’s websites, “do not support this implementation.” One representative added that sitemap files don’t have to be uploaded to a website’s directory, and another that the Sitemap Declaration within robots.txt couldn’t be added at all, as it would “have to be added to all our client’s robots.txt.”
Anyway, hope they were just being lazy, but would have been fun and interesting to test out.
Open Source For The Win
We were able to test the above hypothesis thanks to one of our clients migrating to WordPress. Interesting is Google’s language within the “Error details”. Why would a URL responding 410 be considered an “error”?
Ultimately, we were expecting speedier de-indexing with this methodology, but, whatever – this beats abusing the Remove URLs tool.
It would be interesting to test similarly sized websites (perhaps also with similar crawl data in GSC) and perform this operation but split between an XML Sitemap and a TXT version. During next month’s “batch” of 410 URLs, the last modification (absent in the above) date will be added to test for any differences.
The Increasingly Poor Decisions of A 2nd Party Digital Vendor
Below is a timeline of the mishaps of our department’s favorite web “partner”, who we “work” with on behalf of a mutual client. The client and the owner of this marketing company are, to the misfortune of their digital well-being, friends.
Upon visiting the client’s site for the first time…
- Chrome indicated insecure resources were being requested, and displayed the website as such.
- Inside the .htaccess file, rewrite rules for the different property versions were redirected via 302.
- Analytics data lead to the discovery the 2nd party orphaned the original “specials” page – the optimized one with traffic and inbound links – for two specials pages, to account for New and Used inventories (and without redirection). This was discovered by looking at Exits – the new specials pages required form fills to get the coupons. Exists were well above site average on these pages.
- Users on automotive retails websites will typically navigate to the “Used Inventory” page after landing. The dealership expressed concern they were receiving less leads for used vehicles. Well, the 2nd party had decided to take over hosting. Each time a User selected a new parameter to narrow inventory search, load times averaged 5-7 seconds – I’m sure they chose the cheapest managed option.
- They were not employing a CDN.
- The multi-megabyte Home page slider images were not Lazy Loaded, even though the plugin in-use had this option as a feature.
These issues were resolved, however…
The most unique aspect for me throughout this timeline is that, at the same time, the dealership has a second, older website, indexed by Google for the same business location. The most numerous and most relevant backlinks point to this other domain. For all queries the business would like to rank, this domain has the most real estate on the SERP. Yet, we are instructed to direct our marketing efforts towards the new site.
- Concerns with web performance lead to a site redesign by the 2nd party, and a new, proprietary plugin to display their inventory feeds…
- Static URLs for the KPI landing pages of New & Used inventories were changed – changed, not redirected – to parameter-based URLs dynamically generated by the plugin.
- Structured data for schema in JSON-LD was stripped site-wide.
- The new New & Used inventory pages each had the canonical link element pointed to a new page, /vehicles/.
The client will be OK in the long run, it has just been interesting to witness organic visibility being put into a blender.
Using Google Analytics to Steal Candy from a Baby
A Ford dealer expressed concern that foot traffic on their lot was decreasing after an impressive summer. They were surprised by this sudden change in “ups” from what had otherwise been a solid, positive trend.
Digging into Google Analytics, Sessions had increased nicely until the beginning of September, where they drop by 33% and do not recover. Excluding an unnatural spike in direct traffic on September 14, a month-over-month comparison before and after September 1st showed the majority of this loss came from the Display channel, while metrics within the Organic channel did not significantly change. Direct traffic also took a hit, perhaps as a result of decreased brand awareness from display campaigns, as was explained to the client. The client informed us this drop aligned with when they “quit their previous agency.”
Today, not a week later, the dealership handed us they keys to their $34,000 monthly budget.
Automotive can be an odd bunch, over concerned with their radio advertisement while ignoring trends in consumer preference and digital marketing. Most likely, this dealership simply had grown nonchalant towards where their digital marketing budget was being allocated when severing relations with their previous agency. Thankfully, analytics provided the hard data for an informed decision by the dealership to get back in the game.
When Scale Goes Wrong
“A description for this result is not available because of this site’s robots.txt.”
A site audit illustrated that URLs generated dynamically when a user refined their product search were noindex, and that the canon was set to the parent page. However, these URLs were included within the 1,270 indexed pages a site specific search modifier identified within Google.
An audit of the robots.txt file revealed the problem: Googlebot was prevented from crawling this section by a Disallow entry. Googlebot couldn’t see that these pages shouldn’t be indexed. Most likely, the canonical element and noindex logic was implemented in between the client’s first site with the company, and a recent redesign (it was a proprietary CMS). Robots.txt was most likely then copy/pasted from the old site to the new, without consideration of SEO – an example of scale gone wrong.
When Scale Goes Wrong, Pt. Deuce
A large web vendor in automotive recently developed and pushed a feature within their CMS allowing clients to better manage and post individual special offerings provided at their dealership. And this is nice of them to have done so, as landing pages aggregating coupons for services, discounts, and rebates on new and used vehicles are the third most entered category of pages on automotive websites.
The individual URLs for the coupons and specials have the canonical link element set to the primary landing page for the specials. Except, the URL for the canon does not exist. Sampling 5 other websites using the same CMS reveals the same mishap.
It’s not the biggest issue in the world, as deals come and go (and perhaps the URLs should be self-canonicalized), it’s just surprising to see so widespread. A historical crawl of their domain is in order to see if this URL once provided any value. Moz’s Link Explorer isn’t showing any data, however.
A Prototype for Long-Tail
It’s funny – most automotive retailers will never get to compete organically (versus the AutoTraders or the TrueCars or the CarGurus of the world) for long-tail, high purchase intent queries, like used ford f150 for sale in example town, state, not even for Users in their own backyards. They simply haven’t built pages to do so. Furthermore, by the time any existing vehicle page will have acquired signals for ranking, it would be a problem for the business if this vehicle wasn’t 410.
Below is a prototype to be submitted to the client for additional content creation, and our strategy to take on the aforementioned contenders for these keywords. The page buckets all of the condition/make/model of the product within the internal search results, with supporting content and structured data. Ideally, the dealership will populate the content from their unique prospective and locality – in their voice. Select used vehicles will then be linked from the “Used Inventory” HUB page, likely prioritized by most popular makes and models.
The pages will also assist in an internal linking campaign to boost the visibility of the “Used Inventory” HUB page itself (used car search volumes and physical sales greatly outnumbers those for new). If the client has any high performing content, such as blog posts, supporting these vehicles (unlikely, unfortunately), these will be linked from appropriately. For what it’s worth, Google My Business Posts will be created to highlight these vehicles/pages.
First Links Count Rule
With this experiment in mind, what I once thought simply to be poor optimization of pages displaying new and used vehicles – titles and H1 tags like New Inventory or Used Inventory – a far worse picture is painted:
The anchor text on the links themselves share this same verbiage. Google Trends data shows nearly no one searches “Used Inventory” when looking to buy a used car, and these queries might not even be specific to automotive:
Although we’ve only just implemented this on our own SEO clients, it seems like optimizing the anchor text within the navigation to reflect how consumers are actually searching vs. the verbiage used by the automotive industry is a no-brainer:
Just another day inventorying tags on new automotive dealership clients:
Mishaps in XML Sitemapping
An automotive dealership approached our company interested in digital marketing services, and requested an audit of their SEM operations and SEO.
They weren’t able to locate credentials for Search Console (read: “We don’t know what Search Console is, and we don’t care”), so this data was unavailable during our audit. However, their live XML Sitemap revealed an error: the omission of their state abbreviation at the end of the URLs. Without this affix, not only were the URLs incorrect, but they pointed to a different installation.
The people did not signup with our company, but this we noticed, was corrected pretty quickly after submission of our audits.
It would have been interesting to see data on indexing and crawl statistics before and after the correction.
“#1 Fastest Growing Company in Automotive”
They also made the Inc. 500 list this year. This is the site-wide implementation of the canonical link element, as well as 301 redirection from non-trailing slash to trailing slash, delivered by this web development agency:
Indexability Status: Canonicalised
Canonical Link Element 1: https://www.toyotaofhb.com/bad-credit-financing
Status Code: 301
Redirect URI: https://www.toyotaofhb.com/bad-credit-financing/