Technical SEO:

Cross Domain Content Indexing Guide for CDN’s

SearchDex February 26, 2015

Step by Step Cross Domain Content Indexing Implementation Example

Reading the documentation on cross domain content indexing can be quite confusing even for an experienced SEO and Developer when it comes to getting your content indexed across multiple domains or with a CDN (Content Delivery Network). However, the process is relatively simple once it is broken down into more digestible steps. Below we give an example assuming a domain name of www.altruik.com and a CDN setup with 4 servers for distribution.

We highly recommend reading the part 1 documentation about this process before implementing these steps – please see: http://www.searchdex.com/xml-sitemaps-cdn-cross-domain-content-hosting/

Step One:

Create a robots.txt file for each domain which hosts images or other content.

For example: on i01.altruikcdn.com (which hosts images for the main site) we would create a robots.txt file and put an entry in it pointing to the sitemap on main domain:

User-agent: *
Sitemap: http://www.altruik.com/sitemap-i01.xml

Repeat this for each host in the CDN that has content which is used on the Altruik site (i01.altruikcdn.com, i02.altruikcdn.com, etc).

Step Two:
On the main site www.altruik.com we now create entries in the sitemap for each domain just completed in step one.

For example: in the robots.txt file at www.altruik.com/robots.txt add the following entries (one for each domain above):

User-agent: *
Sitemap: http://www.altruik.com/sitemap-i01.xml
Sitemap: http://www.altruik.com/sitemap-i02.xml
Sitemap: http://www.altruik.com/sitemap-i03.xml
Sitemap: http://www.altruik.com/sitemap-i04.xml

Step Three:
Once the two steps above are completed for each domain, the sitemaps need to be tested and re-submitted through Google Webmaster Tools.

Conclusion:
On completion, this will effectively “prove” ownership of the image hosting domains and will help overcome cross submission errors.  At this point hopefully google will begin to associate indexed images with the altruik.com domain and pages.

If your company struggles with cross domain content indexing due to your content delivery network (CDN). Give our consulting team a call today!

 

Content Delivery Network Resources and References

Reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.1.4

Reference: https://developers.google.com/speed/docs/best-practices/rtt?csw=1#ParallelizeDownloads

Reference: https://developers.google.com/speed/pagespeed/service/PreResolveDns

Reference: http://www.sitemaps.org/protocol.html#sitemaps_cross_submits

GROWTH

A Giant Sporting Goods Store Gets Granular With Organic Search

EFFICIENCY

SearchDex Continues To Grow Organic Search During A Partner’s Risky, Site-Wide Migration.

BRAND RELEVANCY

Trendy Clothing Brand Relies Less On Luck, More On Customer Data To Create Lasting Connections.