AngularJS SEO: Get your site indexed and to the top of the search results.

prerender.io

So, you built a beautiful, snappy, user-friendly website using AngularJS, and now you want more users to start browsing it. Here's how to get your site to the top of the search results.

Serve the prerendered HTML to crawlers, not the Javascript.

AngularJS renders pages using Javascript, but Google and other search engine crawlers don't execute Javascript. When they access your pages, they're most likely going to get a bunch of empty div's.

AngularJS is made by Google, so you know that one day they'll be able to run the Javascript. But for now, and until all the other search engine crawlers catch up, you have to return the completed HTML to the crawler if you want your website to be indexed properly.

The basic steps:

  1. Google notices that your page is rendered using Javascript.
  2. Google requests your pages with a modified url.
  3. You return the prerendered HTML to the crawler.

That's where prerender.io comes in. We provide middleware that detects crawler requests for you. We render your pages by executing the full Javascript behind the scenes, in a browser, then return the static HTML to the crawlers.

Client Side: Follow Google's Javascript crawling protocol

For reference, the full protocol is here.

Google's Javascript crawling protocol is meant for websites that render using Javascript or need to wait on AJAX calls to complete. If you follow Google's protocol, Google will send a request to your server with a modified url so that you can return the static HTML version of the page. Here's how:

If you use urls with #

Nothing after the hash in the url gets sent to your server. Since Javascript frameworks originally used the hash as a routing mechanism, that's a main reason why Google created this protocol.

Change your urls to #! instead of just using #.

angular.module('myApp').config([
	'$locationProvider',
    function($locationProvider) {
		$locationProvider.hashPrefix('!');
	}
]);

Google looks for #! urls and then takes everything after the #! and adds it as a query parameter called _escaped_fragment_. That way, Google effectively fixes the problem of everything after the hash not being sent to your server.

Google finds urls like this:

http://www.example.com/#!/user/123

Google sends a request to your server like this:

http://www.example.com/?_escaped_fragment_=/user/123

If you're using prerender.io, our middleware will see the _escaped_fragment_ query parameter, and automatically serve the static HTML to the crawler for you.

If you use HTML5 push state

You need to let Google know that your HTML5 push state page uses Javascript so that Google can ask for the static HTML page. To do that, just add this tag to the <head> of all of your pages:

<meta name="fragment" content="!">

Google will see this meta tag and send another request to your server to ask for the static HTML page.

Google finds urls like this:

http://www.example.com/user/123

Google sends a request to your server like this:

http://www.example.com/user/123?_escaped_fragment_=

If you're using prerender.io, our middleware will see the _escaped_fragment_ query parameter, and automatically serve the static HTML to the crawler for you.

Server Side: Returning static HTML instead of Javascript

Now that Google is requesting urls with _escaped_fragment_, you just need to return the static HTML.

All you have to do is take the url that Google requested and turn it back into the original url. Then return the static HTML for that url.

Google sends a request to your server like this:

http://www.example.com/?_escaped_fragment_=/user/123

You turn the url back into this:

http://www.example.com/#!/user/123

You can use prerender.io's open source or hosted phantomjs server for rendering and caching your static HTML. You can also roll your own phantomjs server, or even use a grunt task to generate static HTML when you commit. There are lots of different ways...enough for another blog post. For now, just use prerender.io ;)

prerender.io takes care of all of this for you. Your website proxies Google's request to our server where we transform the original url, check the cache for your page, render the page in a browser, and then return the page to the crawler.

Putting it all together

So, to recap:

  1. Tell Google that your pages are rendered with Javascript by using the #! or the <meta name="fragment" content="!"> tag.

  2. Google requests your urls with the _escaped_fragment_ query parameter.

  3. You rebuild the original url from the _escaped_fragment_ on your server.

  4. You serve the static HTML to the crawler for that url.

Now you're well on your way to lots of new users, courtesy of the search engines!

If you use prerender.io, you only have to worry about telling Google that your pages are rendered with Javascript. We take care of all the rest for you!