After you create a new web page, do you wonder how how much time it will take to get in Google’s Index?  It’s something I constantly check for my own website as well as for my clients’ websites.  But over the past several months, I’ve noticed ultra fast Google indexing when you combine blog posts, Google Caffeine and FeedBurner.

And I mean ultra fast – my posts are indexed by Google in three minutes.  The group at Hobo-Web also wrote about this phenomenon.

(Update) In this case – Google indexed the web page in two minutes:

Formula for (Almost) Real Time Indexing

Here’s how I’m set up:

1. My blog is WordPress
2. The RSS feed is connected to FeedBurner (which pings Google Blog Search)
3. My FeedBurner is also using the Socialize feature to update the post on my Twitter account

Here’s what I have observed:

1. Blog post is submitted
2. Search for the targeted keyword(s) appears in SERPs (search engine results pages) almost instantly as in 3 minutes
3. Rankings bounce around for a few days before stabilizing

Google Caffeine, FeedBurner or QDF (Query Deserves Freshness)

Admittedly, this post raises more questions than it answers, but hopefully you will share your own observations and experiences.

The time to get in Google’s Index is likely tied in to the Google Caffeine infrastructure change, FeedBurner integration and the QDF or Query Deserves Freshness part of Google’s algorithm.

QDF – which is covered in more detail by SEOmoz here – deals with a part of Google’s algorithm which identifies certain topics which are deemed to require more frequent updating.  So webpages covering topics which fall into this category get indexed more rapidly.

However, I believe Google Caffeine, FeedBurner and technology advancements have more to do with this than the QDF part of the algorithm.

Google Does Not Index all Web Pages at the Same Rate

No kidding right? But seriously, in the last two weeks, I’ve experienced three very different indexing situations.

1. Blog Post Indexing
   • When connected to FeedBurner, new posts on my WordPress blog are indexed within a few minutes.
   • When the blog posts are not connected to FeedBurner, it takes significantly longer.
2. Non-Blog Web Page Indexing
   • A client’s home page was revised – content, title and meta information only, not a redesign
   • 12 days later – the changes are not reflected in the SERPs
   • Standard (not a blog) HTML webpage – not connected to FeedBurner or RSS
   • Bing picked up the change after about a week
   • This is a Page Rank 4 website that gets a respectable amount of website traffic
3. New Website
   • Created an e-commerce website for another client two weeks ago
   • Website was added to Google Webmaster Tools
   • Sitemap was submitted
   • Links were added from authoritative, established websites
   • Website is not indexed in Google, Yahoo or Bing after 2 weeks
   • This is what is referred to as “The Sandbox” effect

Shorten the Time to Get in Google’s Index

If you want to shorten the time to get get in Google’s index, hook up your blog to FeedBurner.  Whether it’s the new Google Caffeine infrastructure, FeedBurner or a refinement to the QDF portion of Google’s algorithm is a technology question.  If you want your blog posts to get immediate visibility – consider adding FeedBurner.

By now, most SEO’s are very familiar with Google’s new infrastructure update, referred to as Caffeine. The new infrastructure is expected to roll out after the 2009 holiday season.

Why Caffeine? Because of the way it indexes websites is like old Google jacked up on caffeine (and the old Google was already faster at indexing than the competing search engines).

As of this post, Caffeine results can be seen at this data center: 209.85.225.103.  The Caffeine search results appear about 50% of the time so you may need to refresh your search if the results are the same as current Google.  Try a search by clicking on this link:

http://209.85.225.103/search?hl=en&q=marketing+practicality&btnG=Search&aq=f&oq=

So What Does This Mean For Your Business’ SEO?

Well, your rankings may change.  Caffeine is not necessarily believed to be a change in the way Google ranks website pages, but it does enable Google to crawl more pages more quickly.  So if more competition enters the rankings, it may affect how your business ranks.

On the plus side, I have seen websites jump in rankings.

The websites that improved rankings with Caffeine were a result of changes or improvements to web pages I would have done if Caffeine was there or not.  But the indexing speed is currently incredible – hours, not days.

Optimizing your Small Business Website

No one will really know until everything goes live after the holidays, but your SEO should be working on your website rankings now and checking them in the Caffeine datacenter.