An Engineer

This idea came to me by way of TechCrunch, which covered the movement of Feedburner founder Dick Costolo from Google to Twitter. In the article, Mike Arrington references Steve Gillmor’s piece from May where he proclaims RSS dead.

RSS never really caught on with mainstream consumers as a way to follow news. But you can’t argue the success of RSS as a technology. Every blog and every major news website publishes an RSS stream. Those RSS streams, which are really nothing more than URLs where you can pull, via HTTP, an XML document with a list of recent headlines and often the full body of articles, make up the basis of news aggregation services like Google Reader, Google News and MyYahoo.

Why didn’t RSS take off with the mainstream? I believe there are two fundamental reaons. First, discovering new content is hard with RSS and easy with Twitter. Second, because Twitter limits itself to 140 characters, the stream does not carry the full text of articles. Viewers are driven back to the publishers where they see advertisments and fully engage with the publisher’s brand. Hence, publishers love twitter. It brings them traffic to their website. Publishers have a love hate relationship with RSS.

Discovering New Content

Twitter is all about discovering new content. By default, you can see which Twitter users any given person is following and follow the same users. There is no central place to this with RSS.

Google has attempted to address this deficiency by adding features in Google Reader to share what you are reading and find out what popular people read via RSS. But these features are buried and only appear in Google Reader. They are not built into the RSS system.

Twitter has a short memorable namepace making offline communication of feeds possible. ABC news can put right on the TV that you follow them at twitter.com/abc. Twitter serves both the publisher of a feed and the consumer of the feed. RSS never had this. The closest thing we had was Feedburner. Consumers have no idea what feedburner is. It means nothing to them. It was glue technology between Google Reader and a Blog.

Getting Publishers on Your Side

Web publishers live and die by getting traffic to their website where it can be fully monetized via advertising. Twitter drives people back to the publisher. RSS takes the content and makes it available outside the source site. That is not publisher friendly.

I use Google Reader to follow blogs. It works amazingly well. Like most folks, I prefer blogs that publish the full article, not just the headline, so that I can stay within Google Reader rather than jumping to the source site to read the article.

By using Google Reader, I completely avoid the advertisements and I miss out on the comments (there are some ads in the RSS stream, but only a small % of what i would see at the publisher’s site). Publishers have a love hate relationship with RSS for this reason. it helps get them readers, but if everyone used RSS to consume the content, the advertising model falls apart.

Because twitter limits messages to 140 characters, putting the full text of articles into the stream is impossible. Instead, nearly everyone posts via a URL shortening service, driving the reader back to the source site. This is much better for publishers who then get to show ads. It’s arguably better for readers too, since they can engage in the comments.

Where do We Go From Here?

Looking at Twitter as a replacement for RSS, the average person, who mostly consumes content, would rarely post anything to twitter. So forget the long tail of people tweeting what they had for lunch today.

I don’t believe RSS will go away anytime soon. If nothing else, Google relies on it today to index blogs posts for search. But RSS will certainly be marginalized by Twitter. Twitter already enjoys much wider name recognition than RSS amongst consumers. I challenge to you find a person on the street who knows what RSS is. RSS will be relegated to being a glue technology on the web.

My belief is that twitter will evolve much like blogging. While there will be a long tail of personal streams that are read by nearly nobody, the vast majority of the traffic, as measured by publisher to subscriber messages, will be dominated by more mainstream publishers and bloggers who will be sending links to less personal and more news-like content. In the world of media, the vast majority of time spent by consumers is spent on media created by a relatively small number of talented content producers. While the web certainly lowers the cost of entry to becoming a publisher, most people still lack the talent to entertain a large audience.

Further, if Twitter is to replace RSS, then the importance of the twitter clients can’t be overstated. Hence, apps like Tweetdeck and Seesmic will shape our experience and be very valuable, just like Google Reader was. Expect Google to create or buy a killer AJAX twitter client to integrate into their suite of applications.

What’s interesting to me is that Twitter will likely win over RSS precisely because it brought together several important features that you need to cobble together using RSS: discovery, managing multiple subscriptions, and reading the streams. The accident of restricting messages to 140 characters so that viewers needed to go to the source publisher sealed the deal by making Twitter much more publisher friendly than RSS. Publishers were more interested in promoting their Twitter feed than their RSS feed.

I wonder what other services that are successful with the geek set but largely ignored by the mainstream can be popularized by fixing the deficiencies and integrating the various parts under one roof.