Instant messaging has become a ubiquitous communications feature of the internet. Yet it has been mainly confined to requiring the user to download, install and run a separate application. Prior to the launch of meebo, the concept of an in-browser instant messenger had not been executed well enough to match the features, responsiveness and stability of a standalone messenger program.
As meebo was being built, the technology goal was very simple: the Web site-driven instant messenger service had to work across as many platforms (varieties of browsers, operating systems, and networks) as possible. It had to be extremely responsive. It had to give the user an online communications experience similar to, and as flawless as, that of a standalone instant messaging application.
"That meant no [software] download, communication strictly over port 80, and a truly synchronous communication experience without the page reloading," says Jian Shen, lead software engineer at Meebo, Inc..
Instant messaging sets a very high bar when it comes to responsiveness, because the term "instant" implies and the user expects just that. As Shen explains, "Most users are not aware of Web browser limitations and don't care. If a site advertises instant messaging as a capability, the user is going to expect to send and, more importantly, receive messages in near real-time."
Comet powers the "instant" aspect for instant messaging on the Web
The ability to create an "instant" communications experience through the Web is the strength of the Comet recipe. The use of long-polling via the XMLHttpRequest is a critical technology to meebo. It enabled meebo to implement low-latency bi-directional communication on any modern computer that can access the Web, without the need to download software.
Requests are not expected to return immediately but rather within a specified, longer time-out window (like one minute), hence the "long" in "long-polling." "The 'polling' part refers to the fact that when we receive a response from an XMLHttpRequest, the server closes that connection, and we launch another request immediately afterward -- hence, another 'poll' on the server -- for another longer than usual time-out window, again something like one minute," explains Shen.
There are disadvantages, though, to this method. A typical pitfall in developing asynchronous Web applications is failing to recognize that solid-server architecture is crucial to scaling a Comet-style application. Perhaps the biggest surprise most other developers learn about meebo is that its back-end server architecture has been heavily iterated on from the very beginning specifically to address the needs of instant messaging.
The average meebo session lasts over two and a half hours. For every session, meebo's servers need to maintain a connection with the instant-messaging networks (which include that of AOL, Yahoo, Microsoft and Google) as well as with the user's Web browser. Traditional Web-based middle layers begin to break down very quickly due to memory consumption and the excess overhead of framework creation and teardown for each HTTP request made.
"We decided to skip FastCGI and instead replaced our existing Apache Web server with lighttpd, and wrote a server module from scratch in C/C++ that just spoke plain old HTTP for the browser and server to communicate over. The result was a lean, event-based server architecture that scaled extremely well for instant messaging," says Shen.
But exactly how does the code of meebo's server module differ from that of other instant messenging servers? Shen compares it to other CGI containers like PHP and Python. "If we decided to go with an existing module, we could write a PHP script that grabs request variables and then starts a meebo session, and use PHP's built-in session management libraries to maintain session state. All we've done is packaged up that script's logic straight into the module itself," says Shen. "Another way to think about it is if you wrote a PHP script and then had the PHP module set up on your server so that all it would do is execute that same script for every incoming request. You might as well turn that logic into something that is compiled, rather than something that is interpreted at runtime."
While this solution that Shen and his team devised works well for meebo's service, its architecture probably would not function as well for other similar projects that plan to develop with Comet.
For example, a Web application that requires streaming large amounts of continuous data, like real-time trades and bids in a stock market, would be better off using XMLHttpRequest streaming where the connection never closes and data is continuously processed as it is available. "On the server side, you would optimize for large numbers of clients requesting the same stream of data. Maintaining session state might take a back seat to simply making the data available as quickly as possible," says Shen.
"Doing Comet-style requests to a Web server requires a lot of server-side tweaking long after you're up and running. It pays to think about what you're going to be bound by: CPU, memory, storage or bandwidth?"
Long polling compensates for spotty connections
The first big technical hurdle that the meebo development team encountered was dealing with the limit of two, simultaneously open HTTP connections. But this limitation turned out to be an opportunity because it forced them to be extremely disciplined in the way they built meebo's connection layer both in the browser and on the server.
A couple of interesting design choices came out of this challenge. meebo's developers settled on a fixed long-polling duration that both sides of the connection had to respect. Once that time ran out, the connection was dropped and a new one was created. If either end of the connection became "orphaned" -- for instance -- if the user is on a spotty internet connection, then the other end would know this pretty quickly, because another request was not being made at the designated interval.
On the other hand, if the same spotty internet connection became cut off on a streamed connection -- one where the Comet connection is open for as long as possible -- the architecture may never completely know if the connection has been broken with the server, or if there is an issue with the connection at all.
"The long polling method simply allows us to place our eggs in different baskets, giving us room to recover more quickly should there be a network interruption," says Shen.
While this result would not necessarily have been a "disastrous" architecture design, it would have been more difficult to stably maintain all of those connections on the servers, and it probably would not have scaled easily.
"The current method is more of a side effect, but the end result was a simple and easy to understand design for handling server communication. The two-connection limit had a large part in influencing that design," says Shen.
Could Comet bring us the "live" Web?
Overall, Shen's opinion of the Comet formulation of Ajax, which he and his engineering group at meebo would help to pioneer, is positive. "What started off as a hack appears to be fulfilling one of the most basic needs of the Web, which is live synchronous interaction," says Shen.
He thinks there will be a maturation of the way developers decide to use synchronous and asynchronous Web programming. Traditionally, search engines operate on the static and index-able nature of content on the Web. The "live Web" requires a different kind of search engine from the current search technology used to trawl for Web content. Comet could become the key for encouraging the creation of interesting innovations in making live content on the Web more easily discoverable for the regular user.
"[Web site] metrics are changing and businesses are recognizing that engagement on Web sites are no longer dictated strictly by page views anymore, and that the asynchronous Web will significantly benefit from the synchronous one," says Shen.
"If you find a chatroom event that's a few days old, it's essentially irrelevant. Real-time search will be a very unique problem to tackle. But the end result will be a place where you can find wildly relevant conversations, as well as content around a particular subject, and join in right then and there. That will be exciting."
Architectural Profile: meeboThe Technology:
meebo allows users to access their instant messenger accounts through a Web site. The service works with the IM networks of AOL, Yahoo, Microsoft, Google, ICQ and Jabber.
Software technologies: Gaim/Pidgin, lighttpd, MySQL.
A combo of Dells and Penguins running CentOS.
The Lessons Learned:
Jian Shen, lead software engineer at Meebo, Inc.:
We've learned that there are two levels of scalability breakthroughs. The first is getting your scaling architecture easily repeatable. At this point, you start to get back your nights of sleep because you have metrics in place to monitor your load, and you have a strategy to easily expand resources -- for example, throwing a server at the problem becomes a trivial task. Most companies will get here, and it's certainly quite an achievement.
"The second level involves achieving an architecture that is orders of magnitude more efficient than existing solutions. I don't think we're quite there, and it's important to recognize this because it's just too easy to call it done when you've reached repeatable scalability. At this point, you start to lose your nights of sleep again because you're hungrily striving to break that next barrier. Repeatable scalability is, of course, a pre- requisite and at the end of this cycle, you'll want to be back at square one and start again.
"Specific to Web application development, balancing the workload between the front-end browser and the server is a constant learning process. We still haven't figured out the best balance and we're definitely going to be focusing a lot of our optimization attention in that area in the near future.
"Finally, meebo's desktop user experience was intentionally designed to ease people who are familiar with desktop IM clients into a Web environment and it's been wildly successful with our users. However, the browser limitations -- lack of access to files or rich OS integration -- make some things very difficult for us, because users are expecting a truly desktop IM environment. It's very difficult to communicate to all users why we aren't able to do certain features and a lot of current Web applications out there suffer from this.
"There are some things that the Web does better than the desktop -- for instance, stored state online, access anywhere, native access to external Web content -- and those are the things that Web application developers should focus on rather than focusing on making it as desktop application-like as possible."
Howard Wen is a technology reporter specializing in writing about software development and interviewing developers about their projects. He also contributes to Make Magazine, Computerworld, LinuxWorld, JavaWorld and other technology sites and publications.