Sockets, HTTP, XMPP, and Leap Frog

My efforts at Gnip have caused me to re-consider a few things on the protocol front. Growing up at Netscape in the "netlib" group (really just 3 of us) gave me an interesting perspective on "internet protocols." Over the years it became apparent that HTTP was the end all. Much like twisted-pair copper wiring pervades "connectivity" into homes in the US, HTTP will be around "forever." However, I'm realizing that, also much like twisted-pair, HTTP is becoming a dinosaur in today's modern network infrastructure.

Connection throughput, latency, and reliability is  amazingly good in today's tubes. We had a room

full

of win 16 installations with over a dozen different TCP stacks (winsocks) installed to test a zillion different connection scenarios (yes... socket->open(); "did it work?"). Today, the socket is an assumed reliable, and consistent thing. This is all to say that stateless transport protocols, like HTTP, were fundamental early on. I felt lucky that a response would come back from the same IP address a request was sent to for crying out loud. Binding any state to that connection was something only a company with the capital resources available like AOL/Prodigy/IBM/Compuserve, could pull off. AOL invested hundreds of millions of dollars building out their backend. They did deals with modem manufacturers to ensure 300 baud connection purity and to minimize drops. Madness!

Protocol Leap Frog

Even 15 years ago the stateless (HTTP/open internet) vs. stateful (FDO/SAPI/AOL) connection war was raging. AOL lost obviously, but they made billions with a connection that rarely broke between a machine in your house, and a server in a data center (over twisted-pair copper wiring); the amount of state moving over that connection literally blurred the line between where the app began, and ended (is AOL a client-side app or server-side?).

The battle is back in my mind. XMPP is a relatively defined protocol (sure HTTP is spec'd to death, but every client and server does things differently, sometimes woefully differently)  that has both stateless (connection up/down) models, and stateful (connection always up) models. The latter is what can get really interesting given the new network topology and quality we live with today. An open version of AOL's transport layer basics if you will. Feels powerful.

With today's connection characteristics, I wonder if HTTP would have been the weapon of choice 15-20 years ago? I doubt it.

Many hurdles remain however. The entire network has been built out with HTTP connection assumptions. The way ports are monitored/firewalled inhibit any new connection types from taking hold. App developers think in terms of AJaX & *HttpRequest(), not publish and subscribe frameworks. XMPP is far from ubiquitous, and even if it's lucky, might not even broach HTTP's market penetration radar, but it deserves some time on the field.

Change like this doesn't come quick, and it doesn't come easy.