I've been trying to formulate some clear thinking for awhile now regarding a major challenge on the network today. It's been brewing for years, and the proliferation of API usage is boiling it over. The power of open discussion is giving the topic some structure and vocabulary, finally, and I'll try to take a step further with this post. Rinse... repeat.
I spoke at Boulder's CTO lunch earlier this week and we got around to talking about the importance of IP addresses/namespaces/blocks in today's API economy. Josh Fraser (attended the lunch as well) does a great job distilling much of the thinking in his recent blog post.
History has taught us that control of information is a powerful economic motivator. Governments and economies rise and fall when regulatory constraints affect how groups of people, or individuals, access to the flow of information/goods. The control may be fueled by greed or survival, but somehow choke points always make their way into the system. APIs in high demand either put business controls on access, or technical controls in place in order to keep backend software running. These controls ultimately boil down to the only entity that is immutable by the time it arrives at a computer system's gateway; the IP address.
IP addresses are discrete and categorized. They are the single unit that can be perfectly controlled at the very edge of your network. Using them, your system can easily determine who to let in, and who to keep out on an individual basis, or by grouping "all requests coming from company X."
The advent of cloud computing allows developers to build applications across large IP address blocks owned by someone else (e.g. Amazon). I blogged about this potential fatal flaw a few months ago. This is a boon for developers, and abusers alike, and it's the latter bucket of individuals that will change the way IP addresses are used for API access. This will be the second coming IP addresses onto the field. The first major control ecosystem for IP addresses came from email and email spam. This new tier will come from APIs, and abuse/spam of them.
For months I'd been trying to figure out how to virtualize the IP address itself. In Ruby land, Heroku has pushed IP address allocation so far up into the stratosphere, that it's the closest thing I've seen to getting the IP address completely out of my way. It's only problem is that I still have to have knowledge of the block of IPs from which it draws.
As a developer I don't want to have to think about the IP address from which my software is making requests. I don't want to know if it's "clean" or "dirty" from the POV of the service I'm trying to access. I just want to access the service, within the bounds of its ToS. However, with cloud computing, I may be being punished because the IP address I'm using may be in a block of addresses that the service provider has "blacklisted" and constrained access to. This is bad and I currently have to spend time thinking about it, and working around it, much like legitimate email senders had to do yesterday. Today however, they can pay an intermediary to ensure the email gets through. I want an intermediary to ensure my API calls will get through.
In the coming months we're going to see router manufacturers make plenty of money providing more configurable IP routing/blocking/management solutions built directly into their firmware. Companies have productized their APIs, and ops teams are going to need easy solutions to managing the IP addresses accessing those APIs.
More significantly, we're going to see IP address brokerages emerge for APIs just as we did for email. Hundreds of millions of dollars are spent each year to ensure email gets through. Email brokerage is a big business, and I'd like to see those firms provide API brokerage as well (hint hint SendGrid).
Saturday, November 21, 2009
Saturday, November 7, 2009
I periodically checkout the Mozilla Add-ons site to see what's new. I just grabbed Reframe-it given that a decentralized client, non-publishing platform specific, commenting model only makes sense. Sadly, no-one's using it; the sites I visited didn't have others commenting on content. This reminded me of the me.dium (now oneriot) sidebar I worked on years ago. Again, another solid decentralized collaborative client (centralized server) product idea, that consumers wouldn't consume. Again, sad.
As consumers why do we continue to do it the wrong way? Relying on publishers to install walled garden user feedback/comment models (IntenseDebate, disquss, etc) is a disaster. When does commenting/discussion break out into its own client/server/protocol (obviously reuse something on that front) so I don't have to rely on the Publisher to have done something w/ their site to support feedback loops. I wonder if Google Wave will gain enough momentum to cause real change here.
The industry is heavily weighted toward open protocols/standards that allow services to cross-communicate (oauth, openid, come to mind). While a good thing, I think we've swung too far to the server-side in this regard. Look at it this way, sans browsers (clients), we've got nothing, yet we're trying to build something (collaboration) based on servers/publishing platforms that have walls between them. That seems horribly broken to me. The model is ripe to have clients (mind you, potentially server-side operated clients... not strictly client-side-software... though in this case I suspect client-side is opportune) act as collaborative agents in order to get a sense of community across the network (not just within various walled gardens) at large.
I don't buy that customers don't want this; lots of people comment on blogs. It falls squarely into the "we don't know what's in our own best interest" bucket. It's going to take a consortium of the major publishing players (from news sites, media companies, blog services) to buy-in to a standard (ha!), or the client (browser) is going to have to force the UI/UX onto us; optional plugins/addons won't cut it.
The network is desperate for a revolution on several fronts. I'm hopeful there's enough steam in the collective invention engine to make some stuff happen. While I've enjoyed living through the advent of the Internet, I don't want it to have been the only major societal shift in my lifetime.