IP Address Brokers; Please Stand Up

I've been trying to formulate some clear thinking for awhile now regarding a major challenge on the network today. It's been brewing for years, and the proliferation of API usage is boiling it over. The power of open discussion is giving the topic some structure and vocabulary, finally, and I'll try to take a step further with this post. Rinse... repeat.

I spoke at Boulder's CTO lunch earlier this week and we got around to talking about the importance of IP addresses/namespaces/blocks in today's API economy. Josh Fraser (attended the lunch as well) does a great job distilling much of the thinking in his recent blog post.

History has taught us that control of information is a powerful economic motivator. Governments and economies rise and fall when regulatory constraints affect how groups of people, or individuals, access to the flow of information/goods. The control may be fueled by greed or survival, but somehow choke points always make their way into the system. APIs in high demand either put business controls on access, or technical controls in place in order to keep backend software running. These controls ultimately boil down to the only entity that is immutable by the time it arrives at a computer system's gateway; the IP address.

IP addresses are discrete and categorized. They are the single unit that can be perfectly controlled at the very edge of your network. Using them, your system can easily determine who to let in, and who to keep out on an individual basis, or by grouping "all requests coming from company X."

The advent of cloud computing allows developers to build applications across large IP address blocks owned by someone else (e.g. Amazon). I blogged about this potential fatal flaw a few months ago. This is a boon for developers, and abusers alike, and it's the latter bucket of individuals that will change the way IP addresses are used for API access. This will be the second coming IP addresses onto the field. The first major control ecosystem for IP addresses came from email and email spam. This new tier will come from APIs, and abuse/spam of them.

For months I'd been trying to figure out how to virtualize the IP address itself. In Ruby land, Heroku has pushed IP address allocation so far up into the stratosphere, that it's the closest thing I've seen to getting the IP address completely out of my way. It's only problem is that I still have to have knowledge of the block of IPs from which it draws.

As a developer I don't want to have to think about the IP address from which my software is making requests. I don't want to know if it's "clean" or "dirty" from the POV of the service I'm trying to access. I just want to access the service, within the bounds of its ToS. However, with cloud computing, I may be being punished because the IP address I'm using may be in a block of addresses that the service provider has "blacklisted" and constrained access to. This is bad and I currently have to spend time thinking about it, and working around it, much like legitimate email senders had to do yesterday. Today however, they can pay an intermediary to ensure the email gets through. I want an intermediary to ensure my API calls will get through.

Industries Born

In the coming months we're going to see router manufacturers make plenty of money providing more configurable IP routing/blocking/management solutions built directly into their firmware. Companies have productized their APIs, and ops teams are going to need easy solutions to managing the IP addresses accessing those APIs.

More significantly, we're going to see IP address brokerages emerge for APIs just as we did for email. Hundreds of millions of dollars are spent each year to ensure email gets through. Email brokerage is a big business, and I'd like to see those firms provide API brokerage as well (hint hint SendGrid).