Tuesday, December 30, 2008

Food Portion Sizes

If you've ever had a meal out-to-eat with me you've heard me complain about the portion size of food served at restaurants; it's disgusting!

Please join me in a campaign to halve serving sizes at restaurants. We, as a society, need to end this gluttonous reign of Fat Ugly Americans (myself included) pigging out on 1lb+ entree sizes, for under $0.10. Guess what people, "more for your money" is exactly what you do NOT want when dining out.

The next time you dine out, ask for half of what you want to order. If the restaurant can accommodate, you'll feel much better after the meal; I guarantee it. If the restaurant cannot accommodate, at least you've made the point that you'd prefer a portion more fit for human consumption. When a restaurant tells me "we can't do half portions of that" I make some comment about how I'd appreciate it if they'd simply cut the portion down altogether and just charge me the full-plate price; again, trying to make the point.

Of course there are exceptions to this rule. There are restaurants that serve too little for the price, but like I said, they're exceptions; let's leave them alone for now and solve that problem another time.

One of the only ways I see restaurants cutting back is if we, consumers, demand it.

A quick plug for a newsletter I've been receiving for over a decade now. A nice, balanced, view into diet and nutrition in the U.S.; Nutrition Action published by the Center for Science in the Public Interest.

Saturday, December 27, 2008

Holiday's and Communication

A holiday party conversation with relatives last night kicked up the "why would someone use Twitter" and "why do the kids text message so much" questions again. There's nothing like getting four generations in a room to illustrate severe communication differences. The older generations are downright angry at the concept of publicly broadcasting your thoughts. The younger one's don't know any other way; it just is.

The conversation brought up some work I did with a friend little over a year ago. It came out of a day-long, two-man, off-site brainstorming session. We were trying to understand social network and communication dynamics. What motivates people to contribute? What motivates people to look at others' contributions? Why bother with all this stuff? We jammed our summarized thoughts into a yin-yang diagram like so...

It goes something like this... we contribute to social networks/products in order to fulfill our need for self expression... we consume the information within social networks/products in order to discover new things... and the social networks/products themselves provide an innate human need to feel connected. Wrap all this stuff together and you have the nut that comprises the psychology and sociology behind social applications. One's contributions to a social product generally reflect their "real-life" personality facets. If someone's an ego-maniac in "real-life," they're likely to be one "online." If someone's passive in "real-life," they're likely to be "online" as well (if they choose to participate "online" at all). You can often draw connections between one's desire/ability/style to blog, and their contributions to various social networks/products.

Most of this is obvious, but some of it is intensely subtle.

Some examples:
  • I check my Facebook feed to learn about what my "friends" have been up to. Discovery.
  • I tweet about ice skating with my family so my "friends" (or geographically distant relatives) can know what I've been up to recently. Expression.
  • I blog about an exotic experience I had in so anyone reading my stuff knows I had the exotic experience (bragging). Expression.
  • I read the blog post from a friend who just went to a concert to learn about a new band I might be interested in. Discovery.
  • Again, wrap it all together and you have satisfied our human need for "connectedness."
The only real differences between today's communication and yesterday's is that today's is permanently "recorded"/stored and doesn't fade with time, and that today's cannot be controlled once the cat is out of the bag. These are BIG differences and can be used for good, as well as evil.

Saturday, December 13, 2008

Update: American cafes, service, and tipping

Last year I posted about how cafés in the US lack table service. I'm pleased to report that my favorite joint in Boulder, Laughing Goat, has started offering table service in the evening. Finally someone in town stepping it up. Thanks LG!

Wednesday, December 10, 2008

SEO?

A sad artifact of search engines having become the navigation paradigm for the web, is that gaming the system came into vogue. Carpetbaggers in the form of SEO "experts" started manipulating content to get better search engine rankings, and selling their wares to unsuspecting websites wanting "better results page placement."

Note that SEM is radically different from SEO, and I consider SEM a legitimate industry.

Guess what? 99.9999.....% of what these SEO "experts" do can be easily done by anyone, and unfortunately, the "experts" have built up such a layer of falsification and search algorithm manipulation that legitimate content has to fight tooth and nail to be properly represented. If you're interested in all the relevant tips/tricks on SEO, all you need to do is buy the seobook; it'll tell you everything you need to know, and it's updated regularly in order to keep close pace with algorithm changes. We'll leave it to Google/Yahoo/MSFT to sort through the math in order to keep the "right" stuff at the top of the page, but the rest of us simply need to focus on what matters; your product and content. If you deserve first-page placement, you will get it; you can't buy it. The few examples I've experienced in my life wherein someone manipulated the model to get first-page placement, the algorithms caught up to them, snuffed them out, and their SEO investment was lost. If you're reading this thinking "but, I've been on the front page b/c of SEO for months now", your days are numbered; figure out a different model quickly.

The upside here is that the marketplace is evolving. Just like any opportunity that is taken advantage of, and over used, it becomes commoditized. SEO is no different. Some friends of mine have recognized this and are building a service that cuts out the noise, and allows folks that want their site to be SEO'd, to have it done by people that know what they're doing (self-regulated marketplace style), at costs that are in-line with the reality of the market; as opposed to glorified SEO gurus. You'll hear about this firm soon enough; I'm excited to see them set things straight.

There is incredible money to be made by a site owner, as well as by people that know how to SEO, if you get "good" placement on a search result page. The industry needs a reset on the model here however, and I'm looking forward to it.

Thursday, December 4, 2008

Browser Innovation

Hark back to the early home computing days when the apps that came with a computer were built/owned by the operating system vendor. Now think about the product landscape and OS advancements that have taken place over the past 20 years. Those advancements and innovations were purely a function of the underlying operating system opening up it's APIs, and supporting the needs of application developers. The result has been better products for the end user, and billion dollar software industries.

Now consider web browsers the new, modern, operating system. They only way to see the same kind of innovative explosion in the browser space is through extensibility. Netscape/Mozilla have supported cross-platform extensions since the beginning. Microsoft has wedged it's proprietary add-on models (leveraging highly proprietary technologies from COM/DCOM, to ActiveX) into IE. Apple, in a series of miscues with Safari, doesn't have an extensible model for their browser. It was a joy to see Mozilla finally take add-ons seriously once they sput out of AOL. While Netscape had a framework, they didn't provide a clearinghouse, so add-ons (plugins as they were called) had limited success as an industry early on. Mozilla finally seems to be getting it right with http://addons.mozilla.org; though there's room for improvement.

I'm really excited to see some friends pulling together something we'd talked about years ago; addoncon (http://addoncon.com/). The fact that industry hasn't been pulling together on the extensibility front to date, is beyond me. I'm planning on attending, and if you care about how the network evolves, I hope you do too.

Sunday, November 30, 2008

_The_ bubble has burst.

Do you think the "Internet bubble" pop was bad? Assuming so, you haven't seen anything yet. The recent investment banking collapse will change our lives for decades to come. I've been trying to coax out a blog post on this topic for a few months now, and Condé Nast's Portfolio magazine, which recently ran "The End" by Michael Lewis, finally inspired me.

The 1990's were so fun!
The net bubble was a function of greedy analysts & company insiders pairing up with investment bankers to jack up IPO prices for newly minted securities in newly fashioned industries (online advertising, various Internet driven technologies, the PC business, etc). The result plowed large sums of money into the hands of many, and consumer spending took its queue, resulting in big ticket items flying off the shelves (jets, 2nd mansions, $100k watches, fast cars, etc). Many became "millionaires," and if you weren't one of them, you at least knew a few first hand.

2nd Verse; the bubble bursts
When the reality of the situation set in, and folks realized the Emperor had no clothes (you can only support massive P/E ratios for so long before you have to illustrate at least some value), the tech sector cratered, taking it's periphery with it. How quaint this mere $5 Trillion wipe out turns out to have been. Not one to go down with out a fight, the Fed dropped federal fund rates, to keep credit markets cranking and consumers spending. The masses, not wanting the memory of free wheeling spending to fade, saddled on more debt in order to keep spending. Money was cheap and this time 2nd mortgages flew off the shelves, along with home equity "secured" lines of credit. The mortgage industry was on a tear, and "regular" 1st morgtages weren't going to be enough to keep things raging; there are only so many homes you can build and buy. American minds were made up, and we were going to continue spending come hell or high water. If our bloated salaries and cash from equity stakes in "successful" investments during the Internet Boom, couldn't float our spending consciousness, mortgage backed debt would!

3rd Verse; popular leverage

Wall Street has always led our financial, and cultural "success" measure, thinking. Big spending bankers, traders, and brokers have had their place in pop-culture for decades. The masses watched in drooling amazement as finance industry employees were bonused beyond recognition, and spent lots of money on toys. We eat the stories of $10m dollar birthday parties up like turkey on Thanksgiving. For a time only the big kids on Wall Street had access to the real money; the kind that few could actually understand how it was created. Then, overnight, the perfect storm of greed (hungry Wall Street execs), derivative innovation (CDO creating quants), and deregulation entered the room; she was beautiful, single, and everyone wanted a piece. Derivatives have always been a neat trick on Wall Street. Leveraging one equity to create another, in a "side bet" manner, is a model that has been around forever. They've always been hard to explain, but until the mid-1990's, the common man could get their head around them with enough explanation and description. Too many abstractions away from the original asset however yields mysterious confusion, and wool can easily be pulled over another's eyes. It was the employee's turn now though; step aside CEO! Relative peons were creating new mortgage derived securities by the dozens, and selling them like hot-cakes to buyers with massive bank accounts (e.g. investment banks, state pensions, school districts, etc). The equity markets couldn't satiate the post Internet bubble appetite that bankers had worked up. They needed a bigger market with tighter focus. Mortgages and associated bonds were the new game in town.

Before anyone knew it, hundreds of billions of dollars in securities were being repackaged (as new derivatives (e.g. CDOs)) and re-rated at ratings higher than the underlying securities' ratings. That was the real trick! Bankers, and corrupt/lame rating's agencies, were turning shit into gold, and selling it back to every industry you could imagine. One thing to note here, Moody's (the bond rating firm) is 20% owned by Warren Buffet. I've forever admired Mr. Buffet but this new awareness has caused me to take a second look at him on my "most admired" list.

Pandemic
Unlike the Internet Boom's relatively scoped collapse around the technology industry, Wall Street had infected one of the largest finance vehicles known to man; mortgages ($15 Trillion worth in 2008). Mortgages, particularly during a housing/interest rate boom, comprise the underpinnings of the private financial industry. They are hugely leveraged debt vehicles. Think about it, you pony up say 20% of a speculative value in a down payment, then pay the rest off over a few decades; crazy! Undermine mortgages, and bad things happen. Enter present day; here we stand, wondering what's next. The money the commoner has invested over the years has shrunk by a full third on average. Nest eggs have cracked. Banks aren't able to lend money for the foreseeable future, and Americans can't buy houses like they used to. Our personal wells of vast amounts of money for consumer spending have dried up.

Now What!?

Obviously no-one knows, but I predict a rather harsh reality in the coming years. Jobs will be lost. Homes will continue to be foreclosed upon. Housing inventory will shoot through the roof, and their prices will fall. Stratification will find its way back into society; as the "haves" and the "have nots" will become more apparent, now that "having" will be more of a function of one's ability to raise capital/earn money, rather than one's ability to plow a hole into the ground with personal debt. The great normalizer, consumer debt, will be reeled in, and things will get weird.

Our banking heroes have fallen, and I wonder who will take their place. One thing is for sure, free-market capitalism always finds a way.

Saturday, November 22, 2008

Anatomy of a shared memory node failure.

After my previous post about teams, I quickly received several requests to provide more details. Voici!

Gnip's redundant; you can walk up to any of our cloud instances, vaporize it, and Gnip chugs along its merry way. We use shared memory (via TerraCotta) to replicate memory across nodes. As you can imagine, shared memory across network nodes isn't all that cheap. Just like anything else, when its over used, things can melt down.

One of our customers started injecting hundreds of thousands more actors into their Filter rules than we'd tested for in a long time (or... ever, in the true production environment (there's a "you can never actually replicate production conditions in your staging/demo/review environment" blog post brewing in me). This caused one of the nodes to start working really hard to build the objects to support the additional actors. In turn, TerraCotta had to keep up its replication, going on its own merry way. The number, and size, of objects we were asking TC to manage (across clients, and three TC nodes as well (one primary, one secondary, and a third for good measure) caused too much lock contention across the system, and TC clients started dropping (heartbeats couldn't be kept up between clients and servers) because they were spending too much time processing locks. Once a TC client drops out of rotation, it has to be bounced in order to reconnect to the TC server. (in shared memory situations, you can't let your objects between client and server get "too" far away from eachother, otherwise you have bigger problems).

So, a node was dropping out of the TC network, we'd bounce it, it would come back up, try to recreate all the objects again, and crater. We'd restart it, it'd come back up.... rinse repeat, rinse repeat. Viscous cycle.

We resolved the issue by dramatically (several orders of magnitude) reducing the number of objects TC was managing in this code path. We optimized the object model to only keep the bare minimum in TC in order to keep our cherished clustered approach; the rest of the state stays put in local VM space, and is not shared.

There were other side effects floating around which got cleaned up in the process which was nice. We reduced some function call times from 45 minutes at their worst, to 45 seconds. We reduced our TC data set size from 16G to a few hundred meg. In the process, we also upgraded to TerraCotta 2.7 which further reduced in-memory, and on-disk, data set sizes.

Teams, Rookies & Vets

A week ago, almost to the minute, the following message was generated by an internal Gnip monitoring server, and sent to the person on-call.

** PROBLEM alert - production-head2/production-head2-gnip is CRITICAL **

It was the start to a very long, non-stop, few days at Gnip.

Much of the rapport on your team gets defined in moments like this. Your team's ability to solve hard, live, problems is thrust into the foreground. About five hours into the ordeal, my appreciation for having focused very hard on bringing software veterans into Gnip was peaking. The problem was being sliced and diced, and the collective experience of everyone on the team was winnowing things down quickly. "It can't be that!" "It must be this!" "I think we should focus here."
  • Step 1: we isolated the symptoms. exactly what was going on!?!
  • Step 2: we checked configurations/environments
  • Step 3: we identified potential code inefficiencies
  • Step 4: we verified probabilities
  • Step 5: we placed a bet on what we thought the problem was, and wrote code to address it
  • Step 6: we watched our hard work pay off; production issue resolved; it was the right bet
Knowing which bet to place comes from experience. The only problem with experience at a startup is that it can be expensive. Like so much in life, you get what you pay for. Had Gnip been tilted toward relatively in-experienced, in-expensive, junior team members, what turned out to be a production blip, could have been a true nightmare for the company.

Glad that problem is behind us, and we all have a nice new chunk of experience to put into the bag of tricks for future use.

Friday, November 7, 2008

Load me, tease me, please me.


Someone kicked computer start/restart/boot times into the media again. Last time I saw this much mainstream media coverage on this topic was a decade or so ago; before I had turned into a Mac user. To cut to the chase, if you're tired of booting/restarting your computer, just buy a Mac which doesn't need rebooting much.

The conversation is akin to a problem we faced at Netscape many moons ago; software load times. When we were battling it out with Microsoft's IE browser, we kept adding features, and hence size, to the binaries that had to load for the app to run; it became a problem. It became a real problem considering more and more of IE code was being baked into core operating system libraries and components, which are loaded into memory at boot-time, not when you fire up IE. Because of the way Netscape was loading it's code, it was much slower to "start" than IE (orders of magnitude slower). Our response was to start loading base libs at OS start; in effect "pre-loading" all the code before the user fired up Netscape. It worked great; solved the problem.

The same model applies to operating systems and their state. If you're having to un-load (shutdown) and load (startup/reboot) OS libraries and components, guess what, you're doomed to incredibly slow startup times. I see a few solutions to this:
  • Run/use an operating system that doesn't leak enough memory or lock-up often enough for a "long boot time" to be an issue. If you're restarting your computer once a week (or more), the start time really matters. If you're restarting your computer once a month, it doesn't matter so much. Switch to Mac. Why *nix based OSes don't crash/leak as much as Windows is a topic that's beaten to death; I won't beat it more here.
  • Build OSes that don't need much code to start.
  • Dynamically load libs on demand (this is related to the previous idea). This one helps, but reality is lots of code needs to load to bring an OS up, so it often doesn't buy you much.
  • Build faster hardware; faster memory, faster solid-state drives, etc. This one's expensive and takes big science brains (rare).
Unless the computer industry tries to come up with new separations of church and state between hardware, operating system, and apps, some combination of above is really all you can do. I do see the need for such feature reach operating systems going away with time. We use probably 5% max of the code that actually comprises an operating system. Specialized devices will eventually win as computer use cases continue to winnow. Most consumers don't need computers that can run database applications, email clients, graphics intensive games, etc.

Now that I wrote that, I think that's the path. 20 years from now, today's OS will be a relic. Hardware will be specialized, and hence its software will be too. Mobile phone OSes are a good example of this kind of evolution.

Wednesday, November 5, 2008

From Defrag to Glue; simplicity

I was on the last Defrag '08 panel yesterday and I had a blast! We were talking about "glue" as a promotion of next year's Glue conference. Geir Magnusson and Aaron Fulkerson were on the panel with me and we had a fun conversation, well stoked by moderator Seth Levine, around the new way to build apps and the various glue components that keep those apps together.

It was a joy to hear Aaron talk about how they've built Mindtouch. It struck a cord with me as it's precisely how we've built Gnip; simply. Bare metal HTTP/REST (we've even built a custom, lightweight, REST layer) is where it's at! Be wary of heavy frameworks to scaffold all of your business logic. Write the code you need, and run it.

The conversation evolved into some SOAP bashing, and how apps should/will be built going forward; always a fun conversation.

The conversation got me excited for GlueGon. I can't wait to dig in even more!

Monday, November 3, 2008

By default...


Software is all about default behavior. From pure infrastructure plays (e.g. Gnip) to general population consumer facing applications (e.g. Apple products).

Whether you want to call your software's configuration, "preferences," "settings," "configuration," or "config," they're all the same, and the choices you make about their default values define your product, and how your product will be used by the vast majority of your users.

If you screw up a "default" setting, from whether or not a calendar entry should "remind" the user that it's there, to how XML is generated, you will find that out when your users engage with your product; or not, as the case may be.

Coming from Netscape/Mozilla, one of the most configurable products known to man (just type "about:config" into your URL bar to see what I'm talking about), and now being a heavy Apple product user (where giving the user options is considered a design flaw), I'm trying to strike the right balance between default behavior decisions and giving end users control of Gnip.

By default, do the right thing.

Saturday, October 25, 2008

BrightKite for iPhone.

The fact it took BrightKite and Loopt as long as it did to distribute their iPhone apps must be some indication as to how niche the market continues to be. That's a bummer because location is going to be the next axis upon which the web tilts. GPS devices in rental cars are the first sign of more of the general population being exposed to apps that do cool things with latitude, longitude (and eventually altitude). However, we're still a long way away from "mainstream" geocoding devices/applications.

Anyway, a few weeks ago I blogged about Loopt's iPhone app; I'm very impressed with Loopt's implicit take on my location, layered with my personal description of my location (sort of akin to BrightKite's "placemark"), it's a model that better fits in my brain. Tried as I did to suck my BrightKite friends into Loopt, I couldn't. I'd say 10% of the folks I invited are still using Loopt today. 60% of them like BrightKite's iPhone/app model better, and the remaining 40% were friends I was trying to introduce to mobile/location apps on their iPhones, and they're simply not interested (or just had an allergic reaction to Loopt).

BrightKite for iPhone
BrightKite's got some UI hurdles to get over. If you're wondering what the first two buttons you see do, checkout this description. Even my friends who were telling me how great BK was couldn't tell me what those buttons did. I still don't know what all the little "lock" icons on the screens mean. Those are all usability/first-version issues that are easily fixed; which is a good thing.

What I struggle with in general with BrightKite is it's take on "checking in." They've catered to those who want explicit, delineated, boundaries around sharing their location with others. That's my primary gripe with BrightKite for iPhone; I want an app that just "checks me in" implicitly (maximum one button click). That said, with geo-location in particular, most users do indeed want explicit control over sharing their location. I'm an aberration in this respect, and I know that.

Implicit & Explicit
The issue of whether to actually share one's location implicitly or explicitly aside (the case can easily be made for that to be an explicit operation), BrightKite's other features need to become implicit.

A toggle for auto "Find me" goes too far in trying to comfort the user. The raison d'etre for the app is to "find me" and because I have to explicitly share my location, just go ahead and "find me." I'll decide whether or not I want to have my location shared. Exposing this as a top-level UI element adds clutter and confusion to the experience. Bury this down in settings/preferences if users really want the option to turn it on/off.

I like the "snap to" concept, but again, to me that's just something that should "just work" and not be an explicit toggle that I can turn on/off. If you're trying to pinpoint me, and you have an idea of where I am, just use that idea. If I have to manually correct it, I will. Obviously use my placemarks as the first order match (which BrightKite confirmed they're doing already), but the relationship between "pick a place" (in the iPhone app) and "placemarks" (on the website) is unclear.

Proprietary messaging in apps today, particularly phones, is extraneous. Just direct SMS in the future please; no need to reinvent this wheel.

Where's the map? A social application based on geo-location screams for the primary UI metaphor to be map driven.

In conclusion...
All in all, the top half of the default screen of the BrightKite app, the "I am..." section, should just go away. All of that can be implicitly derived, either based on my past experiences (e.g. check-ins), or via matching on the backend. I think Loopt has nailed the UI metaphors and the right level of location/sharing abstraction. I think BrightKite is providing lots of privacy-level/visibility and control. Undoubtedly users are asking for that control, but privacy can be a rat hole and I'd be careful building an app around giving the user everything they want on this topic. Loopt's taking some risks in this regard, but I think they're the right risks. Ultimately the end-user will decide.

For now, I'm using both services, and I suspect I'll have to leave Loopt in the ditch pretty soon because most of my friends are using BrightKite. These are social applications and their usefulness is primarily driven by how many of your friends are using them. I'm just stoked that there's competition in this space. Amazing things will come from location enabled applications.

Wednesday, October 22, 2008

Job Applicant Rejection

Gnip has a lot of candidates applying for various roles we're hiring for, and that means lots of applicants won't make the cut; simple law of averages (not to mention a high bar). That means I have the unfortunate responsibility of letting many candidates know that things aren't going to work out. I've been doing this a lot recently, so I thought I'd write about the process.

We have the following hiring process at Gnip, each step dependent on the previous one going well:
  1. 1on1 phone screen. 30 minutes.
  2. 30-45 minute coffee/tea with the whole team. informal get-to-know-eachother. casual conversation about who we are, who the candidate is, and everyone gets a sense of technical depth and capability.
  3. 4 hour on-site pair programming session. heavy technical discussion, and coding session working on actual code that Gnip needs to deploy; feature, bug, or whatever.
  4. Job offer
If any one of the three steps leading up to a job offer doesn't lead to the next step, I communicate that fact to the candidate (usually over email). What gets said in that communication is generally never easy to say. Some candidates wind up not liking Gnip along the way, and for them, hearing that we weren't interested yields a sigh of relief as they got an easy way out of having to tell us they weren't interested after all. Often this isn't the case however, and I have the un-enviable responsibility of telling someone things aren't going to work out. 

Describing exactly why they aren't going to work out is not easy; "it's not you, it's me" doesn't fly. Folks always say they want the brutal, honest, truth (I know that's what I would want), but delivering that on too deep and analytical a level, can often open up a conversation the barer of "bad news" doesn't have time for. Sometimes a declined candidate will want to discuss why our perspective was wrong, or why they were having an off day.

If you're in the position of having to tell someone "no," it's important to be decisive, clear, and to bring closure immediately. I also feel it's important to provide constructive feedback so a candidate has at least some information they can use to understand why things weren't a great fit. Perhaps they have a misconception about how they project themselves, or their skill-set, that they may want to "fix." Giving them enough to go on, I feel, is helpful and important.

Monday, October 13, 2008

Loopt for iPhone; wow


If you don't want to see a grown man gush like a tween at a Britney Spears concert, stop reading.

I've been a sucker for the pairing of lat/lng based data/apps for many years. Threading location into the fabric of the web "only makes sense." From my first mashup with Google's Map API, to my Garmin Forerunner 305 (and associated activities) I've taken the Geo location bait; hook line and sinker.

The advent of a "location" API on the iPhone (firmware 2.+) warmed my heart as it coupled a device I love (and hate at times) with lat/lng. Sadly, the Apple AppStore has been shockingly slow to pickup location aware applications (aka developers haven't written good stuff, or if they have they've kept it off the AppStore). I was further blown-away at Flickr's continued inability to automatically geo locate images I take on my iPhone, into "my flickr map." That one continues to baffle me; especially given that the majority of mobile phone pics uploaded to Flickr come from iPhones.

While Loopt failed to email me when their iPhone app finally went live, I did pull it down yesterday, and I have to say it's the poster-child of iPhone integration.

The Review
Decoupling where I say I am, from the actual GPS location was brilliant. You may see where I am on a map, but I never actually reveal the abstracted sense of where I am unless I want to. Granted, you can deduce it within a few yards very quickly, the separation was a solid way of giving me control over where the software says I am, and how I characterize my location. For example, Loopt knows I'm at latX and lngY, but I get to label that how I want. That can be "in Boulder," or "at my house;" my choice. Subtle, but powerful line to draw.

Of course, full control over whether my location is automatically logged, or manual; a given when it comes to location sensing/publishing.

Integration with the iPhone's contact list and mashing it up with other known Loopt users was gorgeous. The "add friends" button gives me multiple options, one of which is "from address book." Clicking on that shows me who in my address book is already on Loopt, and I can just check them off and "invite" with incredible ease (or invite folks not already on Loopt of course). Apps like these are only interesting if a critical mass of your friends are on them (same old social app story here).

Search for restaurants/businesses has some great defaults right out of the box, and displays Yelp! reviews on map markers right out of the gate; very well done.

Fantastic UI layout with tabs illustrating just what matters to me: map (where I am, and where my friends are), list (same thing in list view), what's up (a textual/comment/picture view of what's going on, and an opportunity for me to leave a "notice"/message about what I'm up to). Clicking on various UI elements does what you'd expect. Great high-level UI abstractions and "at a glance" usefulness, with anticipated deep-dive when I want it.

Ping functionality "pings" friends: basically a way of saying "here's where I am... come join me if you want." Dirt simple, but powerful.

Messaging friends, leverages iPhone's native SMS client. Call'ing friends uses iPhone's phone. And on and on. Basically, whenever you think the app should leverage existing facilities, it does. Loopt avoided the constant pitfall of building everything yourself. As a result, it's a hallmark for iPhone, multi-service/API, integreation. You can imagine the "how do we build a great Loopt app for the iPhone" product brainstorm, and everything that got put on the whiteboard, got implemented.

Gone are the days where I have to explicitly "checkin" to a location via SMS. Too much typing. Location feels like it's just a part of my device experience now and I can choose to not say anything about where I am, or post comments about it and add pictures.

While we're all enamored with the progress of "web apps," Loopt's iPhone App is why client-side software (coupled with web services obviously) is often still the winning combination.

Can't wait to see Brightkite launch the next salvo. A high bar has been put in place.

Saturday, October 11, 2008

"If there's gold in them hills..."

"If there's gold in them hills, people will find it."



I'm sure that quote's been used many times relative to software applications, but it keeps coming up in my life so I thought I'd take a moment to blog about what it means.

If you knew there was gold in that hill over there, you'd go get it. Software's the same thing. If you know there's gold (meaning something really fun/useful) in an application, you're going to do whatever you have to do in order to use it. Obviously, that's a tad absolute, but bare with me. You'll pay money... you'll register for it... you'll fill out a profile... etc.

People spend inordinate amounts of time trying to streamline the barrier to entry to their applications. Minimizing registration processes and building entire API abstractions (e.g. Open Social) on top of social applications is a constant pastime these days, and it's eating up mass amounts of energy that is often better spent on your true value proposition. I'd like to suggest that if you believe the barrier to entry to your consumer facing application is either the user creating an account, or filling out their profile information (for the 100th time), your application isn't worth your target demographic's time.

Streamlining these processes (OpenID, oAuth, Open Social, and the like) is a nice-to-have feature, but it ain't going to make or break your product.

Think about the "real-life" parallels. You'll spend an hour in line at the Department of Motor Vehicles in order to get your driver's license so you can drive. You'll spend 15 minutes on-hold to get the concert tickets you really want. Users will spend large amounts of time to get to your product if it's worth it, so focus on your _product_, not the one-time chunk of time a user has to invest to get to it. I gave Fly Clear scans of my eye's iris, and my fingerprints in hopes of saving a few extra minutes in-line at the airport (not to mention the intense/lengthy application process).

User's have spent the past several years creating potentially hundreds of accounts to get to what they want online. For the truly worthwhile products (facebook, myspace, amazon, etc), they'll invest hours on their "profiles." These nuissances have become understood and are accepted patterns (unfortunately).

OpenID, oAuth, & Open Social are fundamental as infrastructure components, but don't get lost in your product roadmap thinking you need any of these to make or break your application. The only exception would be if your application's target consumption is within an environment that only supports these standards. Smart people are working to make these standards easy to implement. Smart people can implement them without much energy. But, again, if you're finding your spending a lot of time building these things into you product in order make your product successful, pack it in, and go find a different product idea.

Wednesday, October 1, 2008

Gnip 2.0, and feeling good.

Posts like this are not my style, but I can't contain this one. Last night around 10pm MDT we released Gnip 2.0. In the past 15 hours, we've gotten the following public feedback from our users (engineers), speculators, technologists, and various media. This feels.... good. Of course you can view the live stream of twitter comments about Gnip here.

Some great tweets:
  • from a guy in Switzerland that demonstrated Gnip to his Ruby user group just hours after we went live with 2.0. "demo is over. #gnip and #sinatra presented to #RubyonRailsUserGroupSwitzerland. Thanks to the developer of the #ruby-gnip"
  • "bit embarrassing, new Esendex SDK doesn't easily allow mock api for unit testing, Gnip have done a better job, update looms before public"
  • "way to go Gnip on the 2.0 release!"
  • "Gnip 2.0 hits the ground. C'est magnifique, no? ehhhhhh oui. Bien sur."
  • "Gnip: neue Version, kostenpflichtig für kommerzielle Anbieter"
  • "wow Gnip 2.0 pushing the full twitter public feed via XMPP, that changes things."
  • "A web troika story abut Gnip and content aggregation"
  • "Happy days, Gnip have updated their API. TIme to contribute to their C# library"
  • "Gnip 2.0がビジネスモデル付きでロンチ: 複数のSNS間でデータの受け渡しができるというサービスGnipが、今夜(米国時間9/30.. » link to TechCrunch Japanese アーカイブ » Gnip 2.0がビジネスモデル付きでロンチ"
Some great (perhaps a tad overzealous) comments from various media stories about the launch:
  • "Having read the documentation, this sounds pretty cool. I predict it will signal the death of rss feeds. Applications will essentially behave like live stock tickers rather than explicitly soliciting data feeds."
  • "Well done GNIP. You rock!"
  • "Congrats Eric at team! it’s the next gen of RSS. I finally get the name too."
  • "By the time other people even begin realize how they’ve embedded themselves into the very framework of the entire social web, they’ll be a billion dollar company. They’re going to get as big as the social web itself."
That's how releases are supposed to go. So lucky to be working with this team.

Thursday, September 25, 2008

Pinch me moment


If you don't like over-the-top positive/happy posts, stop reading. I don't write these often because they're generally lame, but had to do it this morning.

How much can go well in a 24 hour period?
  • Yesterday was my birthday.
  • My kids made me fabulous cards.
  • My wife made me a cake and took me out for a great dinner at one of my favorite restaurants; L'Atelier.
  • A company I co-founded, Gnip, had a board meeting in which some incredibly good stuff was finalized (more in coming weeks).
  • Some friends and I have pulled together a unique way to draw programming talent from other parts of the country to Boulder; boulder.me.
  • I won a Google Chrome comic book at auction (this is not an indorsement for Chrome)
  • Some important friendships have found a healthier place for all involved.
  • Hiked Sanitas with a bunch of friends.
Many moons ago I left Boulder to go do interesting things at Netscape in Silicon Valley. It was fun... we changed things. But... the Bay Area became a traffic infested, car driving, grind. No longer worth hanging out there after 3.5 years, we came back to Boulder in the late '90s. I miss a lot of the Valley world. I'd long hoped to mash San Francisco together with Boulder, and it feels like it's finally happening. Just the right amount of mash though; a dash.

My home, and work, lives are in the place I love... home... Boulder.

Friday, September 19, 2008

Meme(me)


Via Chris.

1. Take a picture of yourself right now.
2. Don’t change your clothes, don’t fix your hair…just take a picture.
3. Post that picture with NO editing.
4. Post these instructions with your picture.

Tuesday, September 16, 2008

Code Head

If you've ever written code before, you've experienced "code head." If previous statement is true, and you've been in a relationship with someone else, then they've experienced you with "code head." If you string together sentences with operators like "and," "or," "not," "true," and "false," then you've experienced "code head."

It's not like a cold you can catch, or some disease, but it is a state of mind that impacts the people around you when you have it. It's the result of being heads-down in programming, math, or data analysis long enough that your brain continues to loop and process logic problems long after you've left the keyboard. Guess what!? If you have "code head" you're generally not fun to be with until it goes away.

If you're around others (say back home with the family, or out to happy-hour with friends) with "code head," it's obvious, and you aren't interacting with everyone else the way you do when you don't have "code head." You appear sluggish, and detached (because you ARE), and there's an impedance mismatch between you and everyone else.

If you find yourself around someone with "code head," either break off and re-engage at a later date, or help them get out of it by piquing their interest in another topic. NOTE: the latter can be very difficult.

Hmmm, is this a long way of describing someone being distracted? Oh well... my friend Ingrid used the term "code head" today, and I was compelled to write about it.

Monday, September 15, 2008

Clouds & Inverse Sunk Cost Arguments

Over the past couple of weeks I've had conversations with several folks about Gnip leveraging Amazon's Cloud services, and the fact that they are not.

The vast majority of the time it is clear that the other party is not using the cloud because they've already invested so much in their existing infrastructure, and that they're intellectually justifying sticking with their current solution. That said, if your back-end is mature, baked, and generally static, then the financial math likely does justify avoiding the Cloud, but if you're a relatively young application, the Cloud is your friend.

There are certainly times when someone truly has an application that requires physical hardware ownership/leasing, but they are the exception for sure.

Many times folks convince themselves, and their bosses, that they need their rigid, generally expensive, physical hardware (self, or third party, managed) infrastructure, because of their unique high performance needs. So few applications today require "on the iron" cross-machine latencies and processing speeds, and it's embarrassingly obvious at times; does your consumer facing website really need to be served from hardware you own? Don't kid yourself, it doesn't.

If you've already sunk mountains of money into leasing co-lo services, or buying and hosting hardware yourself, and staffing a team to manage them, acknowledging that such cost is now sunk, and that you could/should move your infrastructure into the cloud is a hard thing to do. However, the industry evolves, and it might be time for you to do so as well.

I'm clearly sold on "the Cloud." As a start-up, the flexibility in being able to setup/tear-down dozens of machine instances to support a bump in traffic, or to perform load tests, is a joyous thing. Tough conversations with a managed infrastructure provider about short/long term contracts, or the Operations team, are a thing of my past, and I'll never go back.

If you're avoiding "the Cloud" because you already spent so much on your infrastructure, do yourself a favor and re-evaluate.

Saturday, September 13, 2008

Small World Experience

Every now and then technology works. It's wild when it does. I had such an experience yesterday that made me realize how small the world has become. None of below is remotely close to mainstream (I need to do another blog post on technologies we "think" are mainstream, but are far from it, e.g. RSS), but it was an amazing experience.

A colleague, Kevin Marks, was sitting in a conference (BearHugCamp) that he thought Gnip (either Eric or I) should be attending. He emailed Eric saying as much, and Eric forwarded the mail to me. Within the email was a link to the live webcast being hosted on twit.tv. I clicked the link, click-ed the video into full-screen mode, and I... was... there. Within 60 seconds of Kevin making the suggestion, Gnip was "there." The content was indeed totally relevant, and I had the sense I was "attending" a conference taking place in another state. I gleaned enough to find value, and I moved on.

Not much to the story, but it made me realize just how far live video feeds have come. Latency isn't much of an issue anymore, quality is high, and full-screen is supported by every player now.

The machine is evolving.

Monday, September 1, 2008

Chrome


It's been a long time coming. Google's impending release of their web browser marks the first notable new browser entry since Apple with Safari. I still think Apple made a big mistake going with Webkit/KHTML for rendering with Safari (over Mozilla Gecko), but what's interesting about Chrome isn't that they chose Webkit too, rather the "fresh look" at building browser in the in the 21st century. It's a different world today.

Gecko & Webkit
I'd prefer modern browsers leverage Mozilla Gecko as it covers more rendering compatibility breadth than other engines for sure, but folks continue to be impressed with Webkit's rendering speed. I don't perceive enough of a speed difference between the two, but load tests indicate otherwise. Whatever. Rendering may indeed be becoming commodity. Hard to imagine, but it is 2008.

Javacript via V8
Google's choice to go with V8 as a Javascript VM is monumental, and could be a real game changer if Chrome gets any market share (I predict sub 10% for years). Ground up JS engine construction can't be taken lightly, and I am impressed, again, at the perspective; looking at JS engines in today's web app heavy reality. It is kind of incredible that JS UI drag/drop on IE/Mozilla based engines is jaggedy. I mean c'mon. V8's performance could wind up being impressive given that they leverage precise garbage collection, actual machine byte-code compilation and execution, along with hidden classes; all overdue. I wonder how it stacks up against Tracemonkey?

Put It On My Tab
Breaking pages/tabs down to process-per level is an age old idea, and it'll be neat to see it in the wild. I'm sure it'll have some benefits (obviously execution/memory/crash isolation), but intra-browser expectations we've all grown up with will take time to get right. Simple things like visited link coloring (in _real-time_), cookies, and session history all will have to be synchronized where they often weren't before (though IE started sharing cookies across processes roughly 13 years ago).

Gears & Software Patterns
I'm most excited at the notion of baked in Gears support. The Gears work Google's been doing of late is the most innovative client-server web app work I've seen. The prospect of this kind of functionality being an integral part (not an add-on) of the browser is revolutionary. Again, here comes that pesky market share penetration challenge. Without server-side apps leveraging Gears functionality natively, it'll be destined to research project status.

I have to wonder if there's room for Gnip integration either in Gears, or in the browser itself. Event driven actions across the network are a major part of modern web apps, and tighter integration with client-side software could be tasty.

Design
The UI design stuff is "meh" at best. It's design, and promoting tabs to the top-level UI element feels right, but.... it's design. Whatever. If you don't get it right, someone else will.

Looking forward to trying it out...

Friday, August 29, 2008

What kind of Cloud is your solution?

Cirus - What IBM is building.

Cumulonimbus - Amazon

Stratus - Mosso

Cumulus - Google App Engine

Fog - we've seen a few of these crop up. they're generally random, small, traditional managed hosting providers jumping into the game with 25% baked APIs to run on their virtual slices.

Friday, August 22, 2008

iPhone 3G/2.x firmware disappointment

I had two concerns just over a year ago with leaving my Palm Treo 680 for the iPhone. One, how was I going to do going from a physical keyboard with plenty of tactile feedback, to a touch screen with none? Two, how was UI responsiveness going to be?

Physical, tactile, UIs are generally a requirement for me. "Touch screens" usually suck, but, I've gotten over that hurdle with the iPhone. The benefits (and new UI gestures) outweigh the issues; I've adapted.

iPhone firmware 1.x releases abated my fear around UI responsiveness. Contact lists loaded fast enough, and text input fields acquired cursor focus fast enough. My fear w/ the initial iPhone rev was unfounded.

I've been using the iPhone 3G with the 2.x (yes, the latest update as of this writing) for awhile now, and I'm utterly disappointed with the UI responsiveness at the OS/UI framework level. The waiting I have to do at text entry fields (iTunes password prompts, contact search entry fields) has negated the speed increases that the 3G network provides for data transmission. Net-net, I'm still wasting the same amount of my life waiting on the device to do something, that I was on the old first gen network and 1.x firmware.

The only reason I bought the new 3G device was for increased network data transmission, because I wasted too much of my time waiting for email to download, and web pages to load. While that issue has been resolved, I'm now wasting all my time waiting for client-side UI elements (namely entry fields) to accept input. Sad.

I suspect many/most Blackberry users will avoid the iPhone based on this issue alone.

Please fix it Apple.

Monday, August 18, 2008

Joyous Milestones & Unfortunate Circumstances

The family went by my son's kindergarten classroom this evening to check things out (which desk is his, meet other families, etc); it was adorable. All these little kids jumping into the big education pool. I can't wait until Wednesday; his first day of school.

We opted for public schooling in the Boulder Valley School District for our son (and daughter by extension over time). Overall we're very pleased with the schools and teachers in BVSD (the district administration/body itself is another story, and another blog post). We noticed something odd at the school tonight however. At least half of our neighborhood families/friends who have kids in public school weren't there. They'd opted to Open Enroll in other BVSD schools. Open Enrollment boils down to your child[ren] being able to attend the school of your choice (e.g. not your neighborhood school) if said school isn't full. Several of those families were fighting hard to keep our previous neighborhood school open just before BVSD shuddered it a few years ago (yet another, separate, blog post topic), generally on the grounds of it being a "neighborhood" school. So, it's feeling hypocritical to now not be attending the new "neighborhood" school.

We knew some of our friends were not going to be attending their neighborhood school for awhile now, but it really sunk in this evening now that school is actually starting. The closure of our previous neighborhood school literally tore the neighborhood apart in the end. Families are scattered all over town now. Walking, biking, driving, busing their kids all over. Close friends that we'd anticipated walking/biking to school with, over the course of the next several years, will not be part of that experience now. Sad.

My reasons for being upset over our initial neighborhood school being closed in the first place, are further solidified now. I'm a proponent of choice in systems, and Open Enrollment is something I still believe in, though I'm questioning it for the first time. I would have appreciated those families that were standing side-by-side with us on the "neighborhood schools are what matter" platform, to have stayed true to their colors. In the end however, we, as parents, all make choices we believe are best for our children, and I can't blame anyone for those decisions if they differ from mine (we all have our own criteria). I do continue to blame BVSD for continuously making bad choices. The district is simply too diverse and large for one body (the puppet Board which simply ratifies a bloated administration/staff/consultant agenda) to effectively manage. I'm a supporter of a Boulder group seeking to split the district into more localized, attentive, districts, and I'd encourage you to get involved as well; http://www.bvsdcape.org/.

All that aside, and back to where I started on this post, we are overjoyed about our neighborhood school. Our interaction with its staff has been positive thus far. We love the facility itself. The teachers, at least those we've interacted with, are great. It's part of the International Baccalaureate program which takes a global view on education (e.g. "civil war" occurs all over the world, and has many flavors/outcomes), as opposed to the fiercely U.S. centric view that pervades public education in general. We look forward to being a part of our neighborhood school, and welcome everyone else in our neighborhood to join us. We are ecstatic to be with the neighborhood families attending as well.

Thursday, July 31, 2008

What matters in a career, a company, and a Product.

This post started as an internal email, but I realized I should post it here for would-be Gnip applicants, and for anyone who may want some insight into how my brain works.

I consider myself one of Earth's luckier human residents. My post-University career started with a Netscape founder doing interviews one day at the University of Colorado at Boulder, just prior to graduating way back when. He interviewed me and passed me along to employee #5 for perusal. I flew out to Mountain View, interviewed, and was offered a job in the Networking Library group (netlib) for the core browser; I took it.

I had several offers coming out of college. I took the one with Netscape, not because of what Netscape was, or was becoming, but because of the people. They were great. They were smart (some of them genius). They were strong. They were engineers. They were business people. They were passionate. They were pragmatic. They were FUN! We are all great, lifelong friends. We lived and breathed "the browser." Trips to Tahoe for skiing in the winter, and boating in the summer, were a beautiful mix of work and play. I'll never forget debugging a User Agent problem over the phone with someone while riding up on the ski lift. In those early years, I learned how to code, build, have fun with "work" and how to be part of a team.

The wisdom imparted on me, the new guy, has guided me since.
  • "all that matters is percentage equity; fight for it. salary pays the bills; nothing more."
  • "do not be a jerk."
  • "do not work with jerks." (I violated this _once_ and paid a very heavy toll)
  • "do not settle for a half baked idea."
  • "engage with prior success."
  • "create small, tight, feedback loops, and make sure they always close."
  • "if you see a snake, kill it. do not play with dead snakes."
  • "it's a bug."
  • "we're doomed." (it's a figurative, contextual, sarcastic thing; not literal.)
  • "A players hire A players. B players hire Cs, Cs to Ds. Party's over."
  • "think of the world in terms of URLs. life is on the other end of them."
  • "HTTP is all that matters. People will layer on crap from there."
  • "do not break the build."
  • "make sure you're always working on the coolest thing."
  • "if you manage people, fight for them to the death."
  • hire people who are smarter than you. if you find you're the smartest person in the room, something went drastically wrong; restart.
Booms cause these rules to get twisted and watered down and broken. People get hyper and take short cuts, and things fail (bubbles burst).

While I have no desire to live in the past, taking learnings from it, and applying them today is fundamental. I'm back home in Boulder, CO now, and my friends and I are building Gnip. We are building a great product, and a great team, in a great place. Join us.

Wednesday, July 30, 2008

Gnip is hiring.

Gnip is looking for a few new engineers to join our team; checkout our job posting here. We've got a great product going already, with phenomenally positive, and voluminous, public response, but we need to get more functionality into the system, and we need help doing it. We're still very small and young, so if you fit the bill, your involvement will have tremendous impact on the product, technology, team, and company.

We're located in downtown Boulder, CO, so good food, coffee, and tea are all within arms reach.

Thursday, July 10, 2008

me.dium social search is here

after a lot of work, me.dium social search is finally here! blending real-time human behavior with search results provides a powerful view into information on the network. it's in its infancy, but I encourage you to check it out here. http://me.dium.com/search

if you like it enough to want it in your search bar, you can install it from here http://valeski.org/me.dium.social.search.install.html

Friday, July 4, 2008

Who's in Boulder this summer?


I live and work in downtown Boulder, and as a result, I'm always around the locals and tourists that comprise the city. Each summer Boulder sees its fair share of tourists, and I like to anecdotally measure the origins of the tourists by listening to languages and accents people are using.

Here's a list of languages I'm hearing this summer (American English & Spanish excluded) in descending order of frequency.
  • German
  • Japanese
  • French
  • Korean
  • Russian
  • British
  • Arabic
  • Czech
  • Mandarin
  • Hebrew

Saturday, June 28, 2008

Flagstaff Amphitheater == Montalvo Amphitheater?

Years ago I was lucky enough to see a show (Amiee Mann) at the Montalvo Amphitheater in Saratoga, CA. The small outdoor venue was ideal (as was the music). Not having ready geographic access to it anymore is a bummer. I wonder if Boulder, in conjunction with Boulder Theater and/or BMOCA, could support a similar Arts/Theater/Concert Series via the Flagstaff Amphitheater? Bus people up from downtown, drop them off, then pick them up after the show. I guess Chautauqua's Concert Series kinda does what I want, but the venue, while neat, is pretty poor in terms of intimacy and comfort.

Thursday, June 12, 2008

Clouds and Sympatric Speciation

I just canceled Gnip's account with its traditional hosting provider. We had a good experience with them, but we moved our stuff to the AWS/Ec2 recently so they became obsolete for us. I also just received the following email from our account representative there (a good guy).

"Hi Jud. I'd like to keep you as a customer. I think that PROVIDER-NAME has a good reputation for working hard to provide good service. We have some great hardware and data center, too. I am prepared to create a web-based API for you to create, resize, shutdown, reinstall VPSs. You'd be able to do that on host servers you own (most cost effective) or on any of our regular shared host servers (which would give you the instant scalability you need). Plus with our setup each server instance would retain its IP address between restarts, and you'd have a file system that remained between reboots (and did not require re-setup), plus you'd still have the option of using S3 for shared data or large data sets if you needed it. What would it take to keep you as a customer? We can implement most things for you and hopefully we can save you a lot of time, hassle and effort changing hosting providers."

The email struck me as a form of sympatric speciation. Sympatric speciation is an form of evolution in which two things from the same pool evolve in different directions (for a variety of reasons). All the hardware on Amazon's racks is the same as that in my previous hosted provider's racks, yet, Amazon just sympatricly speciated away from its genetic bretheren. If evolution has taught us anything, it's that the "other species" does not go down without a fight. I'm reminded of some lyrics by Dave Matthew's "Don't Drink the Water;" "Whats this you say, you feel a right to remain? Then stay and I will bury you."

Tuesday, June 10, 2008

Calendar Dancing

Every few months I sit down and try to mash all my calendars together, in hopes that the various calendar services I use have fixed all their bugs. You'd think that in today's world of well known ical formats, ics subscriptions, and new browser protocol handlers, webcal://, that things would work well together.

The energy is definitely here. Dozens of products and services offer calendar "imports" or "subscriptions" which purport to import calendar entries. Some do this well, but most either don't work with arbitrary services (e.g. they only work with a select set of other calendar feeds), or are severely buggy. I'll keep dreaming of seamless integration and mashups, but for now, here are some prohibitive issues that consistently come up.

HTTP Basic AUTH and Subscription access control - If you've been anywhere near me over the past few months, you'll know my stance on HTTP Basic AUTH. In short, it's woefully underused. An old colleague has a great post that personifies my perspective. If more software supported HTTP Basic AUTH it would be the perfect way to support access control to calendar subscriptions; something that effectively doesn't exist today. Protocol level auth is often the only way to resolve this kind of mashing up.

Google Calendar - oddly the worst at importing/exporting. I'd expect a company known for parsing/derivation to have imports nailed; hardly!
  • webcal vs. http schemes - the transport is identical, please just treat the URLs the same.
  • get your own imports to work. I can't even import a Google Groups calendar into a Google domain hosted calendar.
  • importing 30boxes shared calendars via tags doesn't work. the same calendars imported into Apple iCal work just fine.
30Boxes - my personal favorite.
  • again, webcal vs. http schemes - they're the same thing folks... allow both and determine what format you're dealing with in the HTTP response that comes back.
  • Many ics subscriptions are PRIVATE and use the VFREEBUSY blocks for entries. Please support these so I can import more content into 30boxes.
Dopplr - great tool for communicating overlapping travel schedules
  • I've noticed webcal vs. http scheme madness here and there, but overall fabulous import facilities.
Apple iCal - great, except it's client-side, and I need hosted calendars for arbitrary endpoint/location access.
  • the gold standard in importing and exporting. if it doesn't work here, it won't work elsewhere. I wish other apps worked this well.

Wednesday, June 4, 2008

You've Got My Vote, Go Get Someone Else's!

It looks like the Democratic party might have what it considers a reason to select a candidate. Boy oh boy that would be nice considering all the intra-party bickering, and money wasting, that has been going on for nearly a year, while McCain has been lounging and resting as the Republican's nominee. If you didn't catch his appearance in a recent Saturday Night Live "Weekend Update" bit, check it out, it's spot on!

I'm embarrassed to be lefty after this mess.

Living in a "liberal" town, Boulder, CO, I'd ask that we, as Dems, not spend the next year promoting our candidates and party in our own backyard. Guess what, I'm already sold on our candidate, and I don't need to see party volunteers pumping up me and my fellow Dems in the area. As a party we need to spread the word in areas that aren't predominately Democratic. We need to convince folks who are "on the fence" to vote Democratic, not eachother.

I'm not looking forward to all the "Obama!" PR that'll be all over my town for the next 12 months+. It'll be yet another tremendous waste of spirit, passion, & money.

Let's move on.

Aside: I picked a Russian propaganda poster ("Don't blab!") for this post as the old mother Russia had propaganda nailed. Russian propaganda posters are fascinating to me, as they illustrate the power of communication, and Russia's effectiveness, at convincing their populous of a rather bizarre set of beliefs and fears.

Monday, June 2, 2008

Sockets, HTTP, XMPP, and Leap Frog

My efforts at Gnip have caused me to re-consider a few things on the protocol front. Growing up at Netscape in the "netlib" group (really just 3 of us) gave me an interesting perspective on "internet protocols." Over the years it became apparent that HTTP was the end all. Much like twisted-pair copper wiring pervades "connectivity" into homes in the US, HTTP will be around "forever." However, I'm realizing that, also much like twisted-pair, HTTP is becoming a dinosaur in today's modern network infrastructure.

Connection throughput, latency, and reliability is amazingly good in today's tubes. We had a room full of win 16 installations with over a dozen different TCP stacks (winsocks) installed to test a zillion different connection scenarios (yes... socket->open(); "did it work?"). Today, the socket is an assumed reliable, and consistent thing. This is all to say that stateless transport protocols, like HTTP, were fundamental early on. I felt lucky that a response would come back from the same IP address a request was sent to for crying out loud. Binding any state to that connection was something only a company with the capital resources available like AOL/Prodigy/IBM/Compuserve, could pull off. AOL invested hundreds of millions of dollars building out their backend. They did deals with modem manufacturers to ensure 300 baud connection purity and to minimize drops. Madness!

Protocol Leap Frog
Even 15 years ago the stateless (HTTP/open internet) vs. stateful (FDO/SAPI/AOL) connection war was raging. AOL lost obviously, but they made billions with a connection that rarely broke between a machine in your house, and a server in a data center (over twisted-pair copper wiring); the amount of state moving over that connection literally blurred the line between where the app began, and ended (is AOL a client-side app or server-side?).

The battle is back in my mind. XMPP is a relatively defined protocol (sure HTTP is spec'd to death, but every client and server does things differently, sometimes woefully differently) that has both stateless (connection up/down) models, and stateful (connection always up) models. The latter is what can get really interesting given the new network topology and quality we live with today. An open version of AOL's transport layer basics if you will. Feels powerful.

With today's connection characteristics, I wonder if HTTP would have been the weapon of choice 15-20 years ago? I doubt it.

Many hurdles remain however. The entire network has been built out with HTTP connection assumptions. The way ports are monitored/firewalled inhibit any new connection types from taking hold. App developers think in terms of AJaX & *HttpRequest(), not publish and subscribe frameworks. XMPP is far from ubiquitous, and even if it's lucky, might not even broach HTTP's market penetration radar, but it deserves some time on the field.

Change like this doesn't come quick, and it doesn't come easy.

Wednesday, May 28, 2008

Gnip's Head is in the Clouds


Our transition to AWS/Ec2 is complete. It's been a fascinating three weeks of porting. Last night, we internally deployed Gnip to several Ec2 instances. Our current partners are still on the "old Gnip" which is hosted at a traditional managed hosting environment, but we'll be cutting everyone over to the cloud soon.

Here's where we wound up:
  • We managed to completely drop our dependency on a database for the actual application. I'm stoked about this as DBs suck, though I'm not naive enough to think we'll be able to get away with it for long; feature needs will ultimately force one into the system. We do use Amazon's SimpleDB for account management stuff, but could just as easily be using flat files stashed on S3 (remember, no persistent storage on Ec2 yet; lame!).
  • We went out on a limb and transitioned from a messaging based app, to traditional object/memory mapping for our data model. In order to cluster, we're using a network attached memory framework called TerraCotta. The basic promise of it is that you write your single instance assuming app code, run it, and TerraCotta manages the memory across an arbitrary number of nodes/instances. Basically, replication of your app's memory across as many instances as you'd like. Conceptually, super cool and simple, technically, wicked cool memory management across nodes. I'm sure we'll wrestle with memory management tuning fun, but, the lacking multi-cast support in Ec2 meant we'd have to cobble together our own point-to-point infrastructure for off-the-shelf Message Queuing services (we were using ActiveMQ, and were liking it), or use Amazon's Simple Queuing Service, which didn't taste so good after a few bites.
  • We're using RightScale to manage the various AWS instances so we wouldn't have to build our own tool to setup/tear-down instances. It's a little pricey, but we saved the few days it would have taken "doing it ourselves."
  • Performance. It's slower than running on raw hardware; any virtualized service will be. Our load tests between our hosted hardware, and Ec2 however, were close enough to go for it. Gnip has a premium placed on being able to setup and tear-down instances based on potentially highly variable load. Our traffic patterns don't match consumer based products like web apps which tend to grown organically and steadily, with a digg induced spike here and there. When we gain a partner, we have to flip the switch on handling millions of new requests at the flip of a switch.
What I want to see from Amazon:
  • A multi-cast solution across Ec2 instances.
  • A persistent storage solution for Ec2 instances/clusters.
  • Connection monitoring and optimization across their backend domains. If they see that a service is gobbling up the equivalent of a dozen physical machines, and doing intra-machine communication (memory level, socket level, whatever), please cordon off those systems so they get the efficiency gains they normally would in a more physically managed hardware scenario. This will hopefully come with time.
  • Their own failover scenario between Data Centers. Mirror Data Center A onto DC B so if A starts failing, you kick everything over to B. Super pricey, but so what; offer the service and I bet folks will pay. Otherwise, you're doing the macarena dance to manage failovers; see Oren Michel's post on how they do it over at Mashery; lots of legwork.
As an aside, the advent of "cloud/grid computing" I find funny. The marketplace is doing the best it can to find the sweetspot, and let's assume it all settles into a warm cozy place. However, during my time at AOL, I got to know process virtualization that puts all the currently deployed solutions in the market today to utter and complete shame. It was one of AOL's greatest lost opportunities (there were many). Most of the acronyms have faded from memory, but the "old guard" at AOL had pulled off true server computing miracles (SAPI it's roots were called; server API). Imagine a world where the entire stack was custom written from the ground up (OS, socket/connection layers, object models). You could write an app and deploy it across multiple data centers to ensure redundancy, without thinking about clustering; the ability was baked into the API itself. Your app could transition, in real-time, from data center to data center (that implies intra-DC machine xfers as well), without a hiccup in end-user experience. While imaginable in today's stateless web architecture, imagine doing it with stateful, persistent, socket connections. Yup... blows your mind huh?

Sorry... tripped down memory lane.

Tuesday, May 27, 2008

Hiring; and Personal Responsibility

Gnip is growing, so I'm face-to-face with the beast that is hiring. Finding smart, compelling, passionate, committed, persistent people is hard. Assuming you find one (big assumption), testing your thesis is the big gamble. Even if you do a half-dozen face to face interviews, some "try before you buy" contracting projects, and spend some time with the person socially, you won't know what you've got until a couple of months into the arrangement. It all boils down to, yup, you guessed it, gut.

Some folks recommend contracting for awhile before committing to a full-time job offer. While contracting provides some flexibility, for both parties, it's pros are precisely its cons. There's generally little skin in the game, and its that skin that can make or break a company. A contractor will generally lose little, if any, sleep over your company, and there's little that's more motivating to solve a problem than the prospect of not being sound asleep tonight at 2am. I've had good luck with small, short, contained contracts. I've had not-so-good luck with long-term "pretend the contractor is a full-time employee" contracts.

I've seen some funny/odd scenarios wherein a hiring manager blames the hiree for not meeting expectations. While we all screw up here and there, and hire the wrong person for the job, this needs to be the exception when determining whether or not you're good at hiring. Sometimes you simply miss. However, you need to assume responsibility for the situation and work to rectify it. That may mean you find a new role (with more, or less, respsibility) for the individual. Spend more time with them to help get them on the right track. Or, let them go. I've seen situations however wherein the hiring manager makes bad call after bad call after bad call. Needless to say, it's the hiring manager's manager who needs to take responsibility for what's going on, and rectify it. Ultimately, building a team is a job responsibility (albeit probably the most difficult, and important one you'll ever have), and if you do it well, you'll be rewarded; vise versa of course.

Finally, hiring jerks can be fatal. Robert Sutton wrote "The No Asshole Rule", and I highly recommend it. If it looks like a duck, quacks like a duck.... it... is... a... duck.

Monday, May 26, 2008

Cloudy skies and rain.

I'm a data junkie. Monitoring data brings awareness in so many forms. Some of it is very personal, and some of it's great for business. One of my favorite data generators has been the Vantage Pro weather station I mounted to my house five years ago or so. Yes, it's ugly (it took some serious negotiation with my wife before I could install it), but I get great real-time weather data that is highly relevant to my personal being. This post is related to my previous post about sensors (they have, and will continue to, change the world).

Anyway, the image in this post is a dashboard screen shot of my weather station. I also publish station info every 120 seconds here, at weatherunderground. If you live in downtown Boulder, I'm the most relevant station for your up-to-the-minute weather needs.

There isn't a ton of practical info I can gather from the station, most of it just satisfies personal itches for real-time data regarding my surroundings, but I do know when I don't have to bother watering the lawn (rainfall greater than 0.1" means I don't have to water).

What can I say, I like to understand my surroundings!

Thursday, May 22, 2008

Two things that will change the world.

Last night my backordered "smart" powerstrip arrived. It kills power to other outlets when the master voltage drops (e.g. the TV goes to sleep or is powered off). Simple, genius. I had the same level of excitement plugging things into it as I do when a new Apple laptop arrives. Total rush!

Sensors
The world will change when sensors are everywhere. From RFID tags, to power consumption sensing, once little bits of information are available, so many cool things can happen. Disclaimer: I'm a data junkie.

Feedback Loops & Awareness
I have friends at two energy management companies that are on fire right now. Tendrill, and GridPoint. Both firms promise to give me control, and data, around the power my home uses. Sadly, we've been running on dumb power grids for decades, with zero awareness of peak energy useage times. Viewing my home's power consumption patters at a URL is something I've wanted ever since buying my house. I feel lucky that the town I live in, Boulder, CO, will be the first in the nation to have smart power panels installed on most of the homes in the city by the end of the year. Closing loops changes how we think and act, and power consumption loops have been left open for far too long.

My Macbook Air...
... is so smart. It senses ambient lighting and adjust keyboard backlight, and screen, brightness in real-time. Clouds roll in, blocking out the sun, and my laptop brightens. When I'm sitting in a dark room and someone turns on the lights, my laptop adjusts. So cool.

lat, lng, alt
With location based sensors, latitude, longitude, and altitude will be associated with vast amounts of new data. Cameras will embed geodata into every image they snap. Phones will literally know where you are, and you can pump that info to products like brightkite with nary a button press (or even automatically).

Exciting times
While I'm totally stoked at what the future will bring with sensors, I'm bummed that I'll personally be further down the foodchain in it's creation. I'm a software guy, not a hardware guy, and this game starts with hardware and devices. Sadly, that restricts the playing field for innovation dramatically as well. Hardware notoriously evolves like a glacier. Software on the other hand iterates faster than a speeding bullet. That said, I anxiously await the arrival of the new data, so I can build software to use it and make the world a better place.