Modern Software Labor Challenges: The Perfect Storm

Many moons ago when I graduated from the University of Colorado with a degree in "Computer Science Applications" (the University's attempt at hybridizing Computer Science with a School of Arts theme (Political Science for me); think "double major"), two things were obvious: software engineers were making bank, and software was going to be the foundation for the new economy. Sounded good to me.

The Typhoon
It sounded good to a lot of others at the time too. By the year 2000 "newly declared CS majors" peaked at an all-time high. As with any trend, everyone was late to the party; that was also when the bubble popped. As with any trend, everyone left CS degrees in the dirt. By the fall of 2006, the pursuit of CS degrees had literally dropped by half. Unfortunately, we've been wallowing around ever since then. Industry has been trying to figure out how to economically recover. Smart humans (particularly "knowledge workers") have been trying to figure out how/where to apply their brains. Given the tremendous plummet in demand, as you'd expect, secondary level education around building software has not innovated or kept up with industry's demand for talent. Secondary, even primary (though I'd argue pre-University has always lacked), software education still has 1995 as its base from which it attempts to prepare humans for a 2012 industry.

The Hurricane
Something else coincidentally happened along the way. DHTML (e.g. JavaScript, CSS, better HTML) and LAMP'ish (HTTP server + PHP, Python, Ruby, Rails, Perl, blah blah blah) stacks thrust programming logic outside the compiler. Literally everyone with a computer and text editor could now "write code." In many instances this circumvented the need for "higher education." Back to one of my two "obvious" observations in the first paragraph, in the early 2000's you could "make bank" without even going to school.

The Perfect Storm
Now that the software industry is healthy again, it's in a lurch. There is a dearth of labor that understands, and can solve, the "hard" software challenges. The talent wars over the past couple of years are truly frightening. We're fighting over a dwindling resource.

If the macro system was working, the education system would meet this industry demand by churning out the right brain power, at roughly the right volume. Unfortunately that's not even close to happening. While CS degree pursuit is increasing, it's still not where it needs to be in terms of volume, nor in terms of relevance. Today's software construction environment is radically different than the last wave. The broad  adoption of Agile, TDD, and XP methodologies have turned everything on its head (not to mention the advancement of the underlying technologies and programming languages themselves). The education system hasn't adapted or embraced these changes.

Years ago the large software firms defined the curriculum. They did so in Waterfall environments with armies of standardized engineers in homogeneous hardware and software stack environments while making massive platform plays themselves. It is a different world today, and we need different education solutions. I'd love nothing more than our existing public education infrastructure to solve these issues. After putting my eggs in that basket for the past four years, I've have to take them out however; it's not working.

I'm looking forward to the changes that industry is going to force on the system. It will be far from perfect for awhile, but initiatives like The Academy For Software Engineering and Code Academy (two name only two out of certainly dozens) are exemplary of industry solving this challenge on its own. A handful of us are cooking up some solutions in Boulder as well. One of the cool things about this challenge is that we're at the beginning of a sea-change. Those things always yield a good time!

It goes without saying that even in tumultuous times amazing people pop out of sub-par conditions everyday. Even poor education conditions can't hold back true talent.

Partnership

I had a great conversation about partnerships with Chris Moody this evening. Chris and I have been working closely together for six months now at Gnip. As one of our investors put it, "the company is ending the year having completely exceeded any rational expectations." I personally described our year to my wife as having "f***ing killed it!" One of the contributors to our success so far has been effective partnerships within the company (certainly including ours).

Over the past few years the notion of "co-founders" and "partners" has pervaded much of the entrepreneurial/investment landscape. Techstars pushes the notion of "co-founders" hard. Generally a "technical founder" and a "non-technical founder" are considered essential to success. I have strong opinions around artificially binding two people together in a "partnership." Life-long buddies can yield priceless relationships in business (also disaster of course. it takes a special initial friendship to do business together down the road). However, putting two people in a fishbowl and seeing if they'll thrive together is a hard thing to do. Your odds of successfully partnering are much better if...

• the way each party generally wants to spend their day is different.
• if partnerA wants to wash the dishes, and partnerB wants to wash the floor, that's good.
• if partnerA wants to wash the dishes, and partnerB does too, that's bad.
• the line separating responsibilities reflects the parties' interests, and is clearly drawn.
• if partnerA wants to wash the dishes, and partnerB wants to wash the floor, and partnerA is responsible for the dishes, and partnerB is responsible for the floors; that's good.
• if partnerA wants to wash the dishes, and partnerB wants to wash the floor, and partnerA is responsible for the floor, and partnerB is responsible for the dishes, that's bad.
• each party has the skills/ability to do great things in their area of interest.
• if partnerA wants to wash the dishes, and partnerB wants to wash the floor, and they're both good at what they want to do, that's good.
• if partnerA wants to wash the dishes, and partnerB wants to wash the floor, and one, or both, of them is not good at doing what they want to do, that's bad.

All of this assumes fundamentals like the belief that "two heads are indeed better than one" for a given challenge, honesty, trust, rapport, and overall common interest in the goals.

If you're in a partnership, or considering one, check against the bullets above to make sure the partnership is setup for success.

iTunes vs. Amazon Fire

I've been intrigued by Amazon's Silk browser and associated Fire product. In the process of talking about it on Twitter and Facebook, one of the software industry's brighter individuals (and former Netscape/Mozilla colleague of mine), Chris Saari, has been asking me good, hard, questions about my thoughts/experiences/impressions/criticisms. He's working on Amazon's A2Z. He got me thinking, and this post is a result of that. He asked "what are the specific features in iTunes that you use?" I took the liberty of translating that into "why do I feel I can't leave iTunes?"

In short, the answer is that iTunes pervades my media consumption life; across devices.


With the advent of iCloud/spotify/rdio/pandora/etc things are changing of course.

I have several thousand dollars invested in iTunes library content (music, tv shows, movies), so part of it is wanting to NOT lose that investment. I'd say that's probably only a small part of it though.

A lot of it is indeed proprietary lock-in stuff. I actually do use voice control/siri to play music on a regular basis... "play artist wombats" on my iphone for example.

I use airplay regularly. I think that's iTunes only.

Toolchain stuff/workflow stuff also has me burned in. I use iMovie each year to build a family video. it "seamlessly" integrates with iphoto/itunes for content creation, and then exports to itunes for sync'ing to appletv and my ios devices. All those nice little integration points are a big deal and add up.

It's a pretty heavy toolchain issue for me. whenever I deviate, life is good for a few days/weeks, then suddenly I go to do something and I can't (or it will take a few more clicks and a new understanding of someone else's UI metaphor set, and I balk and just revert back to itunes' way).

My car stereo knows how to drive itunes on an ios device. I've learned all my car's shortcuts to control basic iTunes manipulation.

Another big one these days for me is offline support. I travel a lot and also spend a fair amount of time in the mountains. In those two cases, I don't have access to network connections and therefore, "streaming" solutions fall down. In a lot of ways I just view itunes as a local-disk sync'ing tool.

There's a big exception; Sonos. 90% of the music I listen to on my home Sonos system is streaming; pandora, rdio, spotify (in that order). pandora when I want to "listen to the radio," rdio when I want to listen to what my cool friends are listening to, and spotify when, in that one in a million random moment when I happen to recall the artist/song I want to be listening to.

My kids know how to drive iTunes (movies on an airplane is a simple scenario that we run into a lot), and that matters a lot. There is nothing more frustrating that trying to give a kid what what they want in a frustrating moment (the reality of traveling w/ children) and either you or they are fumbling with a new UI.

The downsides to iTunes are obviously real. The lockin can be painful, and I often feel like I can't leverage new cool stuff. It doesn't support for 1080i/p HD movies anymore. Movie title availability (and sometimes music) sucks (always). It doesn't support "channels" like Pandora. It doesn't support "heavy rotation" stuff like rdio.

It's hard for me to imagine a world in which I move off of iTunes.

Injury Update

A year ago I sustained the most significant injury of my life. It has been a truly unbelievable experience. Here's an update on where I am, and how I got here. There will be some detail in here that may seem odd to 99.99999% of you, however the details are breadcrumbs for others struggling with some of these symptoms, and they can matter a lot.

MRI results while trying (unsuccessfully) to identify a stroke

Month 1 through 3: What's wrong with me?
I spent the first few months trying to understand what happened. I dragged in everyone under the sun to help. Neurologists, Eastern Medicine "experts" (nutritional + acupuncturists + spiritual types), Physical Therapists, mental therapists, Specialist-this, and Specialist-that, friends, family. After endless appointments and tests, no-one had a clue, and no therapies were having a positive effect. There were a couple of theories (BPPV, some other vestibular disorder, or brain dammage), but no-one could say with any certainty. By the end of the third month, I was very discouraged. I was considering the prospect that things might not ever change. I might have impeded orientation for the rest of my life. I even backed off of sex while I was trying to sort things out; sex made things worse.

Month 4 through 6: No-one can help me.
By month four, it was clear no-one would be able to diagnose or help my situation. I'd scraped up against online forums and blogs that were pretty scary. There were a handful of people out there who's description of what they were feeling matched mine; not a single one of them could diagnose it, or have it diagnosed. There were some pretty hard stories to read (shocker; gotta love the network). By the end of month six, I'd started to resolve that this could go on forever, and that it was odd that the most common characterization of what I might have, BPPV, could go on for up to "two years." I spent a long time thinking about the "two year" stat that was floating around. How could you actually measure a medical condition going away after such a long duration. I resolved that people suffering from BPPV either killed themselves after two years of pain and suffering, or that they stopped complaining about it and just went on living in the misery indefinitely; falling off of any survey/doctor radar after about two years. Toward the end of month six, the sex cut-back was taking it's toll. I decided to take the condition regression over abstinence.

April
One of the dichotomy's in April's and my relationship is that she is as empathetic as they come, whereas I'm of a different, much more selfish, cloth. We've traditionally been polar opposites in this regard. Her unwavering belief that "things would get better" defied all logic. There was no evidence that things would get better. On the contrary there was a fair amount of evidence to suggest things in fact would not get better. She believed it though, and spent countless hours booking appointments for me, filtering doctors and experts to remove the wheat from the chaff, and most importantly talking me back from the edge. She's magical at this. It is who she is, and one of my favorite things about her.

That said, things were feeling dire. I didn't know how long I could last.

By now I'd bucketed my condition into two buckets. I either was suffering from BPPV, or brain dammage (that wouldn't show up on a scan (MRI, or CAT)) in the occipital region of my brain. I sussed out the BPPV angle, I went six months without caffeine or alcohol. I like to blame a friend of mine for turning me straight after a rough drinking binge at his house one night, but the reality is that I was cutting out chemicals that impacted the viscosity of the unique fluid that fills our inner ears. If the issue was BPPV, you can (not always, but sometimes) actually hack the fluid density/viscosity (which impacts how the calcium crystals move in the fluid) of your inner ear by controlling diet. After six months of cutting out these two substances, and no discernable difference in my condition, I jumped back into coffee and socially drinking (wine, beer, scotch).

If the issue was brain dammage, the scarier of the two in some ways, I'd bank on my brain re-wiring itself to work around the bustage in due time.


London: Inspiration in the oddest places
On a business trip to London I randomly found myself with a business acquaintance and her significant other... in a bar. This guy was five years my senior and about six months earlier had suffered a very severe mountain biking (downhill) accident in Malibu, CA. Clear brain damage, a week in a coma, smashed chest/ribs, punctured lung; the works. This guy was beaming! He'd fought through this mess tooth and nail. We empathized with each other for awhile. War stories. What therapies helped. Which didn't. Then he said something that has stuck with me. Him: "don't you hate it when someone says 'we're getting old' in the context of our injuries!?!" Me: "I sure do." Him: "Bullshit I'm getting old. This is something else, and I'm going to own it, and then I will destroy it." I can't tell you how many times I'd thought to myself "I'm just getting old." He was right. Screw getting old (as true as it was).


Months 7-9: Bottom of the Barrel.
After a couple of months of doing reasonably well, I took a family vacation that involved a plane, followed by a ferry, and ultimately an old house (built mid 1800's) with crooked walls, floors, stairs, etc. I'd been flying every couple of weeks throughout this whole ordeal for work, so I'd resolved flying wasn't an issue. This was my first boat though. When I got off (I was fine while actually on the water) I was a little affected, but not more than a previous large-boat-ride in the Chicago river with some friends. However, after 24 hours in the old-house, I was a mess. It felt like complete regression. This was a tremendous blow given that things were going ok for awhile. I chalked it up to the lacking flat surfaces or right-angles to ground myself in, inside the house. It really threw me.

Upon returning to home, I was in a bad place. A month or two went by with no "good days." Work and family were keeping me distracted so I was getting by. Then, one day...

Boulder Creek: Epiphany
The kids and I regularly bike up into the mountains along the creek to enjoy life. On this particular trip I decided to push past the disorientation and "fuck it." I was going to move and act like it didn't exist. While playing in the water with my daughter I turned my head quickly to catch sight of my son further downstream. I moved so fast that my eyes didn't have a chance to focus and track along the way. When my head stopped, I wasn't dizzy! That move previously would've resulted in some decent disorientation. What happened?!? It clicked. Right then and there it clicked! For 8 months I'd been hyper-controlling my vision and eyes. Every waking moment of every day I'd been focusing as intently as you would if you were trying to read small letters from a distance. I'd been inflicting unbelievable strain on my eyes, the associated muscles, and associated brain matter in order to control the disorientation. Somewhere along the way I'd been actually causing much of my own agony, in an attempt to control whatever was underlying. At that moment I resolved to do the inverse with my vision. I'd actively not bother focusing on things for a couple of days to see if that helped. Turns out it worked like magic! I'd found at least one culprit in what was dragging this mayhem on. Ureka!

Months 10-12:
The past few months have been great. Things are nearly back to normal. There are acute instances in which symptoms appear, but I've learned to live with them or ignore them (they're fleeting at best). I still don't know what's wrong; I'm just as lost as to whether it's some sort of BPPV (or some other vestibular disorder) or actual brain dammage. Multiple MRIs show no evidence of stroke either. I do know after 12 months however, that whatever it is, it hasn't stopped me from being who I am. I've been able to do everything on "my list" of things (tests really) in the past 12 months; _everything_. Well, there's one exception. I haven't been on a trampoline since I realized bouncing on one could aggravate things. If the result of this injury in the end is that I can't jump on trampolines, I'll be really bummed, but I can live with that one for sure.

self plot of perceived severity on scale of 1 (red); bad, 2 (orange); good, 3 (green); great

My To Do List (to validate that the orientation related things were still ok to do)
Swim: check (various pools, lakes and oceans)
Swing Olympic length pool underwater while holding breath: check (multiple times)
Run: check (ran on all sorts of surfaces for many miles. ran the Bolder Boulder)
Bike: check (I've biked a ton, both easy and hard rides)
Wrestle: check (plenty with the kids)
Handle jet-lag: check (several trips to East Coast, and a couple of trips to Europe)
Spiral Staircases while jet-lagged: check (catacombs in Paris)
Fly on airplanes: check (tons of flights)
Drive: check (driven plenty)
Jump off high objects: check (rocks, walls)
Induce Vertigo: check (stood on very narrow rock formations greater than 40' off the ground with sheer drops on either side). I still need to do glass floor at Seattle Space Needle.
Use computers and small devices/screens: check (everyday)
Walk through house in the dark: check
Sit down with eyes closed: check
Ride bike in a straight line indefinitely while head looking off to side (peripheral vision thing): check. this one took awhile, but I'm baaaaaack!
Drive car in straight line indefinitely while head looking off to side (peripheral vision thing): check.
Play Video games: check
Have intense sex: check
Massage: check
Cross Country Ski: check
Downhill Ski: check
Skateboard: check
Balance across logs: check
Everything else that I can't remember at the moment: check


My Extended List: Stuff I've done in my life, but I'm not going to bother repeating.
Trampolining
Fist fight

Finally: Today
Things are not 100% better, but they're 99% better. I've had a few consecutive months of feeling really solid, and I've noticed a true shift in that time period. I can confidently say I'm in a good place now. Even if I regress at some point in the future, I know I can have multiple months of goodness after several months of badness, so if I lapse back into it, I'll have light at the end of my tunnel.


I can empathize with others now to a degree I've never been able to. This is really powerful and I'm really happy about it.


The human mind/spirit prevails. A friend of mine who'd suffered a life-threatening accident on snow a couple of years ago told me "the mind/body will heal itself. calm down." I doubted him gravely, but he was right. The mind/body will find a way.


If you think you're suffering from BPPV (or some sort of vestibular disorder onset), and believe that is what I have/had, then I can tell you things are better now, and none of the reset maneuvers worked; none of them. There was a definite progression from bad to good; granted it took a year to get here. Hang in there.

TechStars Cloud (Skynet)

Building server-side applications/systems has changed forever thanks to virtualized infrastructure. I'll never forget asking a colleague at Netscape when I'd just started my professional career "why are we thinking about hardware at all?!? Shouldn't software just abstract all of that?" He chuckled at my naiveté. Lo-and-behold ten years later I was able to think about hardware like I did software; disposable resources that I used a command prompt to setup and tear down, script, load, configure, and tightly integrate my software with.

Beyond exciting.

Over the past four years at Gnip, we've used nothing but virtualized hardware for a highly-scalable piece of infrastructure sustaining countless, sustained, high-bandwidth connections to customers all over the planet. The only hardware we own are the laptops we use to write the code. Wow... every time I say that I get chills down my spine; so cool.

I've been a TechStars Boulder mentor for a few years now, and I'm stoked that TechStars spun-up a focused "Cloud" program. I'm excited to be engaged in a more infrastructure focused thread of TechStars (something I've criticized the program for in the past). TechStars is an amazing thing, but I'd always wanted a more explicit systems/infrastructure twist (my bag, frankly, because I think these kinds of plays are more impactful).

If you've considered using TechStars in the past, but thought it might be too consumer focused for your idea, you should consider TechStars Cloud as a mechanism to get your idea squarely into the execution phase. Even if you haven't considered TS in the past, and you're looking for a place to hone your idea into something with traction (or at least die trying), check it out.

You can apply for admission here.

I hope to meet you soon, and to have an opportunity to help you build something cool using our new fangled "infinite resource" that is now Skynet (errr... the Cloud).

Lijit Acquisition

Wow. What an illustration of how sticking with a company/team through thick and thin can yield big dividends. With a little time, amazing things can happen. I'm overjoyed with what the Lijit team pulled off. After four years (at least) of various strategies, business models, and approaches, they made it rain. I'm just blown away.

• It can take years for something to come to fruition. If at first you don't succeed, try try again. We've learned this lesson over at Gnip (my co.) as well. It took a couple of years, but the pistons caught and things are going great. I can only hope for an exit on the order, or greater, than what Lijit just pulled off. Patience.
• If you're an employee at a startup, buy every share of stock you have access to, as soon as they become available, no matter how bad things look/feel. If you quit, or are let go, buy everything you have access to. You never know what can happen downstream. To those who left Lijit and didn't acquire your stock, I'm sorry, but you broke the golden rule. You simply can't beat the odds and economies of scale that employee private stock option exercises offer. Sure odds are stacked heavily against a startup, but the potential upside dwarfs anything you can do in open public markets.
• Todd & Manny stuck with co-located hardware for their system. I always thought this was odd (still do). Gnip is 100% in the cloud. Lijit does go to show buying/leasing/managing your own iron isn't dead (yet).
• Seth Levine at Foundry Group is going on to be a board member of the acquiring/combined company. He's obviously becoming a powerhouse in the adtech space at large. Nice having him in our backyard (even though he trounced me on a gorgeous fall ride the other day).
• This was a huge win for the Boulder, CO software community. Undoubtedly big economics involved here, in Boulder.
• Hardcore systems engineering == intense value and amazing product. No web-app here folks, just nose-to-the-grindstone engineering (Manny... you and your team are "the [wo]men").
• AdTech remains a massive marketplace and economic driver on the internet (whether you like it or not). The deal sizes that continue to happen 15 years after ad networks really kicked in, on 15 yr old frameworks and models, tell me that AdTech is evermore poised to evolve to take into account, or even be overrun by, real-time behavioral frameworks and models that social content enables (go Gnip!) in the AdTech arena. 15 year old models built on cookies and clicks still driving the economy... you've got to be kidding me. Time for change.

I'm proud of the entire Lijit team.

Sate

I was talking with a friend awhile ago about how I behave under certain circumstances. He was trying to read me and I clarified things by telling him that when I'm notably happy, things are likely very stressful and heading in the wrong direction. When I'm cranky, stressed and a mess, things are likely going unbelievably well. Pretty twisted, I know. To illustrate this another way, when I'm stressed I don't eat and I lose weight. When I'm happy... I'm fat and happy.

I've been lucky enough to have achieved several major personal goals over the past year. Furthermore, I've had some significant (good & bad) personal events hit me over the same timespan. Simply put, a lot has changed for me over the past year, so I've been able to reflect on the personal behavioral hypothesis I posited awhile ago with my friend.

Unfortunately, or fortunately depending on how you look at it, I'm realizing there's some void I'm trying to fill. I can't measure its volume, so I can't predict what will fill it. The rate at which I can fill it changes frequently, which therefore complicates the attempts to fill it. This leads to a constant lack of satisfaction. I'm sure this is starting to read like a resume describing how "my biggest weakness is that I work too hard," but this goes way beyond work. It permeates my being.

As my wife can attest, living in the "now" is very hard for me; I'm always somewhere in the future (the past is dead to me the moment the time passes). We walked in the door as a family the other day after a long exhausting, though fun, day, and before I realized it, she was frustrated and blurted out "we haven't been in the house 30 seconds and you're already doing 10 new things." When I wake up in the morning there is no "warm up" period that I hear people talk about, I am just "on."

I used to think this would change with certain milestones being met. Well, I'm "older" now, and while that's certainly had it's impact, it hasn't calmed me down. It hasn't slowed me down. It hasn't shown me any light to make me content.

I don't want to be content. Ever! My highs will be very high, but the trade is that my lows can be very low. Meandering through life with a constant level smile feels boring. Someone at an Ignite Boulder event last year made a remark that stuck with me. Something like "if you want to be content, go take a warm bath!" I do take baths from time to time (sorry Cosmo Kramer), but they're recharge moments, not relaxing experiences.

Every few years I'll have one of these completely blissful moments. They last only a few minutes at the most, but they're truly pure elation. Nothing is wrong in the world, and life is perfect. I haven't had one of those in awhile, but I'm looking forward to the next one. They're so cool.

More please.