Tuesday, April 24, 2007

Fish bowls, attention streams, activity, and monitoring.

I find twitter funny on so many levels. One that jumps out at me is the attention deficit disorder that twitter perpetuates. As if we're not sliced and diced enough these days with media coming at us from so many angles on so many different topics, now we can post, and read, bite-sized chunks of whatever we feel like talking about. The "bite sized" nature of twitter separates it from traditional "blogging."

The "reality TV" of blogging has hit a new level with twitter and "nano-blogging." As the volume of twitterers increases, the relative quality of content will decrease. Just like blogs degraded the quality of editorial content at large, nano-blogging will bring it down yet another level. With that said, opening up writing to a greater number of users will obviously un-earth some true writing genius.

Anyway, what I really wanted to write about was the difference between explicit data collection, and implicit. While explicit data collection services like twitter, Flickr, and blogs capture a mind boggling amount of interesting information, they're all still rooted in a fundamentally limited, event driven form of data collection. While limited, you can still do some interesting analysis of said data; a friend of mine did some fun and interesting analysis on twitter posts.

Things really change when our lives are truly, implicitly and passively, monitored.

Imagine a necklace that captures 24 hours of audio and video. Now imagine wearing that necklace 24 hours a day, and it sync'ing each day. Upon each sync, the collected data is processed, and all of your activity (who you saw, who saw you, who you talked with, who talked with you, what you did, where you went, what you saw, etc...) is broken down, logged, and analyzed. Obviously the technology needed for this level of broad data processing doesn't exist, but it's an interesting notion.

Perhaps we won't ever get comfortable enough with sensing and monitoring to go this far. Perhaps we'll only ever be comfortable with pro-active, explicit, data sharing. I believe history is showing us otherwise though. We're seeing a conscious degradation of privacy. "Reality TV" is the tip of the ice-berg. Many people today expose much of their lives to the online world. This exposure, to date, has largely been in the form of pro-active, explicit, data sharing. As a result, it isa very limited view of our lives and activities. It is a highly editorialized view of things. As we grow tired and bored of this censored/editorialized stream of information, the next step is implicit, passive, yet controllable, data sharing.

Taking large chunks of our activity and passively collecting information about it takes today's notion of information sharing and removes the self censorship and editorialization. This gives us a true view into our activity, how we think, what we do, and so on.

Me.dium, my employer, gives users the ability to implicitly share their click-stream/attention-stream with other users (anonymously, or otherwise). As a result, your friends (or everyone) can see what you're doing as you surf the web. While entertaining, the true value is in the processing of all the data Me.dium collects. Me.dium aggregates everyone's attention-streams and correlates URLs amongst the population. The result is a "match set" of URLs that represent the sites people visit when trying to accomplish a particular task. For example, if you're looking for information about military aircraft, Me.dium can recommend the URLs that others most regularly visited, relative to the URLs you're visiting and the searches you're conducting. You can see what others did while exploring a particular topic. It's a great example of the value of processing implicitly collected data.

We also correlate users, and chat conversations that are algorithmically relevant to you as a user. We're just getting the service off the ground, and after you use it for awhile, you can't imagine browsing without it. As we tune our system to enhance the correlations, things will only get better. Me.dium is generally focused on "online" activities, but back to my "necklace" concept (its not mine, by the way, someone else talked about it years ago) the analysis and correlation that can be done with more and more data can truly start to change things.

While all the current, pro-active, explicit data collection tools and services are fun and entertaining, I'm looking forward to more productive uses of data as passive, implicit collection becomes more mainstream. We all have to work to ensure we don't wind up in some sort of "skynet"/1984 conundrum however. Any kind of sensing service/framework must build in fine grained user control.

No comments: