Saturday, March 20, 2010

Get A Job! At Walmart.

This post inspired by a conversation I had with my mother this morning.

My mom has been self-employed for decades via a small business she runs. Unfortunately her industry has been all but obliterated by the usual "faster/cheaper" competitive model. She refuses to acknowledge that the American consumer is driven almost exclusively by money. She longs for, as do I, altruistic motives on the consumer's part, and moral high-ground. Specifically she's saddened by the decimation of local (state and smaller) government tax revenues, and the fact that consumers choose cheaper products over her more expensive, higher-quality, products.

The conversation led to "buy local" as the solution to the problem. While I generally agree, and do buy local all the time (spending more than many of my peers do; for better and worse), I realized long ago that a perspective counter to the one I'm surrounded by is significant. Namely, if you want to really affect global change, go work for Walmart.

Activists constantly fight "the man" in the form of Walmart, when in fact what they should do is go work for Walmart, and change things from the inside. Identifying processes that save Walmart money, and implementing them, will have more reach than me diligently buying produce at my local farmer's market ever will. For some perspective, checkout some generally recognized facts about Walmart, the largest company in the world.

When Walmart says "jump" every one of its vendors says "how high." If Walmart said "go green" every one of its vendors would do so overnight. Go figure out how to get Walmart to say "go green" or "create local, high-paying, jobs."

Oh yea, and if you don't like what Walmart does to your local economy, don't shop there.

Wednesday, March 17, 2010

Amazon S3 file deletion FAIL

I'm a huge fan of Ec2 and S3 offerings from Amazon; they're the future. However, there's a severe deficiency in the S3 API; when you need to delete billions of files, the API breaks down. While there are batch operations, they don't handle enough keys to be of use when dealing with very large file counts, and enough requests are made over the long duration (days) of running your script, that expected errors wind up crapping out the whole process. Rinse-repeat doesn't work when the mean-time-between-failure is on the order of days.

Yes, I'm running the operations on an Ec2 instance. Yes, I'm using the almighty s3sync.

Amazon, you're printing money with AWS, please open up the deletion API so ppl can better manage their files. The current API's inability to a) delete non-empty buckets, and b) handle larger batch requests, feels a lot like greedy "lock-in."

Sunday, March 7, 2010

Fast & Slow

Pondering the meaning of life while bombing down Poorman the other day on my mountain bike, I realized the two things that have had the most impact on my two my favorite activities (coding and mtn. biking) serve two radically different purposes.

Disk Brakes. I got my first set (Hope manufactured) just last year. I'm a late adopter on the bike equipment front. The slowing/stopping power of these has totally changed how I ride.

Solid State Drives. I've been using these since they came to market. I'm an early adopter on the computer junk. The I/O speed of these things has totally changed how fast I can build & run tests.

It's the little things.