Thursday, September 27, 2007

Why the Kosmos open-source file system matters

Congratulations to Sriram, Anand, and our other friends over at Kosmix on the release of the Kosmos distributed file system.

This is important stuff. Why? Google. Google has built up a platform advantage for doing web-scale computing that puts them in a different league than everyone else. Their hundreds of thousands of servers are old news, but what's regularly ignored is the superb software engineering layer that makes it possible to run large analytic processes -- like search crawling, search relevance, and ad targeting -- seamlessly across those many boxes. GFS, Google's distributed file system, is part of what makes that possible, and this Kosmix open source release is the next big step, after the Hadoop project, for this critical technology to be available to the Rest of Us.

In the same way that widely available technology like Linux and MySQL made what we now call Web 2.0 possible, layers like KFS will enable major companies (hello, Facebook and the social graph) to scale and compete better today, and allow the next generation of startups to flourish.

I'll be writing more about this area -- a lot more -- as Zvents is at play in the same sandbox. In the meantime, Rich Skrenta is on the story, too.

Uncov on Zlango: Ruder than Onotech.

Uncov has a great piece on Zlango, which is appropriately rude. I think I was too nice when I wrote about them in 2006. (page down after click because blogspot's permalinking sux)

Sunday, September 16, 2007

CEO conversation: Startup personalities

I had a great strategy discussion about the immediate future of the Internet with a couple other CEOs and a VC partner this past Saturday. Most of what we talked about can't see the light of day, but one snippet is shareable and worth sharing:

"There are three kinds of startup employees: People who want to win, people who want to be right, and people who want to make money. You can hire all three, but especially in senior management, you have to understand what kind of personality each person has; because they'll be motivated by different things, and under stress, they'll react (and fail) in different ways."

My CEO friend went on to say that his A-round backer told him that he'd decided to fund the company because the CEO is a "want to win" kind of guy, and the VC believed that the biggest startup successes are founded by "want to win" CEOs. I am sure that other VCs would strongly prefer "want to make money" founders; and some of your core engineers (particularly the ones in scaling and operations) had better be "want to be right" guys.

I think I'm somewhere in between "want to win" and "want to be right". I'm definitely not a "want to make money" guy -- I assume that will be a considerable artifact of the other two, and in the meantime, I've got a nice garden, a nice woodshop, and a nice view -- none of which I get to appreciate as much as I should, because I spend all my time working on winning and on being right :-)

Startup life.

Tuesday, September 11, 2007

2191 days later -- September 11th, 2001

Six years ago, I lived in London, and September 11th unfolded on a beautiful fall afternoon. This is the essay I wrote and sent to dozens of friends that day. Six years later, I stand by every word.


An English friend called me on my cell phone. "I thought you should know...," he began. A hijacked plane had been crashed into the World Trade Center, he related, "...and one of the towers has collapsed." He was concerned, for me, for America, for his friends in New York, for this world as we know it. As I sat there on the bus, I was sure, in my heart, that it was a rhetorical flight of fancy. A hijacking, certainly. A bomb, a crash even, an explosion. But, a tower collapsed? These things, these most awful of nightmares, do not happen in our world. Or, at least, they had never yet happened in mine. Well, youth is fleeting.

Within two minutes of his call, I'd received the same information from two other sources -- a startled man entered the bus, Walkman radio earphone tight in one ear, and announced to us all, the passengers of Bus 88 assembled, that two planes, not one, had hit the building. He told the driver first, as if that man's status as a government employee somehow made his prior notification an important duty. This was, he said, bad for world peace. For the United States must retaliate, surely. At the next stop, a woman walked along the sidewalk, talking so loudly on her cell phone that, through the window of the bus, I had no trouble hearing, "...bombed the World Trade Center...!" before we moved off. The news was spreading.

I sat on the bus, as it slowly wended its way northward though London traffic and remembered a pale foreshadowing. In the spring of 1995, I was sitting in a corporate classroom in Plano, Texas. Our head instructor walked in, interrupting the lesson. He said, "I thought you should know...a bomb has exploded in Oklahoma City at the federal building. They think it was terrorists." Sitting there in Texas, I *knew* that it was Islamic terrorists. This was retribution, for America had just bombed someone, or done something, somewhere Over There, as we tend to do -- as the long historical record of military somethings, somewheres, that we have written as a nation over the past century testifies. I knew this, but I was wrong. It was instead an American, who felt betrayed by America, betrayed by the very system that earlier freedom fighters and founding fathers had created to govern the free, by the just.

My immediate reaction was to think of who I knew in New York, to see how closely this touched me. As it happened, two friends from London are there, both of them working in the financial district, one for Chase Manhattan, one for Salomon Smith Barney. Were they all right? I tried to call one, a quixotic impulse at best. Not surprisingly, the connection of one UK cell phone to another in New York was not deemed a high priority by the overwhelmed telecoms network.

My second reaction was to gather all news, any news, obsessively hunting scraps of news, furtively collecting small mental piles of the scraps, and playing with, puzzling over, and piecing them together to make sense of this tragedy. What of the fourth plane? What of the fighter jets sent to intercept it? The reports of car bombs at the State Department? The mysterious plane crash in Somerset County, Pennsylvania, hundreds of miles from the coast, where there is little of value save the local inhabitants, including my grandmother, fifteen cousins of varying magnitude, and about thirty thousand white-tail deer? Sometimes, impersonal tragedies work in mocking ways, giving me a vague personal connection when, in truth, I desire none.

My third reaction brought me more peace. I need no personal connection, because I will rise above the solipsism of this age, this self-centered, individual-obsessed era, and weep for the many, cry for the lost, for the spirit of society, the dreams of the nation, the ideal of the masses who yearn still to be free. Thousands are dead, and we will never know them. They were not heroes, cast in bronze. They were not perfect. They were human, they were ordinary, and they have died not by choice, but by coincidence, and died not for a cause, but for a crime.

What do we believe in? Do we, America, deserve better than this, this crematory pall of ash over Manhattan? Will we, the many, take away from this great conflagration only some sound bites, only some collected tales of heroism and shattered normalcy, only some ephemeral connection to the reconstructed media shadows of the fallen few, or will we take away something more?

In this terrible time I find clarity, and in my clarity, I find belief.

I believe in justice.

America must act with restraint before knowledge, and with righteous anger when knowledge is obtained. We cannot allow this monstrous act to go unanswered, but most certainly we cannot allow ourselves to be sullied with the same taint of slaughter that these criminals have brought upon themselves.

I believe in freedom.

America must change its practices and, perhaps, its laws, so that this does not happen again. But America cannot, it must not, sacrifice its ideals in that pursuit. Freedom lost is too high a price to pay for peace of mind renewed, and Pyrrhic victories serve no one. We must not give up what we seek.

I believe in leadership.

America has led the free world for over fifty years. We have not always been wise, but our actions have largely been of good intentions, and, despite the difficult path that good intentions tread in this uncertain world, largely of good result. We must not take this devastation as a sign that leadership is a burden no longer worth bearing, and we must redouble our efforts to lead through wisdom, and to lead through justice. Let us lead the law-abiding, let us lead the angry and dispossessed of foreign lands, let us even look within and lead the doubters of our own America. Let us lead by example, an example of justice, and of freedom.

I believe in peace.

The world has known little enough of peace, across the globe, and throughout history. Some would claim that the day will never come when all lay down their arms, and I say, that day will come, and may it come soon. I see countries and peoples consumed by anger, laying waste to the land, poisoning the minds and the lives of their children, and bringing down death and destruction upon their neighbors. Rarely do I see this where I see justice, freedom, and leadership. Peace is the fragile child of all these three, and may it be borne of them again.

I believe in humanity.

When I look at us, I see no ideal, no shining vision, of what we could become, what we should become, of what is 'best' in us. I see who we are, we grubby, smelly, apes, and I love us. May we live our silly lives in happiness, may we pursue our plebian dreams, may we bounce our grandchildren on our knees, argue with our annoying relatives, drink our nasty potables, and watch yet another rerun of classic television. In that ordinary society I see happiness, I see beauty, and I see dignity, of friends and family known and loved, of work well done, of Sunday afternoon naps well taken, and I say, let it be so.

Let us all be human, and let us all one day soon, weep only for the tragedies of the past, and no more the tragedies of today. Let there never be another September 11th, I pray.

Saturday, September 01, 2007

Google's highly dynamic indexing

For a while, my friend Prabal Dutta has been in the #1 spot on a Google search for his first name. Recently, I checked to see if he was holding his spot. Nope! Some other dude had it, instead, and Prabal had dropped to #2. I sent him a teasing email, "Dude, you lost 'Prabal!"

He wrote me back from Japan, where he is attending a conference, "I seem to have won it back," and enclosed a screenshot showing him in the first position.

I immediately checked on my end. Nope! Still some other dude in the first position.
While his location (Japan) might be at play here, it's even more likely that with him logged in, Google is blending some personalization in to the results. That itself is quite interesting, but not the most interesting discovery to come from our exchange. After we swapped screen shots, I clicked on the link to his page for whatever behavioral boost it might give to him, and went off to have a fine dinner party.

Tonight, 10 hours later, I checked again. I'll be damned. Not only is (my friend) Prabal Dutta back to #1 in my search results, but Prabal Gadh has absolutely vanished. I am not logged in for any of these searches, though I am almost certainly cookied.

The big news here is that not only is Google crawling and integrating new content incredibly rapidly (as I mentioned in my recent Scoble post) but they are re-indexing and modifying ranking incredibly rapidly, too. It's a highly dynamic index -- vastly different from the once-every-three-months crawls and refreshes of 2002, and worlds away from the crawling and indexing policies of trailing quasi-competitors like Yahoo.

It's as if they had the world's best distributed computing platform, or something.

I'll have more to say about that platform soon.