Airlines vs. Fuel Costs = Dirigibles?

I just read a fascinating article in the New York Times about Boeing, Airbus, and the future of commercial aviation. Boeing, as airplane geeks will know, has bet heavily on the 787 'Dreamliner', a highly fuel-efficient midsize plane designed to serve point-to-point international routes -- San Francisco to Shanghai, Athens to Boston. Airbus, on the other hand, is building the massive A380, a plane designed to carry 500+ passengers on core 'hub and spoke' routes such as New York to Tokyo, with smaller planes serving as 'feeders' to these routes.

This multi-billion-dollar faceoff between two different philosophies and infrastructures is reminiscent of so many other distributed-vs.-centralized battles. FedEx created a business by going with centralized hubs. "Mesh" networking is supposed to be more efficient and resilient by getting rid of hubs. However, it's clear in this case that one simple calculus will decide the fate of Boeing vs. Airbus:

Fuel cost vs. customer happiness.

As recently as 12 months ago, I would argue with friends about 'peak oil' and what an economy looks like as it slides down the back side of a bell curve of production; and I would get signficant pushback that we were anywhere close to such a circumstance. The more libertarian-minded were fond of citing the famous Simon-Ehrlich bet on commodities as a supporting point.

And here we are. Massively rising demand for oil from China and India; potential disruption of significant sources of production ranging from Venezuela and Bolivia, Nigeria and Sudan, Iraq and Iran, to the Khazak basin in the former Soviet Union. No massive new sources of oil discovered, and few likely to come on line soon. And wild cards such as the Canadian tar sands are predicated on... high oil prices to make them relatively cost-efficient to produce.



Which got me to thinking: If the tradeoff for airlines is fuel efficiency vs. customer satisfaction, why not dirigibles or blimps? Right now it takes about 10 hours to fly the 6000 miles from SF to London, at about 600 miles per hour. An appropriately designed dirigible could do it in 24 hours at 250 miles per hour, at a vastly (90%?) reduced fuel cost -- since a dirigible would benefit both from the cubic reduction in power-required vs. speed flown, and the absence of the need to expend power to keep the aircraft up in the air, which accounts for a large percentage of airplane fuel cost. Imagine that, instead of spending 10 hours on a cramped, noisy, EXPENSIVE airplane, you spent a full day and a full night on a quiet, spacious, dirigible? Broadband internet access would be essential -- not only could you make crystal-clear phone calls, but you could transfer any volume of data. You'd get nice meals from a large kitchen. You could walk around and exercise. You could sleep in a real bed. And in a world of $70 - $140 a barrel oil costs, all of this might be CHEAPER to provide than a miserable 10-hour flight.

The key, of course, is making the experience as business-like for business travelers, and as vacation-like for vacation travelers, as possible. Without doing some considerable aeronautical math, I can't estimate at what fuel price point dirigibles would start to make sene. But it's certainly fascinating to consider.

Portfolio Management, the Yale Way

Jason Zien over at Spotsearch recently wrote a post about Yale's asset allocation strategy, which has garnered amazing results - read Jason's post for the details. In about 2001, I read a book by David F. Swensen, the Chief Investing Officer of Yale University, called Pioneering Portfolio Management. It is one of the better investment books that I've ever read, and I highly recommend it. Swensen apparently has a new (2005) book out, "Unconventional Success," which is also well recommended on Amazon.

Ponytail Sweepstakes: Ted Waitt, Gateway

I've gotten a few emails based on yesterday's post about Jonathan Schwartz. It has been brought to my attention that Gateway founder and ex-CEO Ted Waitt was ponytailed for much (or all) of his tenure at Gateway:



There's a great quote on Waitt in a recent Fortune article, that foreshadows Sun's challenge:

"But while Waitt, the ponytailed visionary, was looking toward the future, boring old Michael Dell was obsessing over efficiency, thinking about how quickly a PC could be assembled rather than how fast it ran. As it turned out, execution was everything."

The ability of the Internet to quickly and easily answer idle historical questions such as "what was the market cap of Gateway during Waitt's tenure" is still limited, but a quick scan of the stock price suggests that it may have been as high as $28 billion circa 2000. This would significantly eclipse Sun's current $17B number under Schwartz.

The race is on, Jonathan!

"What's a Blog?" - Six Apart at Maker Faire

This was my absolute favorite sign at Maker Faire. I promised Elizabeth (the girl in this picture) that I'd blog about it, so here it is.



There was lots of other cool stuff at Maker Faire, including awesome mathematical castings from Bathsheba, which embarassingly I failed to photograph. Oh, and I-Wei Huangs's steam robots!

Scott out, Ponytail in, shares up!

Sun send Scott McNealy off into the sunset today, after a run of 22 years at the helm. I can't think of too many other CEOs who have run a company that long -- though I bet Michael Dell will get close.



In related news, Jonathan Schwartz is CEO -- does anyone out there know of a CEO of a company larger than Sun run by a ponytail-sporting guy? He may have the market-cap record for that particular feature.

I know a lot of guys rooting for him to hold the "return on shareholder equity" record for ponytails, too.

Search Trends: The Search For User Intent

There are several interesting trends afoot in the world of search. As I have previously written, concepts as disparate as editorial news and personalization are simply different ways to get to a better search answer. I'm now ready to extend this thesis a bit further, and claim that the next big leap in search is going to be driven by a superior ability to acquire and understand user intent.

In any query:response system, the quality of the output is fundamentally limited by the information contained in the query. In a mathematical sense, it's impossible to have more significant digits in your answer than you had in your input data. No matter how well you tune your response algorithm, not only will "Garbage In, Garbage Out" always hold true, but the more general case of "lack of precision in, lack of precision out" will limit how good your results can be.

On a general search engine (Google, Yahoo, etc.) an inbound user query can, quite literally, be about anything. Presented with a blank box that can encompass anything, the user then types in an average of 2.4 words, and the search engine must give back an ordered list of 10 relevant results (having worked very hard to remove spam, porn, etc.) that may approximate what the user is looking for. Suppose the user types in "Peru". Is the user a 4th grader writing a report for school? A college student planning a backpacking trip? A consultant searching for economic data related to a project? A Peruvian looking for things related to their home country in their local area? All these, and many more, are possible.

It's a very tough problem, and a key reason that search quality has ground to a halt, or at least stopped improving dramatically, is that in this completely general context, it's very hard to understand more about the user's intent than they grudgingly give you in their 2.4 words.

So how can search engines solve this problem of paucity of intent?

Answer #1 is to verticalize. Zvents, my startup, is an example of this trend -- as are Simply Hired, Kosmix, and many others. When someone types Peru into Zvents, we know -- simply because we're a search engine for local events, and nothing else -- that the user is looking for something to do in their local area having to do with Peru. In order for Google to get that kind of intent, the user would have had to type, "something to do in my local area having to do with Peru" or a similarly complex query. Zvents gets a great deal of intent information simply by being a specialist. When someone shows up in our search interface -- either directly, or via a media partner -- we can be confident they're looking for stuff to do. Simply Hired, similarly, would know that the user was looking for jobs either in Peru, or related to it; and Kosmix, under its healthcare filter, could infer that the user cared about health issues related to Peru.

Answer #2 is to personalize. Google is the most interesting case emerging at the moment. Go to the Google homepage and in the upper right you'll find a link to the new personalized home. It's a HUGE departure for Google -- who has built a multi-billion dollar business on a purely "visitor" experience -- to be moving to a registered user experience. It's likely that Google is doing this is to get a much clearer sense of search intent, based on personal past search history. At least one very senior technical person involved in the Google personalized home page has a background in mining extremely large log files to extract business intelligence -- exactly the kind of resume you'd want if you were attempting to derive intent from search history. I've also had some discussions with folks at Yahoo (who are also very happy to let you log in while searching) about the extremely sophisticated state information that Yahoo maintains as you move through the Yahoo portal; all aimed at improving the targeting both of ads and products.

Answer #3 is to get more intent directly from users. Many of you might recall the original concept of Ask Jeeves, which was to "answer your questions in plain English". In a recent conversation with some folks at Ask, they mentioned that one of the reasons they'd retired Jeeves was that delivering on this "brand promise" was nearly impossible - but that because of that historical promise, the length of their typical search query was vastly greater than comparables at the other big search engines. That begs the question -- what if you could successfully parse natural language English like "something to do in my local area having to do with Peru"? It's an incredibly hard problem, but the ability to accurately extract enough user intent to deliver a truly better search result would yield enormous market benefits. I know of at least one current serious search startup that is taking exactly this approach -- building a general-purpose search engine that can better parse complex intent, and accurately respond to it. If they succeed in their quest, and establish a "brand promise" similar to the original Jeeves, they'll have a significant advantage in actually giving users what they want.

There are other answers, but they look less and less like search. Aggregate Knowledge, for instance, is a matching engine for supplementary navigation based on heterogenous item:item matching in a "people who viewed this, also viewed that..." sense. At scale, AgKnow can derive intent from the cumulative behavior of multiple users who were exposed to similar information -- and as their scale increases further, they could slice even more finely to begin to distinguish path dependencies as well. I think we'll see an explosion of search alternatives in addition to advances in all three categories mentioned above, as dozens of companies large and small strive to crack the problem of too much information, not enough user intent.

What does Web 2.0 mean to me?

I was asked this question for a podcast at the recent Under the Radar conference where Zvents presented. You can test my memory by hunting up the actual podcast, but I am pretty sure this is the same answer I gave:

It means two things -- one technological, and one social.

Socially, it's the return of hope and enthusiasm to Silicon Valley. People may mock "bubble 2.0" and naysayers can always point to some particular point of excess, but the reason we entrepreneurs are here is to change the world, and a lot of dreams lay dormant from 2001 to 2005. Those dreams are being knit into reality today, and many of us can see that we'll be able to make the world a very different, and hopefully better place, soon.

Technologically, it's about data as a platform (just like Tim O'Reilly said) and more importantly, it's about fast and simple integration of that data. Mashups may occasionally be dorky, but they demonstrate a level of interoperability that is simply astonishing when compared to 1999. We've all seen network effects accelerate change and growth to warp speed -- what we're seeing now is kind of Metcalfe's Law of web functionality, where the value of every interoperable web service is the square of the number of other services to which it can be connected.

Hmmm. I know I didn't say that one in the podcast. It may actually be worth its own post.
But, back to work. Big customer meeting tomorrow...

...Web 2.0 means the return of customers. Yee Hah!

Aggregate Knowlege launches at ETech

I may be busting the press embargo by 90 minutes or so, but I'm really excited about this one. e, Tomorrow at ETech, Paul Martino and Chris Law are taking the wraps off of Aggregate Knowledge, their fascinating new play on a recommendations web service.

Paul and Chris were formerly founders of Tribe Networks, and Paul (several lives ago) was a graph theory PhD student at Princeton. They've cracked the problem of providing user-specific recommendations across multiple data types in real time, thanks to some seriously fancy math under the hood. We've been using AgKnow at Zvents to power some of the "related events" content on our site (Disclosure: I am also an investor), and look forward to more customers joining their service so we can begin to benefit from further network-effect driven recommended content.

Most importantly in this mash-up world, it takes from minutes to a few hours to fully implement their service on your site. EAI is soooo dead.

I expect them to do very well in both the content recommendation and non-search navigation business. Rock the house, guys!

P.S. I previously mentioned these guys in November.

Technorati Tag: ETech

The limitations of dynamic content creation

To confirm the spelling on the Latin phrase "casus belli," I typed it in to Google. eBay provided some serious unintended humor.

Ether *is* Keen 2.0: Old Wine, New Bottles

Over at TechCrunch, Mike just posted on Ether, a new company driven out of the pay-per-call innovators Ingenio. My immediate thought was, "is Mayfield funding this company?" because Mayfield is (in)famously trolling through old bubble business plans for ideas, and this sounds an awful lot like Keen.com.

Keen was funded by Benchmark around 1999. I first saw gigantic billboards on the Underground advertising Keen in 2000 when I was living in London, and I remember thinking, "Damn, that's a great idea," as I stared at them from the platform across the tracks. It may still be a great idea, but Keen was relegated to a fairly painful fate, morphing into a psychic hotline and not much else. And certainly nothing that could make back the bubble-zillions that got poured into it.

In poking around to check out the carcass of Keen, I discovered a most curious fact:

Ingenio (Ether) *is* Keen! Changed their name sometime in the last seven years. Look right down at the bottom: "Keen is a trademark of Ingenio, Inc."

What was old is new again...!

Feed reader creators: There is so much left to do

I have a confession to make. Here I am smack in the middle of Web 2.0, a founder of a site that pumps out more RSS than you can shake a stick at, and I don't like feed readers. In fact, until about two weeks ago, I had never really used one, save a single desultory experiment back in 2005.

(Gasp!)

Yes, I have been quite embarassed about this affliction. I must admit, I have hidden it from my peers.

"Hi, I'm Ethan, and I type in URLs by hand."

I felt so bad that I finally caved in a few weeks ago and submitted to Bloglines. At least I could get an OPML file out of it, I thought... since Dave Winer assures me that will be useful soon, and Kevin Burton swears it will help me find all sorts of cool stuff on the web. I heard rumors at the last TechCrunch BBQ that it even attracts girls, but I've already found one of them.

But I couldn't stand it. Sure, it's all my reading in one place, and the atom feeds look sort of like their source blogs, but it's all kind of... attenuated. Like sucking Coca-Cola through a long straw, so the fizz is gone. All the formatting is replaced with grey-on-white arial. The fascinating sidebar tidbits and blogrolls and, yes, ads all disappear. And the urgent boldface of posts as yet unread, marching down the left sidebar like dutiful ants...

I thought I was alone.

And then... I found another! There in the endless grey arial sat a wizened little scrap of text, from AVC, blog home of none other than Fred Wilson, who, if he isn't the patron saint of Web 2.0, is, at a minimum, merry Mercury in its Pantheon. And what sayeth Fred?

"I Prefer Browsing To Reading Feeds"

Hallelujah!

Me too, Fred! In fact, this may be the most important post you've ever written. It's supportive of an inclusive and exploratory, rather than exclusive and reductionist, view of the web. While it calls out feed readers explicitly, it also criticizes by implication all the other exclusionary, reductivist drivers in this fascinating new media landscape -- our tendency to read the same old blogs; our algorithmic focus on the same old hubs and authorities; our analytical adherence to the same old scales and dimensions, and most particularly, our (and MY) fear that somehow, by being human, I was wrong.

A quote from Stephen Pinker's book (yes, an actual paper book!) The Blank Slate:

"The belief that human tastes are reversible cultural preferences has led social planners to write off people's enjoyment of ornament, natural light, and human scale, and forced millions of people to live in drab cement boxes."

I love blogs. I love the conversation, and the chase, and the delight of discovery. I love the humanity of user-created content, and I don't want to aggregate and attenuate that away in some misguided attempt at rational efficiency.

Is BuzzMachine the same without seeing that giant press picture at the top of the site before reading it every day? It reminds you, the reader, of the massive machine that is MSM. Like the three tones of NBC News at dinnertime; like Pavlov's dog; I salivate before dining daily with Jeff Jarvis.

Give me that ornament, and human scale, that connection with the writer and their carefully tweaked site, and at the same time make my life easier - and you'll have a dedicated customer. But until then...

...this new/old human-based web needs to remember us humans.

Feed reader creators: There is so much left to do.

Great new maps from Ask.com

Ask announced that they're going to fight for search, and launched some cool new maps functionality. Much of what you see here is a quality re-implementation of Google Maps, but there are some cool new features - for instance, Tyler pointed out that after you enter a route, you can click a "play" button and Ask will walk you through the steps onscreen.

Yawn, you say. Google did most of the useful bits a year ago, and Yahoo more or less matched this last fall.

True.

What I find fascinating is that the leading edge of search is all happening at startups, and that the offerings of the big boys are increasingly undifferentiated.

Why isn't Google building Plazes on top of Maps? Why isn't Google building Zillow on top of Maps? Heck, why isn't Google building Zvents on top of Maps?

It's a very interesting time to be a search startup. It feels like the calm before the storm -- with an absolute explosion of functionality coming very soon.

Classifieds are Transactional, Groups are Relational

Mike recently wrote a second review of the forthcoming Microsoft Expo (aka Fremont) on TechCrunch.

He said, "Expo is centered on the idea that people will trust others within a group, and so is allowing classifieds networks within groups."

I left a comment on the article, which I'll expand upon here.

The original concept of Tribe Networks was that classified advertising would work better within the trust circles of existing groups, or "Tribes". This focus - which wasn't something they publicly discussed - is why their investors included the Washington Post and Knight Ridder. While Tribe is still going, it's fair to say that this concept didn't work for them. Maybe Microsoft can do better, but I am not convinced.

There are two simple reasons why.

First, classifieds are inherently transactional, and long term relationships of trust simply aren't necessary. Each of us has a sphere of activities, interests, friends, and so forth -- and when a buyer and a seller get together to transact a piece of merchandise, none of that is relevant. I have both bought and sold cars via classified ads, and what mattered to me wasn't that the other guy was nice, or trustworthy, or that we had anything larger in common; what mattered was that his cash or his car was as represented, and I came to the exchange with one, and walked away with the other.

There are very rare exceptions to this dictum - just yesterday, my friend Niranjan regaled me with the story of his purchasing an absolutely mint 1964 Volvo P1800 as driven by "The Saint." He still visits the previous (and original) owners of the car once a year to show it to them, and they sold it to him at a below-market price on the basis of his winning personality and love for the car. But exceptions like these prove the rule. Ask yourself - how many goods have I purchased or sold through classified ads; how much did I really care to know about those sellers/buyers, and how much would I be willing to change the mechanics of the transaction - e.g. paying higher or taking a lower price, waiting longer for an item to turn up within my limited network, etc., just to take advantage of that fuzzy feel-good surround?

The second problem with this concept is that people are wildly inconsistent. The certified CPA is cheating on his wife. The marriage counselor that they are seeing is an asshole in business dealings. The ultra-reliable mechanic who fixes their car with fantastic attention to detail can't keep his personal finances in order. Within different contexts, people's behavior can be so different as to be contradictory, and therefore, all the social things you think you know about someone from a groups-like exercise such as this, may disinform as much as they inform.

I'll be very curious to watch this develop, but I don't see any magical secret sauce yet.

Google Maps ego-surf: Everything is higher resolution at the Googleplex!!

Google Maps updated their satellite imagery, offering higher resolution in many places. Of course I had to check out my house - an increasingly less obscure form of ego-surfing. Since I live only a mile or two from Google, I scrolled over there to check out the uniquely weird/cool architecture of the Googleplex. Surprise! The world suddenly gets sharper, more crisp, and clearly resolved, right where the landfill ends and Google begins, even if the roads don't quite match up.

It's kind of cute -- sort of like those airbrushed graduation pictures we all had at age 18. If I were organizing the world's information, I'd probably do the same thing.

Q: Guys, when are you going to stop being coy and make 'Googleplex' resolve in the same way that 'Buckingham Palace' does?

User-Created Content: Market Goods, Cavemen, Schadenfreude

Ethan Prater at Jigsaw dropped me a note after my last post on user-created content and encouraged me to push my thinking a bit further. Specifically, he asked how I'd think of the content that users place into Jigsaw's contacts marketplace. As I understand Jigsaw's business, there are two different categories of information and value that they're getting from their users:

1) Contact information

2) Metadata & feedback about contact information

The first case is a classic "selfish self-entry" good. Either I am entering my contacts into Jigsaw in order to make cash money ('Buy, Sell, and Trade Business Contacts" is their tagline) or I am entering my contacts into Jigsaw in order to better manage them, and to derive some business development value ("trade your contacts") from them.

Metadata about contact information is a bit trickier, and forgive me, Ethan P., if I divert from your actual business model a bit to expound upon it. One of the other taglines on the Jigsaw front page is, "Accurate Data: Because Members Build and Clean the Database." That 'cleaning' bit is what I would refer to as metadata -- meaning, feedback that this is a bogus contact, an outdated contact, or a useless contact - I imagine someone writing in, "this guy may be the VP of Spending Money on Large Humming Boxes, but he takes 18 months to make a decision."

This is exactly the same kind of user-created content that steers the ship at Craigslist -- users can flag any post as miscategorized, spam, etc. -- and that eBay relies upon, with their famous buyer- and seller-rating system. It's the same kind of content that Amazon Reviews runs upon - the guy on the street giving good directions, which he has no particular reason to give.

It's difficult to identify why, exactly, users do this - I'd say that it falls at the delightful juncture of psyche where human beings depart from the rational choice models and start acting like, well, humans. Depending on what you belive, it might be called schadenfreude content, or perhaps caveman content.

The person who discovers that a Jigsaw contact is bogus (or, on Craigslist, that the posting is inappropriate in some way) is very unlikely to be fooled into wasting their own time on that content / contact again -- they've discovered that it's bad, and they'll move on. From a selfish perspective, throwing even another mouse click after that bad content in the form of punching a simple bogosity button only increases their sunk cost. So why do it? Is it because we instinctively recognize that on the other side of the oft-referenced "tragedy of the commons" lies a triumph of the commons, where lots of feedback snippets from lots of people can actually create broadly beneficial value? Or are we simply mad enough at having been misled -- whether it's an abstract anger at a bum steer in a piece of anonymous information, or a very directed annoyance at some scammer who's trying to take our money -- that we strike back through negative review?

I don't know.

In any case, we act; we vote; we warn the other grubby apes that here lay sour berries, here lurk leopards, and we suggest that the tribe move on.

Here is what I would posit: That the urge toward fairness and truth is so strong that it overpowers economic gain - and economic gain can actually get in the way of this impulse, by making people question their own instincts. So without knowing exactly how Jigsaw is managing their user-driven data-cleaning feedback system, I'd strongly recommend that they pay their users for this act in status and righteousness, not money. Compensating them for their contacts is an entirely separate transaction; monitoring the data quality is a moral crusade that I think we all are deeply primed to engage in, regardless of what the dystopian economists tell us about our amoral motivations.

Next post: data exhaust, perhaps the only category of user-created content that *might* qualify for the soulless rubric 'user generated.'

User-Created Content: Selfish, Social, Selfless

I continue to be fascinated by everyone's determination to wrap user-created content into their new web services, despite their flamboyant display of ignorance as to *why* users actually create content. Here are three distinct models that can help you think about why users create:

1) Selfish UCC. Delicious bookmarks and Flickr photos. I primarily bookmark and take pictures for myself -- because I want to remember, or because I like something. I use these web-based apps instead of client-side equivalents because I personally get more value out of them. While the social benefit is apparent, I view that social benefit strictly through a selfish lens -- instead of emailing pictures to my Mom, I can just point her at Flickr. I encourage other people to use these sites for selfish reasons as well -- it makes it easier for me to keep track of their UCC. Another great example of selfish UCC is every auction on eBay. Those sellers are not putting those auctions up there to help anyone but themselves - any general benefit is strictly a secondary effect. It's not necessary for any individual item of UCC to relate to any other item - a picture or a bookmark has its own stand-alone value.

2) Social UCC. Blogs, Slashdot comments, and Yahoo Groups are great examples. People create content in order to express a point of view within a larger conversational context. There is usually an implied audience and an implied reference in most of this type of content. Why do people create conversations? It's somewhere between selfish and selfless - call it social. We want to take part in something larger than just ourselves, but we expect to benefit as well. In this case, personal and general benefit are deeply intertwined, and each item of UCC relates to other items of UCC.

3) Selfless UCC. This is the one that leads most everyone astray. The single great example of selfless UCC is Wikipedia. If I am an expert in subject X, I already know all about it. I do not benefit personally from writing down that information and sharing it with the world -- I am primarily doing it so that you, the reader, can benefit; or out of some sense of "greater good" and contribution to something shared and larger than just myself. It's a very noble aspiration that drives this kind of UCC, and it's not particuarly surprising that this makes it the most rare. Implied for this sort of content is a weak sense of authorship -- if I am personally attached to the content, it begins to look much more like a conversation.

The obvious non-UCC example of this strong authorship point is newspaper reporters vs. columnists - a column is a conversation, whereas a byline is subsumed (largely) by the headline and content of the article.

There are particular issues inherent to each different type of UCC:

1) For selfish UCC, the issue is relevance and commensurability. If I take notes on something for my own purposes, there's no reason for me to care whether it accords with anyone else's notation in any sort of commensurable way. Delicious tries to address this problem by suggesting tags that other people used on an item; this drives convergence on certain terms, making their programmatic understanding of what that item is more robust.

2) For social UCC, the issue is personal opinion over-riding fact. Every conversation implies ego and point of view, and in reading a conversational thread, one has to constantly be asking "what was their motivation in saying that". Bias is inherent to the medium.

3) For selfless UCC, the issue is quality. As people are endlessly pointing out, there is a lot of crap in Wikipedia. You may not agree with someone's opinion on Slashdot, and you may not find my bookmarks in Delicious useful, but there are strong incentives in both contexts for me to at least try to make my content accurate. In the selfless context, well, the incentives aren't just as strong.

This suggests obvious remedies in product design:

1) For selfish UCC, take note of Delicious' example and try to drive general relevance without interrupting the selfishness of your users. Delicious prompts me with terms others have used, making my selfish life easier while making relevance of my actions more general.

2) For social UCC, create programmatic mechanisms (like polls or votes or star ratings) that allow people to quantify their point of view on some abstract scale. This will benefit the user (by allowing them to say simply, "I am *very* Republican," for instance) and benefit everyone else -- who can then say, "He's *very* Republican, and I can easily assess where he's coming from."

3) For Selfless UCC, use human editors to manage quality, and try as much as possible to create "superuser" editors to do this for you. The cost-benefit ratio is outstanding, so get over the stone-ageness of it and go make money.

If you apply this model to a lot of the "Web 2.0" websites out there, it readily explains why there are such issues with the user-created content on most of them. UCC is a two-edged sword, and must be used well to avoid injury.

Note on terminology: I deeply dislike the term "User-Generated Content" (UGC). People don't generate (a mechanistic or animalistic term), they create. Creation is a deeply human act, and we should celebrate it.

Update: Find more at the follow-up post "Market Goods, Cavemen, Schadenfreude"

Ruby on Rails at SDForum Tonight

Tom Hill, one of our senior engineers at Zvents, will be speaking about Ruby on Rails tonight at the SDForum Emerging Tech SIG at the Cubbery Community Center in Palo Alto. Details here:

Zvents is developed primarily in Ruby on Rails, and I can't say enough good things about it. Come find out more about the next wave in Web programming!

SDForum runs all sorts of interesting technical and business events related to software development- you can see their full schedule here.

Squidoo and TechCrunch: Experts Rise Again

When I heard about Seth Godin's new startup Squidoo, I thought, "Aha! That fits perfectly into Nivi's Trillion Dollar Matrix in the 'experts' category." I did a quick Technorati search on the status of the conversation, and discovered that Adam Marsh at EconoMeta has taken the matrix farther in interesting ways -- but pulled the experts column! Not so fast, Adam! In addition to clever programmatic methods like Squidoo for managing and measuring expertise, there are plenty of other expert systems out there. Like Google Answers, or Amazon's Mechanical Turk, which I predict will bifurcate into two main areas of practice; one, the mass processing of tasks where non-expert humans beat computers, and two, the focused processing of tasks where human experts beat computers.

But by focusing on point examples like the Turk or Squidoo, we're missing the hundred-billion dollar present reality that should be smacking us in the face:

Media is about expertise.

I regularly hear about new startups like Squidoo from TechCrunch, which is one of my key personal filters for what matters on the cutting edge of the web. Is Mike an expert? Of course he is, and the fact that TechCrunch has more traffic and better adoption curves than a lot of supposedly hot search startups speaks volumes about the world recognizing the value of expertise.

Talent will out.

The computer era has made clear that talent is much more unevenly distributed than anyone would have thought. In 1975 or 1985, were all the best managers in the world Harvard and Stanford MBAs? No. No. In 1995 or 2005, were all the top engineers in the world at blue-chip corporations and top-tier universities? No. No.

Expertise requires talent.

Let me re-assert Barry Diller's claim that there's only so much talent in the world. Diller's rule of talent (you can probably find the Web 2.0 podcast somewhere, but it parses to "there are few undiscovered geniuses in closets anywhere") means that the best user-created content is a transitory phenomenon. A lot of what Jeff Jarvis refers to as "user created content" is, I would assert, a snapshot of the emergence of talent in new and unexpected places, like blogs. But these people usually don't stay "users." Talented folks who were unevenly distributed vs. the expected structures of recognition, wealth, and power, have a tendency to rapidly move into expected positions, or start reorganizing the structures to reflect new realities.

So we end up with new media, containing new experts, who continue to act as an incredibly valuable relevance mechanism for data.

Hats off to Seth. And hats off to Mike, whose user blog --> expert media site should in no way be thought of as any less complex, cool, or important than startup entities in a different, algorithimic column of the trillion-dollar matrix.

My next post will hopefully not be so long in coming: Thoughts on how it's easy to get stuck between media and practice, drawn from a couple conversations I've had with two guys who are doing both.

Mike: media & practice
Andy: media & practice

My media is here at Onotech; Zvents is my practice.

Tribe Founders Launch Startup Aggregate Knowledge

Paul Martino and Chris Law, both founders of Tribe Networks, have been hanging out at the NSV offices and working really hard on their new startup, Aggregate Knowledge. They have a recommendations web service, WSRelater, live now on the web. I've played with it, and it's awesome. There is some impressive math behind the engine, and the number of cool things I can think of to do with it is rather long. Keep them on your radar screen.

Update: Google Responds, Urchin Works

I sent out an email to the team last night:

Hallelujah! We've got data!
After 49 empty hours, we've got data through 1pm today. Urchin works again!

Google customer service both called and emailed us yesterday as well, apologized for our troubles, and offered to refund money to us. I told them that a refund was nice, but what's really important is that Urchin works. To me, that means our website data consistently available with no more than 2-3 hours' delay.

The big issue is not one startup's angst about its temporary lack of web stats. The big issue is whether it's possible to reliably deliver complex software over the Internet at all. Google is the biggest player in this space, with vast resources of engineering, bandwidth, and servers, but everyone from Salesforce.com to Zvents is attempting to deliver significant software functionality remotely over the Internet.

We are asking our customers to bet their businesses on our ability to deliver. "Flaky but free" simply does not cut it any more. How many of you out there rely on Yahoo or Google email for business-critical communications? What if it went down tomorrow, for 48 hours? What if Salesforce.com stopped working for 48 hours? What if Ebay or Adwords stopped working? Marketing campaigns, communications, and commerce grind to a halt, and real damage is done to real people and real businesses.

Can this really work? The whole premise of the next generation of Web startups is that it can. Zvents is planning on embedding its calendars and serving events into a lot of other people's websites, such as early adopter LinkSV. If we go down, they have no events calendar on their site. As we grow, that customer reliance on us will grow to tens of thousands of similar sites, and if they have no data for 48 hours, there will be anguished screams that will sound very similar to my own post.

I think it can work, but it's going to require not only a business model, but a lot of grownup enterprise kind of thinking, with SLAs and obligations and recompense for failure clearly laid out in contracts. Otherwise, the level of trust simply won't exist to create the very cool future that we're imagining and building right now.

Google: Start acting like a real business or you're doomed

Zvents is an Urchin hosted customer. Yes, customer, as in we pay Google $200 per month to provide our web stats to us. We have been very happy with the quality of the tools that Urchin provides, but intermittently unhappy with the time lag it takes or our web data to appear, and the occasional gaps in our data (we are still missing two hours from last Monday). Google has been kind of OK on the customer service front.

Until today.

I don't know how many other people like us there are - people who paid for hosted Urchin before today's announcement. But I bet that all of them are pissed.

Here's what happened:

I tried to log in to Urchin. Surprise! Urchin now resolves to Google analytics. I typed in our login and password, and I was informed that this account "has to be validated". As it happens, this is a special account just for Urchin and I don't havedirect access to the email address - a not uncommon problem in any organization larger than, say, one. After a few hours of fumbling around and swapping phone calls and emails, I get one of the other guys to send me the validation information.

Next surprise: Because Google is switching to some damn unified login system, whenever I use the login for this account, it messes up any other Google logins I have. Apparently there's a way to pipe this login over to my Gmail account, but it's another hassle-filled step that I didn't have to take yesterday when Urchin JUST WORKED.

Eventually I get in to Urchin. Urchin is getting slammed with traffic. We hear all this hoorah about Google's hundreds of thousands of servers and their ability to host applications, and all I can say is, if casual interest in a service you aren't even accepting new registrations for grinds you to a halt like this, you are nothing like ready for prime time.

Here is a screen shot of what Urchin looks like to me after 10 minutes of waiting and a couple attempts at reloading:



Not very useful.

When I finally do get in to Urchin, I have no data since 3pm yesterday. That's 24 hours without data. Zero. Zip. None. I'm trying to run my business here, folks! Hello?

It's nice that it's now free. But I was willing to pay $200 per month for a service that Just Worked. Google may be all excited about advertising business models, but there are billion dollars businesses built on charging other businesses $200 per month.

Right now, I feel like Google doesn't care about me enough as a customer to tell me that they're changing a product I pay for. They don't care enough about me as a customer to make sure that my login doesn't change, or that they at least ask or warn me before changing my login. They don't care enough about me as a customer to make sure that the re-launch of their product doesn't dramatically impact the people who are already paying them lots of money.

Google isn't acting like a real business, they are acting like an over-enthusiastic Golden Retriever puppy. Oh, they just knocked the vase off the table with their tail, but aren't they cute? Um, no. Google, grow up.

Hand-waving hippie dreams of new media

Uncharacteristically, Jeff Jarvis posted some complete nonsense on Buzz Machine yesterday. He wrote:

Robin Good puts together a few PowerPointable lines on the future of media:

Consumers become producers of content, and niche content surpasses by orders of magnitude the value of traditionally labelled commercial television and film.

The value is not anymore in the best seller or in the blockbuster.

The value is in infinite choice of content and in the opportunity for the consumer to see content when she wants it: prime time is anytime, and anytime is prime time.

Meanwhile, allow me to quote from a recent Slate review of the new Comcast NHL coverage, a piece entitled "The latest desperate attempt to top ESPN":

...if OLN's occasionally amateurish-looking production affirms for viewers that only ESPN can do sports right, then Comcast's national ambitions could find an early, icy grave...

...The network's buffoonish studio show doesn't do much to enhance OLN's credibility....

...The telecasts also gain instant credibility because of play-by-play announcer Mike "Doc" Emrick, who leaves the cliché-ridden competition in the dust—skaters don't merely pass the puck, they "feather it along,"...

...It's easy to sympathize with OLN's early struggles. The channel won the rights to broadcast NHL games...not two months before the opening face-off. At this point in its nascent coverage, OLN's basic charge is to transmit a crisp picture; at least the network's production values compare in quality to local cable broadcasts...

Unless you're so bought in to the new memes of free content from all (dude) you'll note a trend in the Slate review (and yes, they are part of the horrible MSM, in a new-media kind of way)

1) Talent
2) Production values

- "Doc" Emrick is talented and a plus
- The studio team are buffoons and a minus
- Production values are not yet up to ESPN's standard since the OLN only had two months to put the broadcast together, and viewers will notice.

Media is valuable, because our time and attention is valuable. We all choose to consume media from preferred providers, and the two mechanisms by which we choose those providers are the talent on display, and the production values around that talent. I loosely define production values as "capital investments which make the consumption of the core content more pleasant, informative, or easier". For writing, production values include good web design, non-intrusive ads, reasonable colors and fonts (get busy, Andrew!) and so forth. For audible media, production values are things like the recording quality, background noise, the level of editing to remove slow or redundant sections, and so forth. For anything visual, the list of production values begins to be very long, and even more important.

It's a cliche that no one watches home video, despite its existence for decades. The two areas where I can think of home video having real general media value are a) occasional "man on the spot" news clips (Rodney King) and "America's Funniest Home Videos". What links these two together thematically is that they're essentially random -- something either funny or important happened in front of a turned-on video camera, and the value of the content outweighed its lack of production quality.

Here's a graphic that I put together in 2001 for a strategy I was doing for a mobile phone company. Any similarity to Web 2.0 standards is strictly coincidental - I've been thinking in power curves for a very long time. As you can see, based on the empirical data of a search I did on Amazon, there's a nice inverse relationship between cost of production and number of content items.



Why is this? There are two dynamics at play. First, in any gathering of people to create something, some people are more talented than others. I participate in some small-scale local theater and performance, and when there are six people in a room, usually everyone knows that one or two of them are better than the rest. These talented people tend to like working with other talented people, and over a small amount of time, quite a sorting effect occurs. Pretty soon the talented gang invents a name, throws out a shingle, starts charging, and voila! the naescent MSM is born.

When you add money into the equation, the effect is to enhance the differentiation. Perhaps a financier made money once, sometime, in a far distant past, by *not* seeking out the best directorial, writing, performing, and technical talent for his production - but I doubt it. When real dollars are at stake, people seek out the best, and then fund them. Those who aren't the best don't get funding. From movies to music to television to video games, this is an obvious trend even in relatively "new" media like video games.

Example of the "talent self-clusters" point here, here, and here.
Example of the "financier seeking the best talent" point here and here.

Oh, ze world, she turns but she stays ze same...

I do believe that temporarily, at least, some barriers have fallen in our new world. Here I am, writing this blog. But my choice with this blog is very clear: I can either a) invest a lot of time in it to create quality content on a regular enough basis to develop a readership or b) I can get on with my real life, which is running a cool new startup called Zvents, and only post here occasionally because I feel like it, and not worry about becoming a media property. I am never going to out-do BuzzMachine in popularity or readership unless I put some serious work into the effort, which I'm simply not going to do. Therefore I will 98% remain on the "Consumer" side of the blog equation, despite my reasonable levels of inclination, ability, and empowerment to become a producer. Meanwhile, Jeff will sit on the opposite side of the equation, and take each successive step up the production values and monetization ladder which develops over the next few years, and in 36 months, it won't be possible to compare my blog to his with a straight face - if it even is now.

Jeff will have, by dint of wildly disproportonate readership, a "blockbuster" on his hands, giving lie to the baloney in his recent post.

Whenever anyone starts talking about new democracy in media, just remember that for the past 100 years, any normal person with about $50 to their name has been able to buy a typewriter and a sheaf of paper, sit down, and create a novel. For the past 100 years, being a successful novelist has been a well-paying, respected position. The fact that so few have managed to do this, despite the legions who have tried out of the near-totality of all who could have tried, indicates that there's something slightly more fundamental going on than big mean corporations holding back the little guy.

That something is the rarity of talent, the social network clustering effects of production values, and the reality that all of our time is limited, and brands are a great way to short-circuit searching costs.

The wonderment of the current age of the internet is not that the new winners will look any different than the old winners, but simply that a window of opportunity exists to make new winners at all.

The Real Microsoft Memos

Everyone is talking about the transparently leaked Microsoft memos. The version presented here (courtesy of the unchained snark that is The Register) is much more entertaining than the ones you've read elsewhere:

Gates stirs Microsoft with dramatic 'more meetings' plea

By Andrew Orlowski in San Francisco

Published Thursday 10th November 2005 00:04 GMT

Analysis Ever the master of public relations, Microsoft has always been able to figure its way out of a tight spot with the use of a judiciously leaked memo.

...in the spirit of the excellent 500-word "digested reads" offered by some of our better newspapers, we give you the précis of the latest Gates and Ozzie memos. Then we'll put the whole affair in some historical perspective.

From: Bill Gates

Sent: Sunday, October 30, 2005 9:56 PM

To: Executive Staff and Direct Reports; Distinguished Engineers; All TV, print, radio and internet news outlets; All bloggers [delete last five before release]

Subject: Internet Software Services

Microsoft has always had to respond to innovators in the software business and seize the PR initiative.

Ten years ago this December, I wrote a memo entitled The Internet Tidal Puddle in which I attempted to undo the damage caused by my book, The Road Ahead, published just three months previously.

In The Road Ahead I failed to mention the internet at all. The same month, Netscape had floated on NASDAQ, creating the largest ever first day gain. Clearly, there was a puddle in the road ahead. So in my memo I warned that we could either step round the puddle, or step right into it, and risk being drowned.

In 1995 none of our products made use of TCP/IP. Now, a decade later, I can safely say that many of them do.

In 1995 I warned that we risked losing the mindshare of a new generation of internet software developers who chose to shun Microsoft APIs completely. Now, a decade later, I can say that close to 100 per cent of Internet-born viruses, Trojans, rootkits and worms use Microsoft APIs exclusively.

However, to lead we need to do far more.

In order to execute on this opportunity we must look as if we're acting quickly and decisively. Recently competitors [Steve, chair down. DOWN.] have gained massive public attention by rolling out products that remain in beta for many years. This is unacceptable. Products that remain in beta for many years should be first and foremost, in the public's mind, Microsoft products.

For three months Ray Ozzie has been sitting in his orgone accumulator devising a strategy to ensure we can execute. I've attached a memo written by him that I think strikes the right balance between pre-announcement and non-delivery. I'm happy to share it with you.

Bill

Go read the rest here.

NumSum, Nivi, and a Trillion Dollars

Nivi just posted a response to some of my stuff on social web. We also had a really interesting conversation over at Bessemer with James Cham on this subject the other day. Nivi's matrix is empty and hard to play with, so I put in in NumSum, which is Steve Yen's latest project:



Play away.

SDForum Search SIG @ Microsoft Thursday

John Battelle and an interesting panel. I will be here:

Scoble asks what you call 'em? Raw Potatoes!

Scoble asks what you call things like Flickr, Microsoft Gadgets, Google Maps, Amazon Affiliate parts.

Quote:

I’ve been struggling to communicate with others what the new Internet ecosystem is made up of and it hit me a few minutes ago.

They are Internet Connected Components.

Hear me out.

When you go to Kayak Buzz or Zvents, what do you see? Two ICCs. One is a Google Map. Another is a AdSense bar...

...I think we need a non-branded name to generically refer to these things. What do you think?

Here's my proposal: raw potatoes. Why? Because they're what you mash up.

If you call 'em rawpotatoes (no space) you can create a nice searchable tag/phrase that doesn't currently mean anything else. When's the last time you saw a search result this empty?

Hats off to Chris Law for reserving the URL already. I'm sure he'll be happy to build it as an open community resource for Web 2.0 just like he's done with WSFinder.

Yahoo clarifies Maps API

Vince has posted over at YDN about the Yahoo Maps API:

I wanted to take a brief moment to clarify our terms of use for Yahoo! Maps, as many folks have contacted us asking whether they could use the APIs on their site.

Our position has always been to allow usage of the Yahoo! Maps APIs free of charge for non-commercial use, as well as commercial use granted on a case-by-case basis. This is defined in our FAQ which also has instructions for how to contact us should you want to seek permission for commercial use.

In general, if you are displaying mashups featuring Yahoo! Maps on your site or application and you make your stuff available for free to users, you’re welcome to use the Yahoo! Maps APIs. This is true even if your site is supported by ads -- even ads from other vendors.

But while I've got you thinking about monetizing your site, allow me to make a shameless plug for something else we’re doing at Yahoo! -- take a look at the Yahoo Publisher Network BETA. Revenue, relevant ads, no hassle. Easy. Sign up today for your YPN beta invite.

Check out my follow-up post at the Zvents blog for the rest of the story.

The Zvents team is now going to get back to work building product. Google, Yahoo, Scoble, we love you all. Peace and APIs.

Blogosphere Speaks, Yahoo Responds (Cool!)

Yahoo launched a cool new maps service today, which got great reviews, tons of buzz, and complaints about commercial restrictions.

They also got a "Yahoo Maps are Doomed" post from Robert Scoble, in which he used my company Zvents as the poster child for Google's ongoing dominance.

Not so fast, Robert. More on this tomorrow, but I just got off the phone with the Yahoo Maps team, and they said that tomorrow they will be removing the "non-commerical only" clause from their TOS, and that Zvents, as a commercial site, is "golden" to start using their APIs.

Yahoo, you rock!

Robert has raised some fascinating points in his post. I'll address those on the Zvents blog tomorrow morning. Watch that space.

The Sincerest Form of Flattery...

This post appeared on Craigslist the other day. It's funny enough that you should click through to Flickr to read the details:



We thought it was awesome, so we created a proper home for it on the wall at the Zvents office:



Humor aside, this raises a key point about Web 2.0. The same factors that mean that "two guys and a router" can build a business like Facebook in ten days and have it take off, mean that two more guys and another router can build a business and compete with your cool new business.

There are only three ways to get to market differenation: Technological advantage, network effect, or marketing-driven aggregation of eyeballs. There are great examples of success in all three categories; but there are no examples of success that didn't pull off one. And we all know that the first two are waaaaay cheaper than the third.

It's a marathon, guys. Keep running.

Comment Hurdles, Zipless Fucks, and the Extended Conversation

At the end of a great review of Zvents (Thanks!!), Zoli Erdos points to my blog and says,

"the last link happens to be Ethan Stock, Zvent CEO’s blog. Hmm… I don’t see any way to leave a comment or trackback – what happened to the “conversation”, Ethan?"

I thought about posting a snarky comment on his blog, but this is what faced me:



Yuk.

Zoli, it was easier to come here and write this post than it was to face that. You're linked to me, I'm linked to you. That's a conversation.

What's going on here, is the empirical demonstration that anonymity and conversation are inimical to each other; and that in order to participate in a community, you must commit and invest resources. In the classic phrase of Erica Jong, there is no 'zipless fuck' in the Extended Conversation.

Zoli has a blog, and I read it. I have a blog, and he reads it. We can converse through links and references. Each of us has put some time and effort (commitment and resources) into building up this presence in the community, and cautiously, bit by bit, each of us is willing to invest time and energy and possibly a RSS subscription or a delicious bookmark to the other.

Comments on blogs were conceived in a simpler time, or with a simpler mindset, that said that anyone should be able to drop in and speak their mind - the zipless chat. But spammers infected that ideal, as sure as hepatitis and AIDS ended Erica Jong's free-love early 70s.

This isn't a sad story -- community and relationships and investment still yield great results, and if we want unexected, serendipitious conversations with each other, the best way to do that works even better in this era of the Internet -- face-to-face, the old-fashioned kind of communal commitment.

What Kind of Society Are We Building?

I've spent much of the last six months heads-down building Zvents. I am incredibly excited about the opportunity to build a great new product that will, in some small way, change the world and make users happy. Users are these slightly foreign beings to us product-builders, and sometimes it's hard to get inside their heads and understand what their real concerns are. When I read something like this New York Times article on Wal-Mart's HR practices:

"...The [board] memo acknowledged that Wal-Mart, the world's largest retailer, had to walk a fine line in restraining benefit costs because critics had attacked it for being stingy on wages and health coverage. Ms. Chambers acknowledged that 46 percent of the children of Wal-Mart's 1.33 million United States employees were uninsured or on Medicaid.

Wal-Mart executives said the memo was part of an effort to rein in benefit costs, which to Wall Street's dismay have soared by 15 percent a year on average since 2002..."

I am stunned. How can any user be thinking about movies, or bands, or Little League games, when their kids don't have healthcare? Will they ever use Zvents? Not likely. Wal-Mart is the largest U.S. employer and regularly held up as a paragon of new business practices. Why isn't Wall Street worried about the 46% of Wal-Mart kids who don't have health care? Even in the most narrow commercial view, aren't those kids the future customers and employees of the Wal-Marts of the next generation?

Henry Ford single-handedly created the 20th century social compact in 1914 when he unilaterally raised the minumum wage of his workers to $5 per day, reasoning that if his workers couldn't afford to buy the cars they were building, the great hamster wheel of capitalism might stop spinning:

On January 5, 1914, Henry Ford announced a new minimum wage of five dollars per eight-hour day, in addition to a profit-sharing plan. It was the talk of towns across the country; Ford was hailed as the friend of the worker, as an outright socialist, or as a madman bent on bankrupting his company. Many businessmen -- including most of the remaining stockholders in the Ford Motor Company -- regarded his solution as reckless. But he shrugged off all the criticism: "Well, you know when you pay men well you can talk to them," he said. Recognizing the human element in mass production, Ford knew that retaining more employees would lower costs, and that a happier work force would inevitably lead to greater productivity. The numbers bore him out. Between 1914 and 1916, the company's profits doubled from $30 million to $60 million. "The payment of five dollars a day for an eight-hour day was one of the finest cost-cutting moves we ever made," he later said.

There were other ramifications, as well. A budding effort to unionize the Ford factory dissolved in the face of the Five-Dollar Day. Most cunning of all, Ford's new wage scale turned autoworkers into auto customers. The purchases they made returned at least some of those five dollars to Henry Ford, and helped raise production, which invariably helped to lower per-car costs.

We're a tiny little company at Zvents, and soon we'll have a health care plan in place for all our employees. Not just for the highly-paid engineering types, but everyone. Wal-Mart can do better, and if they don't, and if Wall Street doesn't lengthen its short-sighted view, the hamster wheel might start slowing down.

Update on Cringely's Bad Math

Following up on my highly skeptical response to Cringely's "the sky is falling" web-hosting post, Brad Gibson, who designs data centers for a living, dropped me an email suggesting that I check out his blog. Here's a sample of his analysis:

Cringely overstates the actual (as opposed to the name-plate rated value) power consumption of the drive arrays by at least double. He also gets muddled up when he discusses data center floor space; mixing up total square footage with the square footage of the connected load space. He uses an incorrect approach to calculating "necessary" space to support "equipment" space. He makes some incorrect assumptions about the types of drives used in many of the arrays. The Cringely argument also completely overlooks load diversity in a data center.

Check out Brad's excellent piece here.

Brad's post also pointed me to the coverage of the Cringely piece on Memeorandum.

Three Postulates: The 'Long Tail' <> User-Created Content

At this point, anyone who is reading this blog has seen Chris Anderson's 'long tail' graphic so many times that I can *not* post a link to it, and say, "the yellow kind of clashes with the red/pink background, don't you think?" and you'll all be able to bring up a mental image in your head.

Let me tweak your memory further. Remember Chris' examples? The two key ones that stick in my mind were Amazon's catalog, with that fantastic quote to the effect of, "half the things we sell in a given day, we only sell one of"; and Netflix's movie rental business, where 'infinite shelf space' enables even the most obscure movies to add to the bottom line of the business.

Sure, Ethan. So what's your point?

Well, may I humbly point out that pretty much none of the items for sale on Amazon, or for rent on Netflix, are user-created in the sense that all we Web 2.0 junkies think of user-created content. None of them are user-created in the way that blogs are user-created. They're all commercial movies, or commercial books or toys or power tools, that may be rare and obscure, but were created by some professional organization with commercial intent. You do not see things like Harry Potter Fan Fiction making up even a small part of Amazon's long tail. Why is this? Not to be mean or anything, but intellectual property issues aside, there's an obvious grammatical error in the first nine words of the top-rated Harry Potter story on that site, and it doesn't get much better after that.

Let's look at another bastion of user-created content, Slashdot. Slashdot works like this: It has a home page, and a set of topical sub-pages, which are run by editors. They choose what stories make it on to those pages, and then all and sundry comment on the stories, and moderate each other toward nirvana or oblivion. Note that the stories are a) written by someone who is not a Slashdot reader and b) edited / selected by someone who is not a Slashdot reader. Slashdot readers then, with either great enthusiasm or great skill, but rarely with both, comment on the articles, adding a valuable corona of information around the core star of the article.

Are crowds wise? Sure they are. But we all agree that committees are stupid, and isn't a committee just a small crowd?

Ethan's First Web 2.0 Postulate:

"The most successful new businesses will not be built on user-created content, but will utilize it extensively."

Google is a perfect example of this postulate in action. They crawl the web extensively, and aggregate everything they can find. They then apply relevance algorithms such as hubs + authorities to decide which pages to serve up. These relevance algorithims are, as several people have recently twigged to, social software -- aka user-created content. (Note: I blogged about this in late June -- I guess I need more readers!) However, the content which we are seeking via Google - the stuff which the users are ranking through their links -- is almost always professionally created content with commercial intent, although much of it comes from the very long tail of obscure professional content.

Interesting, huh? Google takes on the one hand, an index of all the content out there (most of the useful stuff professionally created) and then adds a relevance algorithm based on what a bunch of users think of that content, and voila! market-leading search. Then they add a bunch of professionally created ad content, and add a relevance algorithm based on what a bunch of users actually think of those ads, and voila! A $106 billion in market cap.

Does this story sound vaguely familiar? How about Amazon's professional-content store, with a heavy wrapper of user opinion and review? How about eBay's professional content marketplace, with a heavy wrapper of user opinion (seller and buyer ratings) plus user-specified value algorithms in the form of final auction prices? I think I see a trend.

Ethan's second Web 2.0 postulate:

"The number of users who will usefully comment or vote upon a topic is one to two orders of magnitude greater than the number of users who will usefully create content for that topic."

Since we all know about Metcalfe's law and its new-meme supercharged cousin Reed's law, which Fred Wilson has been talking about, it's clear that if we want to build highly scalable value in the network, we might be better off focusing on capturing user opinions at scale, rather than capturing user content at scale.

Huh. I kind of think as I write these (snarky readers will respond, "We can tell, Ethan," while super-snarky readers will respond, "Not that we can tell, Ethan"), and I've just realized Ethan's Third Web 2.0 Postulate:

"Every time I think hard about where the web is going, del.icio.us seems right in the middle of it."

Cringely's Math Makes No Sense

I read Robert Cringely compulsively - he's a lively writer, a deep thinker, and totally plugged in to several frontiers of technology. But every once in a while (as with his July column on Skype) he makes claims that are not just wrong, but ludicrously wrong.

Here's the essence of his last column, which I have condensed to save you the three minutes I wasted reading it:

"A lot of money is being bet on a future user computing experience based on web services" but "the cost of keeping [user] data online all the time will be huge. It's an energy crisis in the making." "202 million [U.S.] Internet users..." * "a free Gmail account with two gigabytes of storage..." = "400 petabytes. That's 400 times the current capacity of the Internet Archive."

"that's really only about $25 million in disk drives..."
+ "total power consumption up to just under 10 megawatts, which at typical U.S. industrial power rates will cost about $5 million per year."

"This is the kind of planning and provisioning required to support FREE services...That's a heck of a lot of ads."

"My point here is that we're entering another period of Internet exuberance...the Internet will change even more than it has the ways we live and work. But it isn't going to come easy and it isn't going to come cheap."

Huh? Cringely just wrote that the big portals can create a kick-ass email service for *every single internet user in America* for a disk drive cost of $25 million, and an annual power cost of $5 million. Generously speaking, let's say that the total capex would be $125 million and the total annual opex would be $25 million (5X on both his numbers). That's a capex per user of 62 cents, and an annual opex per user of eight cents. And he says this is a problem? Dude, that's two postage stamps!

There are three great systems of analog user-created content creation and distribution in place today. They are the U.S. Mail, the phone system, and the camera/photography/film infrastructure. Voice, text, and images, analog-style. To give Cringely as much slack as possible, let's look at the smallest and cheapest, film photograpy. I called a knowledgeable friend, who told me that in the United States, there are about 25,000 minilabs for processing photographic film, and that the average cost of these labs is about $200,000. That's a capex of $5 billion. And that's just for photo prints at your local Walgreens, CVS, or Ritz Camera. If we assume one $10 per hour clerk full time to run each of those minilabs, our annual opex is $500 million. Don't forget that behind those minilabs are film plants, paper plants, distribution networks, and SuperFund sites.

By any wild stretch of imagination, the new digital infrastructure being put in place will be vastly cheaper, both to build and to run, than the old analog infrastructure that it's replacing.

The value of that infrastructure, compared to its cost, is incredible: Cringely carps that $30m is "a heck of a lot of ads" the day before Google announced that they'd just sold $1.05 billion worth of ads in their last quarter alone.

Wake up, Robert, and smell the bits.

TechCrunch BBQ 3: This Dog Got his Dinner

Michael and Keith threw a great party for the 3rd TechCrunch BBQ. It's kind of amazing how fast things are moving out here - Just as an example, I met both Kevin Burton and Jeff Clavier for the first time at the second TechCrunch BBQ, and I've seen each of them so many times in the past few weeks that they're starting to feel like old friends.

I have been deeply conflicted about the whole Web 2.0 buzz. I think that a lot of us who have just been trying to build great products for the past 10 years feel like a dog that's been patted and kicked intermittently, and we're not quite sure if it's time for dinner or a beatdown. So everyone is working their tails off, but mixing it in with ongoing self-mockery and self-reflection. At my first startup ('96), we took to calling ourselves "the last software company" as we watched a bunch of "web" companies take over, take off, and fall off a cliff. Now Zvents is running as hard as it can, while looking around at some of the fluff and saying, "maybe we're... kind of... a web 1.3 company?" The Web 2.0 conference made things worse, not better. It was crazy -- at $2900 a head or whatever they were charging, it drew some people who think too much about money, and not enough about building stuff. People who needed to be talked to in big ballrooms by reassuringly profitable, public, branded figureheads, before going out to open-bar parties hosted by a bunch of cool 19-year-olds and saying, "so this is what '99 was like."

No, actually. For it to have been like '99 would have taken a lot more MBAs, a lot more New Yorkers, a lot more 24-year old sociology chicks making $150K in high-tech PR, with a vast fog of general cluelessness about technology AND business overlaying it all. Thank God it wasn't anything like that bad.

With all that angst rattling around in my head, Michael's party tonight was amazingly refreshing. It was pretty much entirely populated by real, genuine, Silicon Valley people. Startup people. Big company (Google, Yahoo, IBM, Ebay) people. People who've built stuff, and done stuff, and made things happen. People who understand that you have to cooperate to make standards, and compete to make great products. People who know that elegant solutions to computationally hard tasks matter, and that compelling end-user experience matters too. It felt very real. There are some great products being built, and we got some fascinating peeks at What is Yet To Come.

I feel much better about this whole Web 2.0 thing after tonight. This dog got his dinner.

P.S. A few pictures:




See Kevin's Feedblog for details on Dave Winer's $20

This post should probably be tagged techcrunchbbq or techcrunchbbq3 or somesuch.
It's the Tag Guess Dance, aka Bubble Up from the Bottom, Baby!

Kosmix.com, Vertical Search for Healthcare, Launches

All of Web 1.0 continues to be reinvented around the search paradigm. Healtheon/WebMD, watch out: Today, Kosmix.com launches to provide healthcare search. I know two very smart search engineers there, Jason Zien (from IBM's WebFountain) and Ram Subbaroyan (from Inktomi). Kosmix is founded and funded by the Junglee/Cambrian nexus, and I expect them to kick some serious booty.

Chris Law Interviews Yahoo Developer NW's Toni Schneider

Chris just pinged me and said that his podcast interview with Toni Schneider is up over at WSFinder. Toni, who was CEO of Oddpost before its acquisition by Yahoo, now runs the Yahoo Developer Network. If you're curious where this whole "web as platform" meme is going, knowing what Toni thinks is a big chunk of the answer. Toni discusses the Yahoo Music Engine, Rollyo, Confabulator, Flickr, commercial terms for APIs, and a lot of other stuff. It's a pretty interesting listen.

3rd TechCrunch BBQ This Friday

Michael is taking the wrappers off of Edgeio. Lots of cool demos. Loads of interesting people. Check it out on Zvents:

Web 2.0 -- amateur vs. professional?

Matt Marshall at Silicon Beat wrote a great, link-rich post on the open/closed debate of Web 2.0. Among other things, he cites Nicholas Carr's essay, The Amorality of Web 2.0, where, among other things, Carr shreds what he calls the 'cult of the amateur':

In theory, Wikipedia is a beautiful thing - it has to be a beautiful thing if the Web is leading us to a higher consciousness. In reality, though, Wikipedia isn't very good at all. Certainly, it's useful - I regularly consult it to get a quick gloss on a subject. But at a factual level it's unreliable, and the writing is often appalling. I wouldn't depend on it as a source, and I certainly wouldn't recommend it to a student writing a research paper.

I think that Carr is right. I think that amateurs are unrealible. One of the most fascinating little plays of the Web 2.0 conference for me was watching people's reactions to Barry Diller's talk. Diller is a very savvy billionaire media mogul, who has pulled off bigger bets against longer odds than anyone else who spoke at the conference - including the founders of eBay and Google. Diller made what I consider to be some obvious statements, like, "there's only so much talent in the world," suggesting that there are limits to what amateurs can accomplish. In several subsequent conversations, people went out of their way to criticize Diller, shaking their heads and mouthing variations upon, "he just doesn't get it." These being the same people who mouthed salutations about Terry Semel's respectable, by-the-book defense of Yahoo's experts+amateurs, us+users positioning in the marketplace.

Well, Diller is right, and I think that Tim O'Reilly knows it. O'Reilly, after all, is possibly the man most steeped in open source on the planet -- and most open source projects are the creations not of a diligent factory of excited amateurs, but rather the crafting of a small, dedicated group of professionals who are working outside the established structures of traditional software firms. Apache wasn't written by a herd, and Ruby on Rails is largely the creation of one really smart guy. If anything, open source proves not just the appalling common denominator of the amateur, but the opposite - the transcendence of the truly talented. I liken open source to the kind of flowering that took place during the scientific revolution in England -- a relatively small group of really smart people working together openly to create some truly revolutionary advances, and then a much larger group of people taking advantage of the new climate to make the world a better place.

I believe in the value of user-created content. But I think that value is centered in, and almost entirely contained by, the province of opinion. Voting. Editorial. Commentary. Not core content, not data. We've got a hybrid us+user model at Zvents, and I'm happy if that makes us Web 1.5.

What's different about Web 2.0? Left Replaces Right

Remember Web 1.0? All the graphs went up and to the right:



Like in this Google Image Search

But now we're in the vastly better Web 2.0...

Where all the graphs go up and to the left:



Like in this Google Image Search

Yeah, it's a different, different world.

Web 2.0 Hype? Stewart Butterfield and Chris Law Bet On It

At the closing reception for the Web 2.0 conference (Thanks, Yahoo!) I introduced Chris Law, a founder of Tribe Networks, to Stewart Butterfield, who founded Flickr. As the three of us chatted, it became apparent that Stewart and Chris have very different views of where we are in the Web 2.0 hype cycle. So different that with some small incitement from me, they agreed to a $20 bet, to be paid at next year's conference. I'm the judge and this blog post is a virtual holding-of-stakes. Here's the bet:

Stewart: "This is likely the top of the hype, and even if it isn't, by next year it is going to feel so overdone that the name will have to change. This conference will still happen, but it will be called something other than Web 2.0 and look a lot more like an ETech."

Chris: "This is nothing like the top of the hype; this conference will not only happen next year, it will still be called Web 2.0 and will be bigger and crazier than this year."

Stakes: $20.
To be paid: Next year's conference.
Proof point: Officially the name of next years' conference, but I think all three of us will know who won the bet no matter what it's called.