Tuesday, May 23, 2006

Dear Google: Arabic Gmail to estock is spam

I like Gmail, and I use it as a personal account for this blog and other purposes. One of the things I like best about Gmail is the one-click ease of reporting spam and having it vanish from my inbox, with the sense of accomplishment that due to my small contribution, the Google anti-spam terrier is getting sharper teeth and a nastier disposition.

Except the terrier seems to be sleeping. Unless I am blind to some subtle i18n character set issues, it should be really, really easy for the Bayesian filtering algorithms over at Google spam central to figure out that hey, *every single* Arabic email that estock has received, he's classified as spam! And look, *every single* cyrillic and Mandarin character set email he's received, he's classified as spam as well! Hmmm. What could we do with this information? I think that just possibly, entropy could be reduced a little by incorporating it.

Ten or so of these a day, every day, get old. No, I do not want to hire any stock brokers from Dubai, even when they write to me in English -- and especially when they write in Arabic. Really. Hey, spam terrier -- bite 'em!

Lunch 2.0 at Zvents -- Wednesday, May 31st noon

Come join us for BBQ in the back yard of Zvents / NetService Ventures -- part of the Lunch 2.0 series started by Mark Jen and the guys at Plaxo. Read about the whole Lunch 2.0 series here.

RSVP by commenting on the Lunch 2.0 web site, or by adding a comment to the zLunch event page: Zbutton

Parking will be tight, so please carpool, or park at the Sharon Heights Safeway, which is about a 5 minute walk west on Sand Hill Road (turn at the light at Sharon Park Dr.)

We'll provide all the food and drink, you provider the conversation!

Sunday, May 07, 2006

Daniel Yergin's "The Prize" - History of Oil


I was chatting with one of my self-identified libertarian friends tonight (he started out the conversation with "were you referring to me in that last blog post?") and I ended the evening by recommending Daniel Yergin's fabulous book The Prize to him. It's a history of oil from its discovery in 1850 through to the first Gulf War in 1990, and it won a well-deserved Pulitzer. It's a huge tome (800 pages, I think) but an outstanding history of the single most important factor in world history during the past century and a half. If you read it, you'll be amazed how the thread of oil runs through the waning days of imperialism, both World Wars, and the Cold War... it really has been the fulcrum of power in modern times. Highly, highly recommended.

Airlines vs. Fuel Costs = Dirigibles?

I just read a fascinating article in the New York Times about Boeing, Airbus, and the future of commercial aviation. Boeing, as airplane geeks will know, has bet heavily on the 787 'Dreamliner', a highly fuel-efficient midsize plane designed to serve point-to-point international routes -- San Francisco to Shanghai, Athens to Boston. Airbus, on the other hand, is building the massive A380, a plane designed to carry 500+ passengers on core 'hub and spoke' routes such as New York to Tokyo, with smaller planes serving as 'feeders' to these routes.

This multi-billion-dollar faceoff between two different philosophies and infrastructures is reminiscent of so many other distributed-vs.-centralized battles. FedEx created a business by going with centralized hubs. "Mesh" networking is supposed to be more efficient and resilient by getting rid of hubs. However, it's clear in this case that one simple calculus will decide the fate of Boeing vs. Airbus:

Fuel cost vs. customer happiness.

As recently as 12 months ago, I would argue with friends about 'peak oil' and what an economy looks like as it slides down the back side of a bell curve of production; and I would get signficant pushback that we were anywhere close to such a circumstance. The more libertarian-minded were fond of citing the famous Simon-Ehrlich bet on commodities as a supporting point.

And here we are. Massively rising demand for oil from China and India; potential disruption of significant sources of production ranging from Venezuela and Bolivia, Nigeria and Sudan, Iraq and Iran, to the Khazak basin in the former Soviet Union. No massive new sources of oil discovered, and few likely to come on line soon. And wild cards such as the Canadian tar sands are predicated on... high oil prices to make them relatively cost-efficient to produce.

houston dirigible postcard

Which got me to thinking: If the tradeoff for airlines is fuel efficiency vs. customer satisfaction, why not dirigibles or blimps? Right now it takes about 10 hours to fly the 6000 miles from SF to London, at about 600 miles per hour. An appropriately designed dirigible could do it in 24 hours at 250 miles per hour, at a vastly (90%?) reduced fuel cost -- since a dirigible would benefit both from the cubic reduction in power-required vs. speed flown, and the absence of the need to expend power to keep the aircraft up in the air, which accounts for a large percentage of airplane fuel cost. Imagine that, instead of spending 10 hours on a cramped, noisy, EXPENSIVE airplane, you spent a full day and a full night on a quiet, spacious, dirigible? Broadband internet access would be essential -- not only could you make crystal-clear phone calls, but you could transfer any volume of data. You'd get nice meals from a large kitchen. You could walk around and exercise. You could sleep in a real bed. And in a world of $70 - $140 a barrel oil costs, all of this might be CHEAPER to provide than a miserable 10-hour flight.

The key, of course, is making the experience as business-like for business travelers, and as vacation-like for vacation travelers, as possible. Without doing some considerable aeronautical math, I can't estimate at what fuel price point dirigibles would start to make sene. But it's certainly fascinating to consider.

Wednesday, April 26, 2006

Portfolio Management, the Yale Way


Jason Zien over at Spotsearch recently wrote a post about Yale's asset allocation strategy, which has garnered amazing results - read Jason's post for the details. In about 2001, I read a book by David F. Swensen, the Chief Investing Officer of Yale University, called Pioneering Portfolio Management. It is one of the better investment books that I've ever read, and I highly recommend it. Swensen apparently has a new (2005) book out, "Unconventional Success," which is also well recommended on Amazon.

Tuesday, April 25, 2006

Ponytail Sweepstakes: Ted Waitt, Gateway

I've gotten a few emails based on yesterday's post about Jonathan Schwartz. It has been brought to my attention that Gateway founder and ex-CEO Ted Waitt was ponytailed for much (or all) of his tenure at Gateway:



There's a great quote on Waitt in a recent Fortune article, that foreshadows Sun's challenge:
"But while Waitt, the ponytailed visionary, was looking toward the future, boring old Michael Dell was obsessing over efficiency, thinking about how quickly a PC could be assembled rather than how fast it ran. As it turned out, execution was everything."
The ability of the Internet to quickly and easily answer idle historical questions such as "what was the market cap of Gateway during Waitt's tenure" is still limited, but a quick scan of the stock price suggests that it may have been as high as $28 billion circa 2000. This would significantly eclipse Sun's current $17B number under Schwartz.

The race is on, Jonathan!

Monday, April 24, 2006

"What's a Blog?" - Six Apart at Maker Faire

This was my absolute favorite sign at Maker Faire. I promised Elizabeth (the girl in this picture) that I'd blog about it, so here it is.



There was lots of other cool stuff at Maker Faire, including awesome mathematical castings from Bathsheba, which embarassingly I failed to photograph. Oh, and I-Wei Huangs's steam robots!

Scott out, Ponytail in, shares up!

Sun send Scott McNealy off into the sunset today, after a run of 22 years at the helm. I can't think of too many other CEOs who have run a company that long -- though I bet Michael Dell will get close.



In related news, Jonathan Schwartz is CEO -- does anyone out there know of a CEO of a company larger than Sun run by a ponytail-sporting guy? He may have the market-cap record for that particular feature.

I know a lot of guys rooting for him to hold the "return on shareholder equity" record for ponytails, too.

Search Trends: The Search For User Intent

There are several interesting trends afoot in the world of search. As I have previously written, concepts as disparate as editorial news and personalization are simply different ways to get to a better search answer. I'm now ready to extend this thesis a bit further, and claim that the next big leap in search is going to be driven by a superior ability to acquire and understand user intent.

In any query:response system, the quality of the output is fundamentally limited by the information contained in the query. In a mathematical sense, it's impossible to have more significant digits in your answer than you had in your input data. No matter how well you tune your response algorithm, not only will "Garbage In, Garbage Out" always hold true, but the more general case of "lack of precision in, lack of precision out" will limit how good your results can be.

On a general search engine (Google, Yahoo, etc.) an inbound user query can, quite literally, be about anything. Presented with a blank box that can encompass anything, the user then types in an average of 2.4 words, and the search engine must give back an ordered list of 10 relevant results (having worked very hard to remove spam, porn, etc.) that may approximate what the user is looking for. Suppose the user types in "Peru". Is the user a 4th grader writing a report for school? A college student planning a backpacking trip? A consultant searching for economic data related to a project? A Peruvian looking for things related to their home country in their local area? All these, and many more, are possible.

It's a very tough problem, and a key reason that search quality has ground to a halt, or at least stopped improving dramatically, is that in this completely general context, it's very hard to understand more about the user's intent than they grudgingly give you in their 2.4 words.

So how can search engines solve this problem of paucity of intent?

Answer #1 is to verticalize. Zvents, my startup, is an example of this trend -- as are Simply Hired, Kosmix, and many others. When someone types Peru into Zvents, we know -- simply because we're a search engine for local events, and nothing else -- that the user is looking for something to do in their local area having to do with Peru. In order for Google to get that kind of intent, the user would have had to type, "something to do in my local area having to do with Peru" or a similarly complex query. Zvents gets a great deal of intent information simply by being a specialist. When someone shows up in our search interface -- either directly, or via a media partner -- we can be confident they're looking for stuff to do. Simply Hired, similarly, would know that the user was looking for jobs either in Peru, or related to it; and Kosmix, under its healthcare filter, could infer that the user cared about health issues related to Peru.

Answer #2 is to personalize. Google is the most interesting case emerging at the moment. Go to the Google homepage and in the upper right you'll find a link to the new personalized home. It's a HUGE departure for Google -- who has built a multi-billion dollar business on a purely "visitor" experience -- to be moving to a registered user experience. It's likely that Google is doing this is to get a much clearer sense of search intent, based on personal past search history. At least one very senior technical person involved in the Google personalized home page has a background in mining extremely large log files to extract business intelligence -- exactly the kind of resume you'd want if you were attempting to derive intent from search history. I've also had some discussions with folks at Yahoo (who are also very happy to let you log in while searching) about the extremely sophisticated state information that Yahoo maintains as you move through the Yahoo portal; all aimed at improving the targeting both of ads and products.

Answer #3 is to get more intent directly from users. Many of you might recall the original concept of Ask Jeeves, which was to "answer your questions in plain English". In a recent conversation with some folks at Ask, they mentioned that one of the reasons they'd retired Jeeves was that delivering on this "brand promise" was nearly impossible - but that because of that historical promise, the length of their typical search query was vastly greater than comparables at the other big search engines. That begs the question -- what if you could successfully parse natural language English like "something to do in my local area having to do with Peru"? It's an incredibly hard problem, but the ability to accurately extract enough user intent to deliver a truly better search result would yield enormous market benefits. I know of at least one current serious search startup that is taking exactly this approach -- building a general-purpose search engine that can better parse complex intent, and accurately respond to it. If they succeed in their quest, and establish a "brand promise" similar to the original Jeeves, they'll have a significant advantage in actually giving users what they want.

There are other answers, but they look less and less like search. Aggregate Knowledge, for instance, is a matching engine for supplementary navigation based on heterogenous item:item matching in a "people who viewed this, also viewed that..." sense. At scale, AgKnow can derive intent from the cumulative behavior of multiple users who were exposed to similar information -- and as their scale increases further, they could slice even more finely to begin to distinguish path dependencies as well. I think we'll see an explosion of search alternatives in addition to advances in all three categories mentioned above, as dozens of companies large and small strive to crack the problem of too much information, not enough user intent.

Tuesday, March 07, 2006

What does Web 2.0 mean to me?

I was asked this question for a podcast at the recent Under the Radar conference where Zvents presented. You can test my memory by hunting up the actual podcast, but I am pretty sure this is the same answer I gave:

It means two things -- one technological, and one social.

Socially, it's the return of hope and enthusiasm to Silicon Valley. People may mock "bubble 2.0" and naysayers can always point to some particular point of excess, but the reason we entrepreneurs are here is to change the world, and a lot of dreams lay dormant from 2001 to 2005. Those dreams are being knit into reality today, and many of us can see that we'll be able to make the world a very different, and hopefully better place, soon.

Technologically, it's about data as a platform (just like Tim O'Reilly said) and more importantly, it's about fast and simple integration of that data. Mashups may occasionally be dorky, but they demonstrate a level of interoperability that is simply astonishing when compared to 1999. We've all seen network effects accelerate change and growth to warp speed -- what we're seeing now is kind of Metcalfe's Law of web functionality, where the value of every interoperable web service is the square of the number of other services to which it can be connected.

Hmmm. I know I didn't say that one in the podcast. It may actually be worth its own post.
But, back to work. Big customer meeting tomorrow...

...Web 2.0 means the return of customers. Yee Hah!

Aggregate Knowlege launches at ETech

I may be busting the press embargo by 90 minutes or so, but I'm really excited about this one. e, Tomorrow at ETech, Paul Martino and Chris Law are taking the wraps off of Aggregate Knowledge, their fascinating new play on a recommendations web service.

Paul and Chris were formerly founders of Tribe Networks, and Paul (several lives ago) was a graph theory PhD student at Princeton. They've cracked the problem of providing user-specific recommendations across multiple data types in real time, thanks to some seriously fancy math under the hood. We've been using AgKnow at Zvents to power some of the "related events" content on our site (Disclosure: I am also an investor), and look forward to more customers joining their service so we can begin to benefit from further network-effect driven recommended content.

Most importantly in this mash-up world, it takes from minutes to a few hours to fully implement their service on your site. EAI is soooo dead.

I expect them to do very well in both the content recommendation and non-search navigation business. Rock the house, guys!

P.S. I previously mentioned these guys in November.

Technorati Tag: ETech

Monday, March 06, 2006

Wednesday, March 01, 2006

Ether *is* Keen 2.0: Old Wine, New Bottles

Over at TechCrunch, Mike just posted on Ether, a new company driven out of the pay-per-call innovators Ingenio. My immediate thought was, "is Mayfield funding this company?" because Mayfield is (in)famously trolling through old bubble business plans for ideas, and this sounds an awful lot like Keen.com.

Keen was funded by Benchmark around 1999. I first saw gigantic billboards on the Underground advertising Keen in 2000 when I was living in London, and I remember thinking, "Damn, that's a great idea," as I stared at them from the platform across the tracks. It may still be a great idea, but Keen was relegated to a fairly painful fate, morphing into a psychic hotline and not much else. And certainly nothing that could make back the bubble-zillions that got poured into it.

In poking around to check out the carcass of Keen, I discovered a most curious fact:

Ingenio (Ether) *is* Keen! Changed their name sometime in the last seven years. Look right down at the bottom: "Keen is a trademark of Ingenio, Inc."

What was old is new again...!

Monday, February 27, 2006

Feed reader creators: There is so much left to do

I have a confession to make. Here I am smack in the middle of Web 2.0, a founder of a site that pumps out more RSS than you can shake a stick at, and I don't like feed readers. In fact, until about two weeks ago, I had never really used one, save a single desultory experiment back in 2005.

(Gasp!)

Yes, I have been quite embarassed about this affliction. I must admit, I have hidden it from my peers.

"Hi, I'm Ethan, and I type in URLs by hand."

I felt so bad that I finally caved in a few weeks ago and submitted to Bloglines. At least I could get an OPML file out of it, I thought... since Dave Winer assures me that will be useful soon, and Kevin Burton swears it will help me find all sorts of cool stuff on the web. I heard rumors at the last TechCrunch BBQ that it even attracts girls, but I've already found one of them.

But I couldn't stand it. Sure, it's all my reading in one place, and the atom feeds look sort of like their source blogs, but it's all kind of... attenuated. Like sucking Coca-Cola through a long straw, so the fizz is gone. All the formatting is replaced with grey-on-white arial. The fascinating sidebar tidbits and blogrolls and, yes, ads all disappear. And the urgent boldface of posts as yet unread, marching down the left sidebar like dutiful ants...

I thought I was alone.

And then... I found another! There in the endless grey arial sat a wizened little scrap of text, from AVC, blog home of none other than Fred Wilson, who, if he isn't the patron saint of Web 2.0, is, at a minimum, merry Mercury in its Pantheon. And what sayeth Fred?

"I Prefer Browsing To Reading Feeds"

Hallelujah!

Me too, Fred! In fact, this may be the most important post you've ever written. It's supportive of an inclusive and exploratory, rather than exclusive and reductionist, view of the web. While it calls out feed readers explicitly, it also criticizes by implication all the other exclusionary, reductivist drivers in this fascinating new media landscape -- our tendency to read the same old blogs; our algorithmic focus on the same old hubs and authorities; our analytical adherence to the same old scales and dimensions, and most particularly, our (and MY) fear that somehow, by being human, I was wrong.

A quote from Stephen Pinker's book (yes, an actual paper book!) The Blank Slate:
"The belief that human tastes are reversible cultural preferences has led social planners to write off people's enjoyment of ornament, natural light, and human scale, and forced millions of people to live in drab cement boxes."
I love blogs. I love the conversation, and the chase, and the delight of discovery. I love the humanity of user-created content, and I don't want to aggregate and attenuate that away in some misguided attempt at rational efficiency.

Is BuzzMachine the same without seeing that giant press picture at the top of the site before reading it every day? It reminds you, the reader, of the massive machine that is MSM. Like the three tones of NBC News at dinnertime; like Pavlov's dog; I salivate before dining daily with Jeff Jarvis.

Give me that ornament, and human scale, that connection with the writer and their carefully tweaked site, and at the same time make my life easier - and you'll have a dedicated customer. But until then...

...this new/old human-based web needs to remember us humans.

Feed reader creators: There is so much left to do.

Great new maps from Ask.com

Ask announced that they're going to fight for search, and launched some cool new maps functionality. Much of what you see here is a quality re-implementation of Google Maps, but there are some cool new features - for instance, Tyler pointed out that after you enter a route, you can click a "play" button and Ask will walk you through the steps onscreen.

Yawn, you say. Google did most of the useful bits a year ago, and Yahoo more or less matched this last fall.

True.

What I find fascinating is that the leading edge of search is all happening at startups, and that the offerings of the big boys are increasingly undifferentiated.

Why isn't Google building Plazes on top of Maps? Why isn't Google building Zillow on top of Maps? Heck, why isn't Google building Zvents on top of Maps?

It's a very interesting time to be a search startup. It feels like the calm before the storm -- with an absolute explosion of functionality coming very soon.

Sunday, February 12, 2006

Classifieds are Transactional, Groups are Relational

Mike recently wrote a second review of the forthcoming Microsoft Expo (aka Fremont) on TechCrunch.

He said, "Expo is centered on the idea that people will trust others within a group, and so is allowing classifieds networks within groups."

I left a comment on the article, which I'll expand upon here.

The original concept of Tribe Networks was that classified advertising would work better within the trust circles of existing groups, or "Tribes". This focus - which wasn't something they publicly discussed - is why their investors included the Washington Post and Knight Ridder. While Tribe is still going, it's fair to say that this concept didn't work for them. Maybe Microsoft can do better, but I am not convinced.

There are two simple reasons why.

First, classifieds are inherently transactional, and long term relationships of trust simply aren't necessary. Each of us has a sphere of activities, interests, friends, and so forth -- and when a buyer and a seller get together to transact a piece of merchandise, none of that is relevant. I have both bought and sold cars via classified ads, and what mattered to me wasn't that the other guy was nice, or trustworthy, or that we had anything larger in common; what mattered was that his cash or his car was as represented, and I came to the exchange with one, and walked away with the other.

There are very rare exceptions to this dictum - just yesterday, my friend Niranjan regaled me with the story of his purchasing an absolutely mint 1964 Volvo P1800 as driven by "The Saint." He still visits the previous (and original) owners of the car once a year to show it to them, and they sold it to him at a below-market price on the basis of his winning personality and love for the car. But exceptions like these prove the rule. Ask yourself - how many goods have I purchased or sold through classified ads; how much did I really care to know about those sellers/buyers, and how much would I be willing to change the mechanics of the transaction - e.g. paying higher or taking a lower price, waiting longer for an item to turn up within my limited network, etc., just to take advantage of that fuzzy feel-good surround?

The second problem with this concept is that people are wildly inconsistent. The certified CPA is cheating on his wife. The marriage counselor that they are seeing is an asshole in business dealings. The ultra-reliable mechanic who fixes their car with fantastic attention to detail can't keep his personal finances in order. Within different contexts, people's behavior can be so different as to be contradictory, and therefore, all the social things you think you know about someone from a groups-like exercise such as this, may disinform as much as they inform.

I'll be very curious to watch this develop, but I don't see any magical secret sauce yet.

Monday, January 23, 2006

Google Maps ego-surf: Everything is higher resolution at the Googleplex!!

Google Maps updated their satellite imagery, offering higher resolution in many places. Of course I had to check out my house - an increasingly less obscure form of ego-surfing. Since I live only a mile or two from Google, I scrolled over there to check out the uniquely weird/cool architecture of the Googleplex. Surprise! The world suddenly gets sharper, more crisp, and clearly resolved, right where the landfill ends and Google begins, even if the roads don't quite match up.

It's kind of cute -- sort of like those airbrushed graduation pictures we all had at age 18. If I were organizing the world's information, I'd probably do the same thing.

Q: Guys, when are you going to stop being coy and make 'Googleplex' resolve in the same way that 'Buckingham Palace' does?

User-Created Content: Market Goods, Cavemen, Schadenfreude

Ethan Prater at Jigsaw dropped me a note after my last post on user-created content and encouraged me to push my thinking a bit further. Specifically, he asked how I'd think of the content that users place into Jigsaw's contacts marketplace. As I understand Jigsaw's business, there are two different categories of information and value that they're getting from their users:

1) Contact information

2) Metadata & feedback about contact information

The first case is a classic "selfish self-entry" good. Either I am entering my contacts into Jigsaw in order to make cash money ('Buy, Sell, and Trade Business Contacts" is their tagline) or I am entering my contacts into Jigsaw in order to better manage them, and to derive some business development value ("trade your contacts") from them.

Metadata about contact information is a bit trickier, and forgive me, Ethan P., if I divert from your actual business model a bit to expound upon it. One of the other taglines on the Jigsaw front page is, "Accurate Data: Because Members Build and Clean the Database." That 'cleaning' bit is what I would refer to as metadata -- meaning, feedback that this is a bogus contact, an outdated contact, or a useless contact - I imagine someone writing in, "this guy may be the VP of Spending Money on Large Humming Boxes, but he takes 18 months to make a decision."

This is exactly the same kind of user-created content that steers the ship at Craigslist -- users can flag any post as miscategorized, spam, etc. -- and that eBay relies upon, with their famous buyer- and seller-rating system. It's the same kind of content that Amazon Reviews runs upon - the guy on the street giving good directions, which he has no particular reason to give.

It's difficult to identify why, exactly, users do this - I'd say that it falls at the delightful juncture of psyche where human beings depart from the rational choice models and start acting like, well, humans. Depending on what you belive, it might be called schadenfreude content, or perhaps caveman content.

The person who discovers that a Jigsaw contact is bogus (or, on Craigslist, that the posting is inappropriate in some way) is very unlikely to be fooled into wasting their own time on that content / contact again -- they've discovered that it's bad, and they'll move on. From a selfish perspective, throwing even another mouse click after that bad content in the form of punching a simple bogosity button only increases their sunk cost. So why do it? Is it because we instinctively recognize that on the other side of the oft-referenced "tragedy of the commons" lies a triumph of the commons, where lots of feedback snippets from lots of people can actually create broadly beneficial value? Or are we simply mad enough at having been misled -- whether it's an abstract anger at a bum steer in a piece of anonymous information, or a very directed annoyance at some scammer who's trying to take our money -- that we strike back through negative review?

I don't know.

In any case, we act; we vote; we warn the other grubby apes that here lay sour berries, here lurk leopards, and we suggest that the tribe move on.

Here is what I would posit: That the urge toward fairness and truth is so strong that it overpowers economic gain - and economic gain can actually get in the way of this impulse, by making people question their own instincts. So without knowing exactly how Jigsaw is managing their user-driven data-cleaning feedback system, I'd strongly recommend that they pay their users for this act in status and righteousness, not money. Compensating them for their contacts is an entirely separate transaction; monitoring the data quality is a moral crusade that I think we all are deeply primed to engage in, regardless of what the dystopian economists tell us about our amoral motivations.

Next post: data exhaust, perhaps the only category of user-created content that *might* qualify for the soulless rubric 'user generated.'

Monday, January 09, 2006

User-Created Content: Selfish, Social, Selfless

I continue to be fascinated by everyone's determination to wrap user-created content into their new web services, despite their flamboyant display of ignorance as to *why* users actually create content. Here are three distinct models that can help you think about why users create:

1) Selfish UCC. Delicious bookmarks and Flickr photos. I primarily bookmark and take pictures for myself -- because I want to remember, or because I like something. I use these web-based apps instead of client-side equivalents because I personally get more value out of them. While the social benefit is apparent, I view that social benefit strictly through a selfish lens -- instead of emailing pictures to my Mom, I can just point her at Flickr. I encourage other people to use these sites for selfish reasons as well -- it makes it easier for me to keep track of their UCC. Another great example of selfish UCC is every auction on eBay. Those sellers are not putting those auctions up there to help anyone but themselves - any general benefit is strictly a secondary effect. It's not necessary for any individual item of UCC to relate to any other item - a picture or a bookmark has its own stand-alone value.

2) Social UCC. Blogs, Slashdot comments, and Yahoo Groups are great examples. People create content in order to express a point of view within a larger conversational context. There is usually an implied audience and an implied reference in most of this type of content. Why do people create conversations? It's somewhere between selfish and selfless - call it social. We want to take part in something larger than just ourselves, but we expect to benefit as well. In this case, personal and general benefit are deeply intertwined, and each item of UCC relates to other items of UCC.

3) Selfless UCC. This is the one that leads most everyone astray. The single great example of selfless UCC is Wikipedia. If I am an expert in subject X, I already know all about it. I do not benefit personally from writing down that information and sharing it with the world -- I am primarily doing it so that you, the reader, can benefit; or out of some sense of "greater good" and contribution to something shared and larger than just myself. It's a very noble aspiration that drives this kind of UCC, and it's not particuarly surprising that this makes it the most rare. Implied for this sort of content is a weak sense of authorship -- if I am personally attached to the content, it begins to look much more like a conversation.

The obvious non-UCC example of this strong authorship point is newspaper reporters vs. columnists - a column is a conversation, whereas a byline is subsumed (largely) by the headline and content of the article.

There are particular issues inherent to each different type of UCC:

1) For selfish UCC, the issue is relevance and commensurability. If I take notes on something for my own purposes, there's no reason for me to care whether it accords with anyone else's notation in any sort of commensurable way. Delicious tries to address this problem by suggesting tags that other people used on an item; this drives convergence on certain terms, making their programmatic understanding of what that item is more robust.

2) For social UCC, the issue is personal opinion over-riding fact. Every conversation implies ego and point of view, and in reading a conversational thread, one has to constantly be asking "what was their motivation in saying that". Bias is inherent to the medium.

3) For selfless UCC, the issue is quality. As people are endlessly pointing out, there is a lot of crap in Wikipedia. You may not agree with someone's opinion on Slashdot, and you may not find my bookmarks in Delicious useful, but there are strong incentives in both contexts for me to at least try to make my content accurate. In the selfless context, well, the incentives aren't just as strong.

This suggests obvious remedies in product design:

1) For selfish UCC, take note of Delicious' example and try to drive general relevance without interrupting the selfishness of your users. Delicious prompts me with terms others have used, making my selfish life easier while making relevance of my actions more general.

2) For social UCC, create programmatic mechanisms (like polls or votes or star ratings) that allow people to quantify their point of view on some abstract scale. This will benefit the user (by allowing them to say simply, "I am *very* Republican," for instance) and benefit everyone else -- who can then say, "He's *very* Republican, and I can easily assess where he's coming from."

3) For Selfless UCC, use human editors to manage quality, and try as much as possible to create "superuser" editors to do this for you. The cost-benefit ratio is outstanding, so get over the stone-ageness of it and go make money.

If you apply this model to a lot of the "Web 2.0" websites out there, it readily explains why there are such issues with the user-created content on most of them. UCC is a two-edged sword, and must be used well to avoid injury.

Note on terminology: I deeply dislike the term "User-Generated Content" (UGC). People don't generate (a mechanistic or animalistic term), they create. Creation is a deeply human act, and we should celebrate it.

Update: Find more at the follow-up post "Market Goods, Cavemen, Schadenfreude"