Monday, October 27, 2008

Breaking News: Obama is a socialist, Ethan still voting for him

There's a tempest in a teapot brewing over in right-wing land over this quote from Obama on a 2001 radio talk show:
"One of the I think the tragedies of the Civil Rights movement was because the Civil Rights movement became so court focused I think that there was a tendency to lose track of the political and community organizing and activities on the ground that are able to put together the actual coalitions of power through which you bring about redistributive change and in some ways we still suffer from that."
If you think that the word 'redistributive' means he's a socialist, well, sure, game over, he's a socialist. But then you probably think FDR was a socialist, too, and I think that he and Ronald Reagan were the two greatest presidents of the last century.

The simple fact is that all of society benefits when there's a floor at the bottom and no ceiling at the top. FDR built the floor, and fifty years later, Reagan fixed the ceiling problem. Now, 25 years later, it's time to repair the floor, and in 20 or 30 years, some new Reagan can come along and fix the ceiling which by then will no doubt need repair.

I'd have voted for both FDR if I were alive and Reagan if I were of age, and I'll be voting for Obama and no doubt for Reagan's future successor to come. Management is mostly about pragmatism, and oversight of management is mostly about choosing the right guy for the problem. FDR, Reagan, and Obama are all that guy, and I'm looking forward to contributing my small voice to the process of choice in a week's time.

Wednesday, October 15, 2008

Portrait of a market in distress

If you had any doubt that we live in interesting times, check out this amazing stock chart:
schizo_nikkei
That's Japan's Nikkei 225 average over the past 5 trading days. It looks like it was drawn by a five-year-old who doesn't understand the concept; since when are stock charts made up of straight lines and gaps?

Even more interesting is that this is an *average* of 225 stocks. Isn't diversity supposed to even out fluctuations? The entire world financial system is now learning the painful lesson discovered by Long-Term Capital Management in 1998 -- that in edge and corner cases, entire asset classes, and multiple asset classes, can unexpectedly be deeply correlated, with the result being severe dislocation. Condos in South Florida and the Icelandic Krona -- who knew?

Everyone who hasn't yet read Nassim Taleb's _Black Swan_, get busy. It is well on its way to being book of the decade.

Tuesday, October 14, 2008

Goodbye, Hedgies!

"For the $2 trillion hedge fund industry, a long-feared shakeout is at hand. Some analysts say one out of every 10 funds could fold."

Seriously? News flash, NYT: When an illiquid asset class that grew on excess leverage and irrational hopes of profit collapses, you'll be lucky if one out of 10 survives.

Ask me how I know.

Fortunately, there are useful things for all those unemployed quants and asset managers to do.

Sunday, October 12, 2008

MBA and law students, please do something useful

This is probably the most depressing thing that I've read all week:

For students who set their sights on Wall Street during the boom years, the end has come just as they are getting ready to join the party... But even as the markets spiraled downward, business and finance students at top universities said they were not panicked about their futures and were confident that the financial markets would recover. For the young achievers drawn to finance, expectations die hard. [Kenton] Murray, [a senior at Princeton,] described the mood at Princeton as cautiously optimistic. No one I’ve talked to is worried about moving back home yet,” he said. “But everyone I know is studying for the LSATs right now, people who a month ago had no intention of ever going to law school.”

Jian Yang, 25, who is in his second year at the University of Chicago's graduate business school...expects to graduate with $200,000 in student debt.
Ye Gods.

Part of the reason that we got into this mess is that too many bright people are doing relatively useless things like investment banking and lawyering. Another key reason that we got into this mess is that once any entity -- from an individual person to the Federal Government -- gets themself under a massive pile of debt, their options are highly constrained. Prison may be more crushing than debt service; but it may not, because in prison, you don't have to spend your sentence productively.

To all the MBA students out there -- please don't double down on this historic mis-allocation of resources. If you are quantitative enough to consider finance, look into data mining and machine learning for product and marketing design and refinement. Translation: search relevance and ad targeting; but these are only the tip of the iceberg. Soon these disciplines will spread, because they add real value both to consumers and to merchants. Operations research for resource allocation is another huge area. The world is undergoing another historic transition now, which is toward connected, data-driven marketplace of real stuff -- not the fake packaged stuff that has been driving Wall Street for the past seven years. You won't ever get a $500K bonus working in this field, but you'll make a good salary, you'll add to the productive economy, and if you join (or start) the right company, you'll get rich beyond Wall Street dreams of avarice -- and deservedly so.

To all the law students out there -- if you are articulate and nuanced enough to consider law, how about a management or leadership career? The world needs more thoughtful communicators and accomplished managers. Running a team is a lot harder than running a trial, but much more rewarding. Understanding and influencing the people who work for you and who struggle to work together is a lot more interesting than navigating a jury selection pool. Negotiating a business partnership is a lot more fulfilling than a litigation settlement. I see the insides of hundreds of organizations that we partner with, and good management and leadership is all too rare. You too can make an enormous difference to the productive economy by refocusing your talents elsewhere. Are we electing Barack Obama president because he's a good negotiator? Or because he's a great leader, communicator, and manager? You know the answer. Be that answer.

And to all of you - please, please be careful about getting yourself buried under a mountain of debt, such that you end up having to work at a career that you may hate, in an industry that may not really provide anyone lasting value, simply because you are crushed by your debt service. I'm sorry, but if it takes $200K in debt, then whatever your parents told you about getting into that 'top school' is probably wrong. The ROI to your life is negative. I flunked out of one top school and left another before completing my PhD, and I was neither struck by lightning nor am I begging on a street corner. If you're truly smart, talented, motivated, and capable, you'll do fine.

There's a lot of broken economy to fix. Let's get to work.

Monday, October 06, 2008

So where did all that housing money go? You already spent it

The size of the numbers is staggering -- $700 billion here, $900 billion there -- and a lot of people are wondering how it's possible that seemingly every bank and every homeowner is suddenly broke.

There have been clear signs for a while that something was amiss in the housing and consumer consumption market, and one great indicator is Mortgage Equity Withdrawal, or MEW.

This is, quite simply, the net amount of cash that people pulled out of their houses to spend on Other Stuff -- either by a home equity loan or, in the case of retirement, selling an expensive house and buying a cheaper one. The financial blog Calculated Risk reports, buried in an article about the collapse of MEW, that U.S. consumers pulled $2.7 trillion out of of their houses during the five years from 2004 through 2008:
Equity extraction was close to $700 billion per year in 2004, 2005 and 2006, before declining to $471 billion last year and will probably be less than $100 billion in 2008.
While some of that might have been spent on the houses themselves, in the sensible form of new bathrooms or additions, a lot of it was spent on questionable upgrades like granite countertops and flat-out expenditures like flat screen TVs, Hummers, and vacations to Fiji. So the reason that all those Wall Street banks are going bust? Because they thought that you -- or your neighbors -- were actually going to pay back the inflated mortgages that backed up that giant sucking sound of the home ATM in overdrive.

For those of you who protest (as I do) your innocence, realize this simple fact: Even if you didn't do such an extraction yourself, whatever business you are in benefited from the huge increase in consumer spending that this MEW generated over the past four years. Retail, manufacturing, advertising, financial services, construction... we all lived a little or a lot better.

And now, not only is it gone, but we've got to pay it back in the form of systemic collapse, higher taxes, or inflation. None of us were as wealthy as we liked to pretend over the past four years, and now we're all going to be a lot poorer than we're ready to admit.

Tuesday, September 30, 2008

Zvents raises $24M for local & mobile search

It has been pointed out to me that for a blog ostensibly about things other than Zvents, there's been hardly anything but Zvents news here of late.

Separately, it has been pointed out to me that for a life that is ostensibly about other things in addition to Zvents, I've been doing precious little but work of late.

To you my blog readers let me say that this is the last Zvents post for quite a while; and to you my friends and family let me say... hang in there, it can't stay this crazy for ever.

That being said:
Zvents raised $24M in funding from Nokia Growth Partners, AT&T, Navteq, and our existing venture investors Vantage Point and Red Rock. We will use this money to make our local search service for our media network even more awesome; advance our very promising new local ad model; and as a warm, fuzzy security blanket against the slings and arrows of outrageous fortune.

Lots of people are writing about our round, including VentureBeat and Paid Content, as well as local gurus Pete Krasilovsky of Kelsey and Greg Sterling.

AT&T even issued their own press release about their strategic investment in Zvents.

How did we manage to raise all that money in tough times for both VCs and startups? Is there a pony in there somewhere? Forbes lays out a nice explanation.

And now... back to work building the future of local search.

Sunday, September 07, 2008

The Google Alphabet: 23% of top search terms are local

Brady Forrest has a nice post over at O'Reilly Radar about the 'Google Alphabet,' the top suggested search terms for each letter. He makes a bunch of interesting historical observations, but through my local lens I noticed something different:

23% of top search terms are local.

The list:
Craigslist
Ikea
John Lewis (a UK retailer)
Sears
Target
Zipcodes
6/26=23%. You could argue that 'Kelly Blue Book' is a 7th, but you could also argue that there are some ecommerce implications to some of the retailer name searches, so we'll call it a wash.

Local search is very big business.

Update: I posted this to my blog and checked out how it looked, and lo and behold, the very first local event for today in the SF Bay Area displaying on my Zvents left sidebar widget was 'Target Free Sundays' at the Asian Art Museum. Apparently retailer Target is sponsoring Sunday museum admissions as a local marketing tool. Yet another example of local merchants using events to promote their businesses.

Tuesday, August 26, 2008

Zvents news: Too much to ignore!

I've got a list of interesting topics right here on my desk, on a couple of increasingly tattered Post-it notes.

There's the 'Poker Analogy' post for machine learning-driven processing of log data.
There's the 'Match.com Analogy' post for how search is a collaborative process.
There's the 'Spice Rack' post on building search relevance.
And there's the 'Smart Delivery of Dumb Objects' post about ad targeting.

In addition to these four fascinating posts (trust me ;-) I've got at least two really cool related product ideas moldering on that same little collection of decaying desktop notes.

So what's keeping me from fulfilling my bloggy destiny? As usual, Lots Is Happening (I almost typed Happyning... can your fingers create malaprops?) at Zvents.

Today's particular news is that as our marketing VP Paul O'Brien has noted at length, we're launching dozens of campus guides for MTVu, powered by Zvents local search. It's gotten a ton of press, written by people less busy than I -- check it out!

Substantive posting will resume in September. I think.

Saturday, August 16, 2008

14-year-old real estate moguls and the end of bubbles

Home in Detroit sells for $1.

Writes the Detroit News, "...some abandoned homes in Detroit sell for $100; vacant lots can be purchased for $300."

"My 14-year-old son could buy a block of Detroit property," said Ann Laciura, senior servicing specialist for the Bearing Group.

Whenever someone tells you that it's impossible for an asset class to go to zero, they're either dumb, or hoping that you are. If the carrying costs and net obligations of any asset exceed the cashflow, it's worth zero or less than zero. We saw this a lot during 2001-2002, as companies with insane cost structures met grim fates, occasionally returning as shadows of their former selves, or reinvented as actual profitable businesses.

Businesses probably worth less than zero right now:
GM and Chrysler
Citibank and Washington Mutual

Silicon Valley got crushed in the last asset bubble deflation. This time around will be relatively kind to us, but there will still be pain.

Winter is coming.

Monday, August 04, 2008

Dancing with Elephants

I try to keep the Zvents news to a minimum here, but I'm very excited to note that we're now powering local event search for the new MSN CityGuides. We've been powering major media sites since our first partner, the San Jose Mercury News, launched in July of 2006 -- over two years now, and it's seemingly gone in the blink of an eye. Even versus our now-hundreds of local media partners, big sites like MSN are a whole new level of scale. It's not just their userbase - which numbers in the millions per month - but as one of the biggest technology companies on the planet, their expectations around SLAs, integration, product and project processes, etc., require us to really step up our game. I'm excited that we're proving that we can play in this league. We may still be a little mouse of a company, but we're dancing with elephants ;-)

elephant+mouse

Some coverage of the announcement -- thanks, guys!

Greg Sterling before
Greg Sterling after (No, Greg, you didn't blow the embargo!)
SeachEngineWatch
LostRemote
Adotas Interactive Advertising
Local Biz Bits

Friday, July 25, 2008

Search relevance and the data-driven web

My frustration with CNET's informative but misguided article on Microsoft's BrowseRank has only redoubled with the reading of some other stuff like Andy Ha's breathless and utterly silly piece on VentureBeat -- I refuse to link to it, but you can go look it up if you're curious. Example phrases (good for searching on, too ;-):

"could really shake up the search landscape"
"it would upend the way companies get attention online."
"it might be able to reverse MSN Search’s slide towards irrelevance."
"a new kind of search"

Aargh.


I decided to channel my powers for good instead of evil, and rather than trolling in the VentureBeat comments, I decided to trawl through the Onotech archives and pull together many pieces I've written about search relevance into this one handy post that I can refer to in the future.

Note that this is not just about search. Search is just one facet to the golden rectangle of the data-driven web -- search, ad targeting, discovery, and analytics -- that makes every modern web company rivers of cash. All of these product mechanisms share underlying concepts and technologies, and I've blogged about them (and the infrastructure that makes them possible) many times over the years. Here, in time order, are some of the better posts on the concepts behind the data-driven web.

Another time, I'll pull together a second summary blog post on infrastructure thoughts - because there are some really cool software technologies that make this all possible.

This is also probably a great time to show off my birthday present from Somer Simpson:
machine learning power to the data poster
Schweet. I may make T-shirts -- email me if you want one ;-)

OK, here we go:

Click data drives not just search, but all advertising

The impact of monetization on search result ranking

Query type differentiation and machine learning

All search is social search because people determine value

On relevance/selection mechanisms, and types of voting

The four fundamental ways to determine ranking

Updated:
How to better understand user intent

CNET on BrowseRank: An informative article with a nonsensical premise

It's great to see a well-written, informative article, "Microsoft tries to one-up Google PageRank," about an innovation in search ranking, published at a major tech news outlet like CNET. Stephen Shankland's piece on Microsoft's BrowseRank is definitely all that. It thoughtfully discusses the concept that people's click behavior is a very powerful (and different) voting popularity mechanism than the link graph of PageRank to assess web page search relevance. All good.

However it's maddening to see yet another naive or deliberately misleading article on an innovation in search ranking, that perpetuates persistent misunderstanding of how search works and what makes search better. Maybe it's just a standard media trope to essentialize any topic to the point of parody, but I'm so tired of seeing pieces that fetishize "The Algorithm" as some singular magical trump card by which search is won and lost. Combining scientific models to produce a ranking function is difficult, obscure, and incredibly important, so to some extent I can understand why the media keeps writing stories focused on this. But it's sort of like watching a Saturn V take off for the moon, and turning confidently to your neighbor and saying, "that thing takes off because NASA figured out The Engine... I hear the Russians are working on something better than "The Engine."

Please. Great search is made up of a couple major areas of competence:
* Scaled aggregation of content. It doesn't matter how good your matching might be if you don't have what the user wants in your cupboard of goodies.
* Scaled user voting behavior to assess value. Be this the ability to crawl and assess hyperlinks, access to and the ability to assess user clicks, or one of (as Udi Manber rightly says) hundreds of other variously valuable and tractable methods, you need access to behavior metadata, crystallized in one form or another
* A scientific process and platform by which you can run many experiments to fine-tune the value of various voting behavior signals.
* A technological platform to rapidly and cost-effectively perform this mind-boggling level of computation, faster than the answers and the questions are evolving on a global scale.
* A bunch of really great scalability engineers to build that platform
* A bunch of really great search scientists to conceive, build, and test models on a continuous basis
* Oh, and a very effective monetization effort to pay for all of this incredibly expensive infrastructure, people, and time cost.

It sounds a lot more like General Motors circa 1955 than it does "genius in a garage cooking up the next great thing," doesn't it? Perhaps that's why the media always falls into this trap - the brilliant loner or breakthrough insight that changes the world is just such a powerful narrative hook, whereas a discipline, competence, process story is just kind of... boring.

The reality of world-class search today is that it's big, complicated, and multi-faceted. It has emerged into a discipline of technology all its own, and advances will tend to be subtle and hard to explain. Remember that whenever you read the next excited story about the Next Great Algorithm.

Update: I've written a new post that channels my grumpiness to productive ends -- a few more thoughts on the data-driven web and an index to my various posts on search relevance. Reporters, please read them all... twice!

Saturday, July 19, 2008

The tech bubble's toxic legacy

An article on GigaOm caught my eye: "With Exits Barred, VCs Keep Investments Flat". In the piece, Stacey Higganbotham relates the reassuring news to entrepreneurs that VC investment in early stage companies remains strong, and that later-stage deals are being done at a historic clip -- 318 in the second quarter alone. But then she drops this little tidbit:
If those companies don’t exit within the next two to three years, VCs will have to start selling at a loss or pushing firms into bankruptcy.

Huh?! What's wrong with this picture? Any 'late stage' deal likely means a C and possibly a B round -- so the company involved has been in business for 2 or 3 years already when it raises the round; and the round should last it for at least another 18 months.

The presumption here is that tech startups don't get to profitability.

This is the worst legacy of the tech bubble, when lots of permanently unprofitable companies were started, funded, and even IPOed. It's got nothing to do with the true legacy of Silicon Valley, where superstars of past eras -- Intel, Apple, Sun, PeopleSoft, Oracle -- became insanely profitable companies. And this presumption is blinding us to what's happening in the Valley today. It's the same mistake that made you not buy Google shares at $85 in 2005, and you shouldn't let it blind you again.

I personally know of at least three highly profitable startups that may go public in the next 12 months, and may not. All three of them have been in business for over five years, and they are throwing off not just revenue but cash profit at a very impressive rate. To those three, I can add at least a dozen that I'm pretty sure are highly profitable as well, I just don't know for certain. All of these companies are the types of startups that are raising the "late stage rounds" that Stacey seems to believe must lead to exit or bankruptcy, because there is no third path.

Every startup that I've built has been intended from Day 1 not just to be transformative to its market, but to be a real, profitable business, with real customers. That's the true legacy of Silicon Valley, and the sooner we get our heads past the failed abberation of 1999-2001, the better.

Wednesday, June 25, 2008

Doug Judd presents Hypertable at Structure '08

Liz Gannes at GigaOm has the story and the pictures. Hypertable will be going beta Real Soon Now, and we're very pleased at the press and traction that it's been getting. It's the featured article in the June 2008 Linux magazine (link soon when it's on the web) and Doug has a pretty packed speaking schedule this summer including OSCON and FOO Camp.

Even more exciting is that we've got Hypertable in production at Zvents, and we're starting to see some of the benefits of a really fast and scalable analytic platform for search relevance, analytics, and ad targeting.

It's not that often that you can do a cool thing, a good-for-the-world thing, and a smart business thing all at once. I'm pretty happy that our two-year adventure with Hypertable appears to be that rare intersection.

Monday, June 16, 2008

Jeff Weiner, CEO of Facebook?

It's pretty common for accomplished executive talent to join VCs in EIR positions -- with the 'E' being either 'Entrepreneur' or 'Executive' In Residence. But this may be the first time that I've seen someone join *two* VC firms in such a role simultaneously. The purpose of an EIR role is to help advance the agenda ( = fund returns) of the VC, while advancing the career of the exec. Hard to do that for two different VCs at once. So unless Weiner is such a rare talent that he got Accel and Greylock to agree to go halvsies on his potential future value creation, there must be some reason that that they're mutually interested in him.

The biggest thing that Accel and Greylock have going together is... Facebook. Now we all know that Mark Zuckerberg is firmly ensconced in the CEO role there. But is he really? A year later, Beacon is a failure. Jerry Yang has spent the last six months giving us all a lurid demonstration that even with fifteen years of seasoning, founding CEOs rarely are the right guys to manage public companies. Facebook has made no secret of its intentions to stay independent and go public. So... Jeff Weiner as CEO of Facebook? Don't be surprised.

Is Weiner a good fit for Facebook? It's a company with serious platform ambitions, which has been successfully recruiting senior technical talent from Google. It's a classic Web 2.0 company, scaled to the max. And it's a company that is poised to inherit the mantle of Yahoo as the biggest property of the social/personal web, as Yahoo continues to bleed away its birthright. Here's a snippet of Weiner's resume from the Accel release:
During his tenure, Weiner helped drive the Network’s Open and Social strategy and expansion of the company’s category-leading consumer web products... Prior to his Network role, Weiner was part of the Search leadership team that directed the acquisition and integration of Inktomi, AltaVista, FAST as well as the development of Yahoo! Search Technology. He helped launch the company's suite of social search products - including Yahoo! Answers - and led the acquisitions of del.icio.us and Flickr. He was also part of the leadership team formed to help revitalize the company's Search Marketing efforts.

Yeah, that experience might just apply to what Facebook is trying to do.

Kara Swisher has some thoughts on the implications of Weiner's departure from Yahoo, and she also wrote the magic 'Facebook' word before I did ;-)

Wednesday, May 28, 2008

It's all happening

I generally try to keep the Zvents posting light. This is not a long post about the company - it's just a quick note, somewhat of a milestone of awe. I've been so silent for so long because I've been utterly immersed, spun up in a tornado of business - for four months now, seemingly without end. It's been incredible. Exhausting. Exhilarating. I just did a quick count, and I've received 195 emails today - 97 of which came after 5pm. This past Memorial Day weekend, from 4pm on Friday through Monday at midnight, I personally did more business than many startups see in a month. At least six customer and partner calls, and endless email. This is not meant to brag. It's more of a sense of wonderment. When everything starts working, the net acceleration is just incredible.

I hope there's not a big wall out there just over the horizon, because we're moving at the speed of light.

Snapshot taken. We'll see in time which story it's telling.

Tuesday, April 15, 2008

AOL: The Network is the Content, the Network is the Ad

There are two big AOL announcements floating up TechMeme today. Most interesting is the news from Comscore that due to Platform A/Ad.com, AOL leads all major internet companies in reach, touching 9 out of 10 users on the internet.

The second is that AOL has acquired Sphere, Tony Conrad's blog recommendation engine, for about $25m. Since it launched three years ago, Sphere has done a great job of bringing its 'related commentary' widget to the content pages of very large sites like the New York Times, Washington Post, and Time magazine.

These two announcements are related, and I'm surprised that the usual commentators haven't picked up on this yet. The future of the internet is becoming increasingly clear -- it's advertorial, or a blend of commerce and content. In a huge number of cases, the content that users consume - presented either editorially, via search, or via a recommendation - is highly commercial in nature. If you care about gadgets, for instance, Engadget or Sony.com are both great landing sites for your online explorations. Cars? You may go to an enthusiast site, or Porsche.com.

Both content and advertising are best distributed via analytics- and algorithm-driven networks. Sphere is quite simply an algorithmic content redistribution network, and AOL's platform A is a (targeting) algorithm driven advertising network. These are different stripes on the same zebra.

Google, of course, has been experimenting with embedding content such as YouTube videos into its huge AdSense network; and there are a number of very interesting large-scale advertorial networks emerging such as Aggregate Knowledge's Pique. (Disclosure: I am an investor in AK). I will be very interested to see how AOL adds Sphere in to its market-leading network, and to watch how this emergent trend evolves.

Monday, March 17, 2008

Alan Greenspan is the Robert McNamara of economics

I was reading Krugman's latest piece on the ongoing meltdown of the capital markets, and he offered this quote:
Meanwhile, false beliefs in the political arena — the belief of Alan Greenspan and his friends in the Bush administration that the market is always right and regulation always a bad thing — led Washington to ignore the warning signs. By the way, Mr. Greenspan is still at it: accepting no blame, he continues to insist that “market flexibility and open competition” are the “most reliable safeguards against cumulative economic failure.”

It hit me: Greenspan, in practice and in retrospect, reminds me of nothing so much as Vietnam-era defense secretary Robert McNamara, the proponent of "the domino theory," and his still-ongoing and increasingly tragic efforts to shift all blame from himself long after the disastrous damage is done.

mcnamara_greenspan

Greenspan's trumpeting of the ludicrous notion of efficient markets and support for the demise of market regulation will one day been recognized as equivalent folly.

The real tragedy here is that both of these men are geniuses in every sense of the word. It's impossible to watch The Fog of War and not come away impressed with McNamara. I highly recommend that film to anyone who has ever suffered even a little from hubris or ego. McNamara is smarter than you, better educated than you, more subtle and strategic than you, and he still blew it about as badly as a human being in power ever has done. Why? I can only guess, ideology.

Which is exactly how Greenspan has sent our country into our current tailspin.

Coming to a theater near you in 2028, _The Fog of Commerce_.

Hopefully, coming soon to a government near you, competence unblinded by nonsense.

Monday, March 10, 2008

NYTimes: It's all about the data, dummy!

Great article in the NYTimes on ad targeting and data scale. This is probably the first intelligent article in the mainstream media that I can remember on this topic - explaining that the combination of on-site and off-site behavioral data by consumers is what's driving the Internet today. An extension to the claims in this piece is that the same data is used to drive *products* (i.e. search relevance) as well as ad targeting; but I'll wait for their follow-up piece to close that gap.

Worth noting: The NYTimes has integrated "blogs" and their conversational style allows Louise Story to add some useful and interesting information to the more formal article. Check out her piece at the "Bits" blog for more data and further insight.

Some key quotes:
The rich troves of data at the fingertips of the biggest Internet companies are also creating a new kind of digital divide within the industry. Traditional media companies, which collect far less data about visitors to their sites, are increasingly at a disadvantage when they compete for ad dollars.

The major television networks and magazine and newspaper companies “aren’t even in the same league,” said Linda Abraham, an executive vice president at comScore. “They can’t really play in this sandbox.”...

Web companies once could monitor the actions of consumers only on their own sites. But over the last couple of years, the Internet giants have spread their reach by acting as intermediaries that place ads on thousands of Web sites, and now can follow people’s activities on far more sites...

“So many of the deals are really about data,” said David Verklin...

Some advertising executives say media companies will have little choice but to outsource their ad sales to companies like Microsoft and Yahoo to benefit from their data. The Web companies may prove they can use their algorithms and consumer information to better select which ads for visitors better than media companies can.

“I think a lot of publishers are going to find they don’t have enough data,” said David W. Kenny, chief executive of Digitas, a digital advertising agency in the Publicis Groupe. “There’s only going to be a handful of big players who can manage the data.”...

Even with all the data Web companies have, they are finding ways to obtain more. The giant Internet portals have been buying ad-delivery companies like DoubleClick and Atlas, which have stockpiles of information. Atlas, for example, delivers 6 billion ads every day. The comScore figures do not capture such data.
I am pretty confident that there is at least one zero missing from every number the Times quotes in this article. This is happening today at a monumental scale, and will only get larger as time goes on.

Data is the new lock-in and sustainable competitive advantage.

So how do you collect, store, and analyze all that data? That's why we built Hypertable...

Monday, March 03, 2008

Google uses prior searches and spelling correction to match AdWords

I imagine that one of the problems that Google has is matching existing ad inventory against all the wonderful random crap that searchers type in - with many searches being really hard to monetize by "traditional" means. Thus the many experiments they run. I wonder what the ROI on this particular example is?

This is a random discovery which came about as I was looking up some old co-workers. First names have been changed to protect the innocent.

First I searched for my old co-worker "Ed" Biasi:
ed_biasi

Note that there are no ads. Name searches are tough to monetize.

Then I searched for my old co-worker "Joe" Satchel:
joe_satchel

Note the ad match - It's for a Biasia satchel - which is apparently some sort of $200 name-brand purse:

Not only is Google doing spelling correction on my Biasi search, they're remembering the immediate prior search term to match the ad. Apparently they only remember one prior search - when I did the same search twice (either Biasi or Satchel) the ad disappeared.

Fascinating.