Google Scalability Conference Talks

Greg Linden has an excellent summary of the Google Scalability Conference with a number of links to the talks. I've seen a few, and highly recommend the first keynote titled MapReduce, BigTable, and Other Distributed System Abstractions for Handling Large Datasets by Jeff Dean. Greg's full post is available HERE. Kyle Cordes also does a good job of summarizing YouTube's scaling experiences HERE. You can see the full YouTube scaling talk HERE.

Human Computation by Luis Von Ahn

I absolutely love StumpleUpon's video recommendation plugin. I now get great video's to watch highly tailored to my taste. Here is a lecture I ran into by Luis Von Ahn, a CS professor at CMU, lecturing on his approach to developing clever and addictive games to leverage human computational power to solve important problems. For example, one of the games allows him to build a database of metatags for images on the web. He argues he can metatag effectively all image content in Google's image search with his user population in a very short period of time. This lecture left me thinking social networking sites are significantly under utilizing their user populations.

How not to find great employees

If you're in a startup and are trying to attract great employees, whatever you do, do NOT post a job ad that looks like this:

CompanyX is a tiny technology startup in Seattle. We are working on some amazingly cool technology, and we face the ongoing challenge of matching people with other people (contextually and behaviorally) and scaling a complex Web application.

We're looking for scary smart people who are willing and excited to jump into a startup, work hard, have fun, make their mark, and work for part equity and part cash. We want to make a lot of money, but you should know this is not your Mommy's station wagon rolling down the freeway at 60mph. This is a pimped out Hummer on extremely rough terrain with life-threatening hazards around every corner. And we're going too fast. If that actually sounds FUN to you, then give us a yell and tell us exactly why we should have a conversation.

Trail Critters and Other Biking War Stories

So after non stop mountain biking for the past week up here at Whistler, I can say with confidence, bears rule the land. A few days back, I was flying through the woods on a cross country ride; I was descending some tight technical single track and ready to shoot out onto a gravel road when I noticed a rather large black bear mulling around in the trail exit onto the road. I stopped as fast as I could, smiled and waved at the bear, and proceeded to back out slowly. He watched me until I was almost out of sight. I then turned my back to him and zipped away -- heart racing of course.

Yesterday, there was a big fella grazing just a hundred yards or so at the top of the lift. I also met a French guy who told me all about his best friends grizzly encounter that had recently happened. Apparently his friend and another friend were mountain biking not too far from here, and a grizzly attacked the other friend. The Frenchman's friend charged the bear to save the other guy, and smacked the bear with his bike. The bear was pushing toward him while he was holding it back with his bike. He then tried to break the bears nose by punching it - to no avail of course. The bear then swiped him with his paw, and apparently ripped deep into his jaw bone. He then passed out, and awoke over half an hour later. Apparently the grizzly left him after determining he was dead, or unworthy. He escaped with only 256 stitches.

This is what riders up here do whenever talk of the furry ones comes up - we all have our war stories. Today, there is a section right above lower Whistler downhill, where you turn a tight corner and launch in the air and down fast into a long open descent. As I was rounding the corner just a moment before committing, I noticed a huge black bear grazing only a few feet from the trail at the bottom. Fortunately I slammed on my brakes and was able to back track out. Another instant and I would have had to race by him praying all the way that I wouldn't blow out a tire or that he wouldn't give chase.

Another rider up here had a mountain lion incident on Tuesday by the mini-disc golf course - he got confronted, and the lion kept pushing toward him. Finally he hopped on his bike and tried to out run the lion. Fortunately for him, he was a downhill racer so he was pretty quick. Apparently the lion chased him for quite a while. The lion story brought me back to my own lion encounter, but I'll save that for another time.

Once this sort of stuff creeps into your mind, its very hard to get rid of the thoughts. The past few days, I've been scanning for critters like mad. Its actually an annoying distraction when there's enough obstacles immediately in your path to pay attention to. Today I was dropping off a steep ladder when I got completely spooked by what turned out to be a life threatening squirrel. Ridiculously, I did end up with quite a crash - fortunately for my ego, it was a tough and terribly slippery spot. Well at least I got to test out my helmet and armor. In any event, I've had enough furry encounters in the past week to last me the year.

The Continued Adventures of Mateo: Fidel Oaxaca

My good friend Matt, now known as Mateo, dropped his life as a Seattle software geek, packed up his trailer, and drove down to the lovely beaches of Zipolite, Oaxaca. He has set up a new life down there. Occasionally I post some of his letters, with his permission of course, to remind us all that there is a world out there away from mortgages and 9 to 5 gigs. Here are some snippets from the latest:

Hey man. Thanks so much for getting back, I really appreciate it. [...] I'm now working with two other Mexican guys starting yet another Mexican non-profit to apply for world bank money to do a deforestation project. This is more Manolo's baby, he worked at the world bank for a number of years and his program was canceled by Wolfowitz. Needless to say he is quite pleased that W got the boot. Anyway he has retained a ton of contacts; I'm helping him prepare English applications and he is helping me find international money for things I'm interested in. The coolest thing, is learning from him.

In Mexico you can't do anything without the support of a local heavyweight. The local heavyweight can be a business leader, a member of an old family, a particularly good fisherman, or even somebody who tells good jokes. It has nothing to do with money or explicit power, it has to do with their personal power, their charm, their charisma, who knows what... Some people have this power [and] people follow them[;] they just know how to talk to all different kinds of people. With them you have a project. Without them you don't, they just take your money and nod. Usually if you go to a small town there are one or two guys who are heavyweights and you will quickly learn who. But one in a hundred, you can take the guy anywhere and plop him down and hell if he can't get everyone behind him in 30 minutes... no matter if they have never seen or heard of him.

Manolo is one of those. He looks like Fidel Castro at 50 with [a] gray beard, rides an old horse everywhere, and is shitfaced by 10:30 every day, but he's an incredible community leader. We went for a business meeting last week in the early morning sober hours. We were all in our bathing suits. We drank cheap Nescafe and he smoked cigars that he rolled from a single tobacco leaf at the table. He gave me the names of about 30 different international organizations that fund the work I'm interested in with tips for writing proposals and the names of his personal contacts. Then Filemon came by to borrow his gun to kill his cat, which had developed a [terribly] incurable disease. Manolo produced a small firearm and dispatched the poor cat, after which he announced he needed to take a ceremonial bath to cleanse himself of the bad vibes from killing an innocent animal. But no matter; we would resume the meeting with Alexis and I at the table and him out on the patio naked in a tub of water under the [bougainvillea] and next to his horse, who lives with him in his house. His house is the most amazing place I've seen, a sprawling decaying concrete estate with 180 degree views of the coast that he built himself over the last 20 years.

[...]

Nothing new here really. Things have quieted down a lot, we are waiting for the rain to start. Best to Ameen and the kids. take care, hasta la proxima,

Matt

Book Review: Come Back to Afghanistan by Said Hyder Akbar

I just completed an amazing book called Come Back to Afghanistan by Said Hyder Akbar, a 17 year old teenager from California. For anyone interested in the current state of Afghanistan, I highly recommend this book.

Books like this, i.e. personal accounts of a foreign and politically relevant land, are typically written by seasoned news reporter type folks, or other adults who already have a solid world view. Witnessing Hyder, a youthful teen, struggle to make sense of this world, often through colorful pop culture references, is so unique. In addition, I've always been fascinated by the recent history of Afghanistan. I was similarly glued to Soldiers of God, by Robert Kaplan, a detailed account of his experiences with the Soviet fighting Mujahideen. Perhaps it is the vivid descriptions of these rugged mountain war zones, places like the Khyber Pass or the Hindu Kush, that get to me. I'm also always interested in understanding the forces of tribalism, especially in South Asia.

Anyway, back to the book, not sure exactly how it happened, but somehow Hyder was given a microphone and recorder by Susan Burton who ultimately produced a show for This American Life, the NPR radio show hosted by Ira Glass. You can listen to the complete This American Life episode HERE. This show kicked off much of what appears in more vivid detail in the book. You can also buy the book on Amazon HERE.

Here's a snippet from a Publisher's Weekly review on Come Back to Afghanistan:

Akbar's refreshingly unsentimental reminiscences of visiting his father's homeland as a teen make for an intriguing portrait of Afghanistan at a time of significant transition. On 9/11, Akbar, who was born in Peshawar in 1984 but grew up in the U.S., was living near Oakland, Calif., where his father ran a clothing store. After the attack, the elder Akbar, a descendant of an Afghan political family, returned to his country to take a job as President Hamid Karzai's chief spokesman and, later, as governor of Kunar, a rural province. The author visited his father for three successive summers, and the result is this account, a closeup view of the creation of the country's post-Taliban democratic government, told from a perspective that's impressively both insider and objective. Akbar reports on chats with cabinet ministers and warlords, and sketches the lay of the land, visiting both sumptuous Kabul palaces as well as bombed-out villages. His youth and curiosity send him on some dangerous adventures (he retraces a mountain route between Afghanistan and Pakistan used by fleeing members of al-Qaeda and the Taliban), and that youthful flavor also infuses the writing with a hip stream-of-consciousness that is by turns funny, insightful and, occasionally, breathtaking.

All in all, I give this book 4 out of 4 stars.

Big Whistler Air

Here's a great video of riders getting some big air. By the way, that reminds me, there's a popular trail up here called Dirt Merchant - there's a lovely spot where you launch over a creek that flows between the launch and landing. Its such a great feeling - a middle aged guy like me can feel like Evil Kenievel for a wonderful few moments. But I digress, back to these far more talented kids, enjoy:

Whistler Fun and Lots of Sun

We got up to Whistler yesterday morning and have been having a blast ever since. Yesterday I had a great time riding a bunch of cross country trails all around the Lost Lake area. There are some really fantastic trails that I've never found before. Lots of great wooden planks and roller coaster fun on Rhinoceros Furniture - my these Canadians sure know how to make some unbelievable trails. Poor Maya didn't make it out of the house today; alas her paws are really sore from our heavy riding yesterday; I really want to find some good running shoes for her.

Today was a hoot doing some downhill riding. I ran into an old friend randomly who I haven't seen in 15 years. We had a blast riding together. And the weather has been just stunning - perfectly sunny and around 75 degrees.

Google Keeps Tweaking Its Search Engine

A recent NY Times article that is lighting up the search blogosphere called Google Keeps Tweaking Its Search Engine by Saul Hansell provides some insights on how the Google ranking system works. For anyone that has ever spent some time tweaking ranking metrics, this is an article worth checking out. Here are some snippets:

“Someone brings a query that is broken to Amit, and he treasures it and cherishes it and tries to figure out how to fix the algorithm,” says Matt Cutts, one of Mr. Singhal’s officemates and the head of Google’s efforts to fight Web spam, the term for advertising-filled pages that somehow keep maneuvering to the top of search listings.

Some complaints involve simple flaws that need to be fixed right away. Recently, a search for “French Revolution” returned too many sites about the recent French presidential election campaign — in which candidates opined on various policy revolutions — rather than the ouster of King Louis XVI. A search-engine tweak gave more weight to pages with phrases like “French Revolution” rather than pages that simply had both words.

At other times, complaints highlight more complex problems.

...

Mr. Singhal introduced the freshness problem, explaining that simply changing formulas to display more new pages results in lower-quality searches much of the time. He then unveiled his team’s solution: a mathematical model that tries to determine when users want new information and when they don’t. (And yes, like all Google initiatives, it had a name: QDF, for “query deserves freshness.”)

...

THE QDF solution revolves around determining whether a topic is “hot.” If news sites or blog posts are actively writing about a topic, the model figures that it is one for which users are more likely to want current information. The model also examines Google’s own stream of billions of search queries, which Mr. Singhal believes is an even better monitor of global enthusiasm about a particular subject.

As an example, he points out what happens when cities suffer power failures. “When there is a blackout in New York, the first articles appear in 15 minutes; we get queries in two seconds,” he says.

Mr. Singhal says he tested QDF for a simple application: deciding whether to include a few news headlines among regular results when people do searches for topics with high QDF scores. Although Google already has a different system for including headlines on some search pages, QDF offered more sophisticated results, putting the headlines at the top of the page for some queries, and putting them in the middle or at the bottom for others.

Clinton tackles the Fox smirk

I recall this being in the news a while back, but I missed it then for some reason. Anyway, its entertaining to watch Clinton go for the jugular.

The Camp Cody Reunion Tour

Nayan, Maya and I had a blast camping out in the tree house and sauna up at Camp Cody this weekend. The weather was a bit schizophrenic, but we still had much fun. Uncle Timothy now has a row boat that Nayan was able to row in at the Camp Cody pond. Since the water was freezing, he was a little tentative about following papa on his morning dives in, but the row boat was perfect. We also got some great mountain biking in. Nayan had a blast on the wide interurban trail, but got a little spooked on the single track. I think next year will be a big turning point as far as father son mountain biking goes.

Other CC updates: with the arrival of the pond, has come some very loud frogs. All night long the endless "ributting" had all three of us, the dog included, combing through the night to figure out where the culprit was hiding. It turns out there are many many frogs. Alas, we fell asleep.

Some juggler friends of Islando were performing at the Bellingham Juggler's Festival. The opening act was just terrific. It was mad Bruce Lee style action fighting, followed by slow motion facial contortions, followed by more fighting, and then, what else, lots of juggling! What could be a better combo then Kung fu and juggling for a 5 year old?

No Nayan hasn't gotten a haircut, this photo is from last year. It was a bit rain-i-er this trip.

Here's a view of the sauna from the tree house. We placed one of the trapeze mats (our friends Islando and Uquoia run their own circus and practice under the tree house) on the benches inside and "camped." We slept with the door open, so Maya was off running frog searching sniffing missions all night. It wasn't the deepest sleep I've had, but pleasant nonetheless.

Beautiful Ballard Karma

It has been raining off and on all day. Ameen and Ananya are in England this week, so I have taken the week off to spend some time with Nayan. We decided to go see the new Shrek movie. We were standing in line in the cold and drizzle. There was a huge line up. A woman came out and said, "Are you here to see Spiderman?" I said "No, we're here to see Shrek. She said, "You know it's sold out, but today is your lucky day." She gave us two tickets. I reached for my wallet and tried to give her some money, but she insisted our movie was her treat. Apparently it was her son's birthday party and there were a couple of no shows. I absolutely love Ballard. The rest of the world may be getting darker and more gruff, but beautiful Ballard is still full of wonderful people like this woman who value simple goodness.

Linus on Microsoft's Patent Infringement Announcement

I ran across an interesting article on ZDNet called Experts say Microsoft's patent quest won't go far. In the article, the author describes how many companies instruct their engineers to not perform any active searches. This is definitely true. I have spent many hours with my patent lawyer, and this is an emphatic point she makes to me repeatedly. Here is Linus's response:

"There are several reasons why engineers should not read other people's patents, only their own. And it's not a 'hide your head in the sand' issue, it's a very practical issue of it being a waste of time," Torvalds said.

For one thing, developing technology without looking at patents lets a person honestly say they developed that technology independently, which helps show that the patent in question doesn't meet the requirement of a technology not being obvious, he said. And engineers aren't likely to comprehend patents in the first place: "Unless you have a patent attorney at your side, patent language usually makes no sense."

He derided Microsoft for spreading FUD (fear, uncertainty and doubt) rather than tackling the issue forthrightly. "If Microsoft were to actually tell people what patents they claim we violate, we could either laugh in their face and show prior art, or just show them to be obvious, or we could do things differently," he said.

Goodness Gracious Me: Christmas Special Part 1

And another great episode of Goodness Gracious Me from the BBC. Enjoy.

Goodness Gracious Me - Typical Indian Parents

Here is another great clip from the BBC's Goodness Gracious Me episode Typical Asian Parents. What Indian child can't relate to this on some level? This episode reminds me of a week I spent in the 7th grade trying to convince my parents to let me play the bassoon. My parents were convinced this would lead to me being one of those sleepy haired guys wearing Scottish outfits playing for weddings in Punjab. Ultimately they relented, which is a good thing, since I would say a good chunk of my work ethic and ability to focus came from my early music instructors.

Startup Ideas - Not Even a Dime a Dozen

I've long had a habit of coming up with random ideas for startup companies. I have a few friends who do the same thing, and often we do so together over coffee or a beer. One of my friends told me about a blog with a startup idea a day. I did a search, but accidentally found this essay by Paul Graham called Ideas for Startups instead. I love that I can still serendipitously encounter wonderful things in the web. In any event, here's a great passage from Graham's essay:

I think people believe that coming up with ideas for startups is very hard-- that it must be very hard-- and so they don't try do to it. They assume ideas are like miracles: they either pop into your head or they don't.

I also have a theory about why people think this. They overvalue ideas. They think creating a startup is just a matter of implementing some fabulous initial idea. And since a successful startup is worth millions of dollars, a good idea is therefore a million dollar idea.

If coming up with an idea for a startup equals coming up with a million dollar idea, then of course it's going to seem hard. Too hard to bother trying. Our instincts tell us something so valuable would not be just lying around for anyone to discover.

Actually, startup ideas are not million dollar ideas, and here's an experiment you can try to prove it: just try to sell one. Nothing evolves faster than markets. The fact that there's no market for startup ideas suggests there's no demand. Which means, in the narrow sense of the word, that startup ideas are worthless.

This is so great. In the back of every entrepreneurs mind is this fear that if they openly discuss their latest great idea, it will instantly get stolen by someone. I now feel motivated to write up my last 30 startup ideas. We'll see if I actually do.

Movie Review: Chameli Starring Kareena Kapoor and Rahul Bose

One of my favorite movies of all time is Chameli, starring Kareena Kapoor and Rahul Bose, directed by Sudhir Mishra, and released in 2003. For those of you who are into the occassional "aaaart pilm" - I highly recommend it and give it 4 stars. If you have never watched a Bollywood movie, and can only watch one, it would have to be this or Sholay.

There are actually very few song scenes in the movie. Here is a snippet from the film when Kareena Kapoor "sings" Bhaage Re Mann.



And here is a snippet from AishFan, an IMDB user:

This movie is definitely a breakthrough in Bollywood cinema in terms of the story and the approach to telling it. The entire movie takes place in just one night. It is interesting to see how a family values, extremely wealthy man spends a night with a prostitute. Beginning with his prejudice, he slowly grows to accept, protect, and, one can even say, love her. It took an entire movie summed up in an extremely brief "climax" to perfectly convince and portray the theme that 'It doesn't matter what the backgrounds, gender, etc. are; there should just be love.'

Microhoo - Microsoft buying Yahoo

Well after many years of speculation, it is now all over the web. Among other places, according to the Financial Times article Microsoft eyes takeover of Yahoo, Microsoft is looking to acquire Yahoo. An outright merger doesn't seem like a great idea for either entity. The two companies have very different cultures and technology preferences. Not to mention, massive redundancy in many if not most areas. I have a hard time believing the internal Microsoft bureaucracies and vested interests won't end up "Hotmail-izing" the platforms at massive and unnecessary cost. As a result of this, and the significant cultural differences, I have a hard time imagining many of the great Yahoo folks would stick around. To me this smells like the AOL take over of Netscape. AOL simply wanted the traffic and name, and the great Netscape technologists fled by the droves. Yahoo has suffered few defections to Google due in part to its great work environment and people, I wonder how long that could hold up in a merger.

It seems to me, a smarter thing to do would be to leave the two companies separate, and create an alliance on the advertising side which is where they are both really suffering. Perhaps it makes sense to merge the Panama and MSN equivalent folks into a new organization where they would focus on providing a service comparable in quality to Ad Sense and Ad Words. This would allow them to pool their respect advertising networks as well.

On a related note, perhaps someone can explain this to me, but for every site I run, and of the handful of folks I have spoken to about this, we are all in agreement that almost all, in my case >99% , of our search traffic comes from Google. I should expect about 70%. Why is this? It is true that Google attracts a different type of user than the other engines, but I still can't understand the massive discrepancy. In addition, one of my advertising friends (this is someone who runs online ad campaigns for a living) says his company has virtually abandoned taking out ads for their campaigns on anyone other than Google. When I asked him why, his answer is they simply can't get the quality of click throughs for the same cost as they get on Google.

Here's an interesting snippet from a Forbes article titled Why Yahoo! Can't Fix Microsoft:

So why wouldn't buying Yahoo! help Microsoft? For starters, Microsoft is too slow and too cautious to keep up. While Microsoft howled after Google snapped up online advertising specialist DoubleClick last month for $3.1 billion, that's just the latest opportunity Microsoft has missed. In 2005, Google beat out Microsoft to buy a stake in AOL. And while Microsoft carped after the DoubleClick deal, Yahoo! rolled up its sleeves and snapped up the 80% of Right Media it didn't already own.

Of course, Microsoft often moves slow for a very good reason: It doesn't want to cannibalize its cash cows. While Google can launch online spreadsheet and word processing applications, Microsoft can't respond without damaging its powerful Office business. Elsewhere, it's more of the same. Everywhere Microsoft makes money, the Web poses almost intractable dilemmas.

Those kinds of dilemmas would make integrating Yahoo! into Microsoft's corporate culture a nightmare. And Yahoo!'s insiders have too much faith in Panama, the newly installed program designed to boost ad revenues, to give up.

The Simpsons - India Outsourcing

Here's a hilarious snippet from an episode of the Simpsons where Homer's nuclear power plant gets outsourced to India.

Lotion: Hair Gel Revisited

In my early morning haze, I have been known to do some odd things. My wife, on occasion, cracks up seeing the milk on top of the refrigerator, and a box of cereal inside. I've even shown up to work with my shirt inside out (usually happens because I turn my shirt inside out when I paint, and sometimes forget to switch it back). Today, my absentmindedness has reached a new extreme. After getting out of the shower, and finishing my shave, I often put lotion on my face. Well today I was helping Nayan, my son, get ready at the same time. I looked in the mirror, and realized all the lotion in my hand was now being rubbed into my hair. Proof, I suppose, that multi-tasking, at least when I'm doing the tasking, is overrated.