Monday, March 23, 2009

APIs Pave the Way Toward Understanding

20 years ago this month, Tim Berners Lee proposed a “universal linked information system” at the Cern particle physics laboratory that would evolve into what we now know of as simply "the web." One of the more interesting phenomenon in the evolution of the web, has been the evolution toward the web-as-platform. Since even before a different, though perhaps no less influential, Tim -- Tim O'Reilly -- articulated the notion in his classic piece "What is Web 2.0", web companies have been rapidly exposing APIs which empower others to build upon their applications and data. This phenomenon has become so prolific, that it's almost rare to find a web company that does not have an API as a core part of their strategy. These APIs are forming the building blocks enabling Tim Berners Lee's more recent vision of the semantic web to be realized.

The semantic web is in essence, a web of understanding, where the actual meaning of web documents is made available to machines enabling a new generation of applications to emerge. The semantic web is being realized by many companies exposing APIs that are in turn used by other companies. For example, Yahoo has a terrific API with a very liberal usage policy. Yahoo has already spent billions of dollars indexing the entire web. Companies like Hakia, and my own, Evri, have taken advantage of Yahoo's extensive web crawl and global ranking to sample the web and index only the documents our users really need; this is a significant cost savings for a start up. Let's take another example; Free Base is busy building an extensive database of things -- think of it as a very highly structured Wikipedia, where not only is there a page for something like Barack Obama, but all of the data is completely structured and machine readable. This is in turn extremely helpful for applications like Powerset and Evri which need to be able to recognize these things in the many web documents their machines attempt to read in a human-like manner.

All of the above examples are, in essence, examples of machines talking to machines. In other words, when Hakia's indexing system leverages the Yahoo API to help determine what documents to index, I doubt a human is in the loop. In addition to machines talking to machines, there is, of course, the case of machines talking to humans. One of the hardest nuts to crack in the semantic web arena, is what a truly successful user experience looks and feels like. If you think back in time, when Alan Kay invented the desktop metaphor for PCs at Xerox Parc, the idea was not completely obvious. In fact, it was more than a decade before the metaphor was popularized by Apple. One of the most powerful motivations driving API proliferation is the appeal of access to a much larger community of innovators than are available within ones own company. For example, at my company we have great product designers and a terrific UI team; I have every confidence we are on the right path toward cracking the user experience problem for a deep semantic search engine. But, however good our team is, the user interface possibilities of building on our platform are endless. Why should we force our partners and customers to use our UIs? I think a similar conclusion is quite common; many semantic companies have some significant technology in the "deeper understanding" arena that enables a great number of UI permutations. So companies like Zemanta, Day Life, Reuters (Open Calais) and others all conclude: technology aspects with individual utility should be freed so its inherent value can be realized.

And of course, "freeing" technology aspects in the form of an API enables different monetization opportunities. In this rough economic climate, its a true advantage for any company to have diverse revenue streams. Many might not realize that almost 10x of Twitter's traffic is API driven. While most of us have little idea exactly what Twitter's monetization strategy looks like, I will wager a guess that making money off the API will be a core part of it. So how are companies making money off their APIs? Some are taking payment in the form of links to other already monetized pages. Many are simply charging for it. Most companies offer a certain volume of API requests free, then charge an escalated amount as a function of API request volume.

It seems obvious to most that social networking applications played a pivotal role in the web's evolution the past 10 years. Similarly, when we celebrate the 30th anniversary of the web, I predict APIs exposing a deep understanding of the web's content will be unambiguously recognized as primary drivers.

4 comments:

Unknown said...

Great post, I totally agree with your vision. It actually will be the subject of my MSc thesis on the Semantic Web. What's your view on semantic annotation such as RDF in the context of APIs?

Kalyan Lanka said...

Good one. I have posted in my blog about OpenCalais and OpenPublish which look pretty promising. I know Evri also is doing some kind of that stuff.

Deep said...

I haven't seen any services used or not used on the basis of RDF or the lack of it. I think its pretty trivial to parse some XML or JSON or RDF; the real challenge most companies face is how to get developers to care about their APIs, and the real challenge developers face is what to actually build to attract users. I think makers of successful web APIs tend to focus on what their developers need, rather than worrying too much about standards compliance early on.

Anonymous said...

You hit on a primary driver on how we are getting "smarter" information from the cloud.

What we're seeing is end-points exposing deep understanding or advanced computational services.

As more services expose "smart" end-points the consuming applications will begin to do some of what we have been predicting for the past XX years.

Developers can now focus on presenting information in the best way for digesting rather than having to figure out all the nitty gritty of how to do y or z.

Related Posts with Thumbnails

Liked what you read? Tell your friends

More info about content in my post