Tech - Search: Curation versus Algorithm

For those who remember the early days of the internet, they will be familiar with the free web hosting sites like Geocities. Before Geocities was taken offline, it's content was archived and is now available for download as a 650 GB torrent. While this seems like a large amount of data, it is estimated that around 2,551,568,697 GB data per day was created in 2011! While the earlier days of the internet were largely image and text based content, data today is in the form of images, text, social media posts, videos, businesses, services and user generated content.

All this only increases our dependency on search engines. More than the challenge of indexing all the data, search engines are starting to grapple with relevancy of results. The added complication is that some of the social media and other sites are not publicly available to be indexed. The two main search engines - Google and Bing seem to be taking markedly different approaches. Fundamentally, the approach is relying on either algorithms (by Google) or curation (by Bing). Let's look at their recent updates.

Power of algorithm

Google Search
In the above example, search for 'mozart' on google. The first set of results are all centred around Mozart, the music composer. So, how does Google do it? The recent updates to the search include the 'knowledge graph'. According to Google, this graph contains all the real-world entities and their relationships. It is a bold aim and I wouldn't be surprised if there was an element of curation in building the graph.

Power of crowd

Bing Search

Bing has decided to take a different approach. Rather than building an context-aware graph, it has decided to use the existing real-world connections. In the example above (currently available only in the US), a search for 'movies' shows the results from the search engine. In addition, it also shows results from your social graph (for now twitter and facebook). Of course, you need to be signed into all these services. Currently, Bing is a distant second in the search engine market. It will be interesting to see what kind of impact this latest update has.

What Next

But, what has not been solved so far is the 'intent' of the search query. There are times when we want to be influenced by our real-world connections and there are times when we want a fresh perspective. It will be interesting how Google and Bing solve this problem.

Also, how far will we take curation? For example, in most cases, currently there are algorithms which power everything - right from the balancing of books in a finance software to recommending music to users. Will some of these be replaced by curation? 

While we will continue to have algorithms, there is definitely potential for curation too. I think we have barely begun to scratch the surface.
Location: Bangalore, Karnataka, India

Comments

Post a Comment

Popular posts from this blog

Cloudera Quick Start VM in Hyper-V

Image
Overview Marc Andreessen penned his now famous essay, “ Why Software Is Eating the World ” in Wall Street Journal in 2011. However, if we dig under the covers, it's really virtualization that is powering this phenomenon. Today, literally every hardware component such as - processor, memory, network, storage, network load-balancer, network router and switch, firewall are virtualized. In fact, even a user is now "virtualized" especially in scenarios such as load testing and synthetic monitoring. Finally, as a concept, money is also virtualized in the form of crypto-currencies. In this post, let's go through how we can implement virtualization in a local machine and get better understanding of the concepts of virtualization.  Virtualization  One key aspect, driver and benefit of virtualization is sharing of the physical resources.  Before we dive in, let's define some concepts. Host Machine - This can be a server rack, single server or a desktop/lapto
Read more

Book Review - The Price of Being Fair

Image
Overview  As the FairPrice group in Singapore completes 50 years, "The Price of Being Fair" book is a timely story of it's birth and evolution. The book has been authored by Sue-Ann Chia and Shing Huei Peh. While being true to history, the book has nuggets of "behind the scenes" information, given the time the authors have spent with various leaders at FairPrice. The book is already a best seller according to Kinokuniya book store. At under 200 pages, it makes for a breezy read. The Usual Suspects Having worked in retail industry for over two decades, the travails of  NTUC FairPrice seem familiar to me. A retail CEO once remarked that retail is a pretty "public" industry. There is not much behind which a retail business can hide. As such, retailers often have to follow the aspirations and concerns of the communities and customers they serve. The journey of FairPrice from being an EDLP (everyday low price) retailer to one having multiple formats, busin
Read more

Azure Chronicles - VM Security

Using the cloud has many benefits, some of them such as elasticity and scalability. On the other hand, some other aspects such as security and cost need to be paid attention to. Let's look at the security aspect in this post. Lynis is one of the most popular Linux auditing and hardening tools. It has community and enterprise editions. As an example, the community edition is being used here. Let's scan a Linux based VM in Azure and find out the results. This post assumes that you have an existing subscription in Microsoft Azure. The below steps show how to scan a VM in Azure. A new class of VM's called B1ls was launched. These VM's are quite small and cheap. As this is just an experiment, let's try this new VM. Create a B1ls VM with the Ubunto 18.04 LTS image on it. Make sure that the VM has a public IP address and use a public key for access. Once the VM is created, login to the VM using a tool such as Putty. Follow the steps described here to ins
Read more