Google… how did the internet work before?

Standard

Co-founders are multi-billionaire mathematicians and computer scientists.

Listen to my BBC radio chat with Malcolm. We discuss Google after 1 minute 30 seconds of ridiculous intro chat about Kenny Rogers in this clip:

[audio http://www.brookes.ac.uk/lifesci/runions/DrMolecule/20120821 – Google and artificial hearts.mp3]

How do you get to be one of the biggest companies on the planet when your website looks like this?

Google grew out of the vision of Larry Page and Sergey Brin who met as PhD students at Stanford University in the early 1990s. Since its inception, it has grown to become one of the world’s biggest tech companies. You know you have achieved a milestone when your company name becomes a verb – this has happened with Hoover (must irritate Dyson), Xerox and Skidoo among others. When we refer to searching for something on the web, we almost invariably say that we ‘googled’ something – even if we used a competitors search engine!

How a search engine works

For 5 or 6 years before Google came along, the internet worked differently. If you found a webpage about a subject you were interested in, you bookmarked it immediately. Webpages were only discoverable by entering their exact html address and it became very tedious listening to people read out all of the words and slashes (is it a backslash or a forward slash?). Gradually, early search engines like Yahoo and Alta Vista started to group links by interest category and this is when the internet took off – companies recognized the need to have their names grouped with competitors. I recall hearing a pundit in 1995 saying ‘any company that isn’t on the web won’t survive.’ I was shocked at the time about the prophecy of the importance of the world-wide web but I very quickly came to see the truth in what was said.

As a student looking for a PhD project, Sergey Brin was interested in data mining. At the same time, Larry Page was studying the idea that the importance of publications was linked to the number of times they were cited… These two interest sets mesh nicely and, in retrospect, I can see how the two came up with the idea of Google. Personally, I’m too thick to have even appreciated what they had done when I saw it in action — even when I started using it habitually. Gradually I stopped using the bookmarks feature in web browsers. It was actually easier just to type in the keyword or a company name and the website would magically appear.

Brin and Page’s idea was that web searchers needed a prioritized list of websites that match search terms entered by the user, i.e. if I search for Hoover, I probably am more interested in the actual Hoover website than just a randomly ordered list of all websites that mention the word ‘hoover’ (noun or verb – for any North Americans that might have accidentally strayed into this blog, we in the UK ‘hoover’ rather than ‘vacuum’). But how do you prioritize the results of a web search? Simple, you could list web pages that mention a search term based on how many other web pages refer to them. The more a page is refered to, the more important it must be – right? Of course, nowadays, the algorithms (methods) that search engines like Google use are much more ‘intelligent’ than that in my simplified example but that was the genesis of an elegant idea.

A web-crawler. These guys do the work so that a search engine can make your life easier

All the search engine company needs to do is to read every webpage on the internet, catalogue every word that’s written into a giant index, and be able to instantaneously deliver your web search results. Sounds daunting but computers are fast and getting faster all of the time. Companies like Google employ spiders – well software that they call web crawlers or web spiders – to systematically search the web for purposes of cataloguing words. That’s why we are told to be sure our web pages have pertinent titles and keywords – so that the spiders find us and display us on Google. This page, for example, has tags like ‘fun science’ and ‘technology’ that the spiders will read. It won’t be catalogued immediately but within a few days it will start to appear in Google searches (how did you get to this page?).

How many computers running crawlers and answering search queries must a company like Google have in order to keep up with the demand? The answer is astounding. Probably more than a million – running 24/7 – and they’re not insignificant computers either. All spread across at least 6 sites around the world.

How does Google make money? Simple, they will prioritize your website to a higher level if you pay them to. Notice the sites that appear at the top of you Google search that are just slightly shaded in color – they’ve paid for the privilege of being at the top of your search return. Many of us just click the first link we see when our search is returned and, chi-ching for that company.

How big can Google get?

Where is the internet going? Not-so-simple. I don’t know, but then again you know that I’m not very good at seeing the need or the promise in new web ideas. I can tell you that I’ve just gotten a new smartphone that runs the Android operating system developed by Google. That purchase was so that I could more easily interface with my email and calendars at work because we’ve switched over to Gmail (what would you guess the ‘G’ stands for?). I am now a member of Google ‘circles’ although I haven’t yet figured that out completely (Twitter takes all of my time!).

So from small things big things come. Will Google get as big as Cyberdyne Systems? Should I get a Google tatoo…?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s