SEARCH ENGINES

What is a search engine?

A search engine is a tool that helps us to retrieve information from the World Wide Web. How?

Search engines rely on automated programs called ‘spiders’ (also called crawlers or robots) to traverse the World Wide Web, following hyperlinks (linked text) from web page to web page. These spiders collect and catalogue data from each web page and store the information in a database called a ‘search index’.

The major search engines have their own search engine spiders and create their own search index which is regularly revised to keep the information accurate and up-to-date. Did you know the individual crawlers actually have names?! Google’s web crawler is called ‘Googlebot’ and Yahoo’s web crawler is called ‘Slurp’.

Data collected and stored in a search index provides an overview of a web page. The page can then be quickly and accurately matched to a relevant search query and included in the search engine results page (SERP).

The goal of a search engine is to provide the most relevant match to each search query in as little time as possible.

The goal of a web page owner is to make sure their web page is matched to every relevant search query and included in the search results page when it should be. Search engine optimisation helps to make this happen.

What kind of information does a search engine collect?

A search engine cannot actually ‘see’ the way a human user can. A search engine finds out about each web page by ‘reading’ the code. (To see this code, visit a web page then choose ‘view’ and then ‘source’ from your web browser).

There are certain parts of this code that a search engine will favour. These are generally divided into two categories:

• ‘on the page’ content which is visible to a user. Eg the web page title, headings, text and links

• ‘off the page’ content which is information contained in the code that is NOT visible to the user. Eg ‘meta tags’ and ‘alt text’ (these are names given to specific parts of the html code that creates web pages).

The type of information that can be indexed from a web page will depend on how the page is written (on the page content) and built (off the page content). This determines what information is made available to a search engine.

Some web page content is difficult or impossible for search engines to access and index. For example, content that is purely ‘visual’ such as a picture of a word cannot be ‘seen’ by a search engine, therefore the typed version of a word is always preferable. Other types of content that a search engine cannot easily index include: dynamic content such as that generated by an online database, frames based web pages, content created with Macromedia Flash and content accessed via JavaScript’s.

The first step with search engine optimisation is to make sure page content is accessible to search engines so that it can be properly indexed. A web page that has been built with care for search engine accessibility is commonly called a search engine friendly web page.

How are search results delivered?

A web search is performed by typing relevant words into a search engine. These words form the search query (also called a keyword phrase or search string). Eg If someone is looking for a house to rent in Sydney they might use the keyword phrase ‘home rentals Sydney ‘.

The search engine then accesses the search index and returns a list of relevant sites in the form of a Search Engine Results Page (SERP). There are often thousands of relevant web pages listed on the results page. For this reason search engine developers created rules for ‘ranking’ the results so they could be listed in order of importance to make it easier for the user to find what they are searching for.

These ‘rules’ are a set of complex calculations called search algorithms.

What is a search algorithm?

Search algorithms assign value to different parts of each web page and then work out an overall score based on how relevant the web page content is to the search query. Some search algorithms also take into account how popular the web page is based on how many other sites link to the page (especially if the linked site is of a similar topic). The best match takes the first spot on the search engine results page. This is often called a No. 1 ranking.

Search algorithms are frequently updated and are different for each search engine. For obvious reasons they are not disclosed however, basic information is published by each search engine in the form of guidelines and recommendations for we page developers.

SEARCH ENGINE MARKETING

What is search engine marketing?

The goal of search engine marketing is to raise the visibility of a web page in search engines and increase the number of times the page connects with its target audience online. There are basically four steps to search engine marketing:

• Search market research

• Search engine optimisation

• Search engine submissions (not required so much these days)

• Statistics analysis & strategy review

The key to effective search engine marketing lies in quality research. This is a critical step. Once the search market is understood a web page can be tailored for optimum results.

What is search market research?

Search market research is a process of information collection. Research results guide web page owners towards the most effective marketing strategy for their page. The type of information collected includes:

• whether your target audience uses search engines

• the size of the search market being targeted

• the competition that may be encountered within the industry

• whether competitors are web savvy or unaware of search engine promotion strategies

• which keyword phrases appear to be most relevant to the web page being marketed

• how often these keyword phrases are typed into search engines and

• how many other web pages are targeting the same keyword phrases

We have access to a number of tools that provide us with this information. Performing a sample search in the target search engine and analysing the results will also provide useful information. Most search engines report the number of web pages that match your query. This can give a pretty good idea about the kind of competition you’re up against and whether your expectations are realistic.

Choosing the right keyword phrases

Choosing the right keyword phrases to target is extremely important. That’s why search market research is required prior to optimisation. To demonstrate this point:

• Optimising your site for the keyword phrase ‘suit hire Brisbane ‘ might not be effective if research shows that 90% of your market is actually searching for ‘suit rentals Brisbane ‘.

• Ranking No. 1 for the phrase ‘elegant suit hire’ won’t be much good if research shows that there were zero searches for that phrase in search engines during the last month.

• Targeting the keyword phrase ‘wedding suit rentals’ will be ineffective if the web page does not actually contain relevant information about ‘wedding suit rentals’

A keyword phrase that returns 2,000 results is generally considered less competitive and easier to target than one that returns 699,000 results. In general, the fewer results returned for a particular search phrase, the less popular it is and the easier it will be to target. Don’t take this information at face value as the results must be placed in the context of your particular search market and strategy.

Single term keyword phrases such as the word ‘rentals’ are the most competitive and difficult to target unless you have access to unlimited resources for promotion. A search for the term ‘rentals’ will return results related to the rental of all kinds of items worldwide. Unless you can actually offer to rent any item worldwide, ‘rentals’ will not be a very effective keyword for your web page to target. A more appropriate (and less competitive) keyword phrase would be ‘suit rentals Brisbane ‘. Note that this also focuses your web page towards its exact market.

Where does search engine optimisation fit in?

Once research is complete, the most effective keywords to target are selected and a strategy is developed. The web page code is then adjusted to deliver this strategy. The web page is tailored so that it clearly targets the keyword phrases selected and is accessible to search engines. This process of adjusting a web page to apply research results is called Search Engine Optimisation or, SEO for short.

Site themes & site wide strategy

If a web page is supported by other web pages full of related information, a site-wide theme can be developed to raise the profile of a website in relation to a particular topic. For this reason, the more web pages optimised on a site the better. Many search engines will place greater importance on a web page that is part of a collection of relevant pages vs. a single relevant page that is part of a site which has otherwise irrelevant information. Eg a single page about suit rentals on a gardening site, will find it difficult to compete with a page that is part of a collection of web pages about suit rentals, on a suit rentals site.

Search engine registration

Once a web page has been optimised and made accessible to search engines it’s time to actually promote the web page by submitting it for inclusion in the search indexes. Manual web page submission is available from all of the major search engines via free services. A site is often included automatically if a search engine follows a link from another web page to yours.

Inclusion in a search index is never guaranteed however, if a site has lots of good quality, unique content, is well built, linked to from other sites and does not violate any search engine guidelines, it will almost certainly be included.

It can take quite a few weeks to get a web page fully indexed and for this reason, paid search engine submissions are available to speed things up. These days, paid services also have ongoing fees.

Once a page has been included in a search index there’s usually no need to resubmit unless changes to the page have been made and the search engine has not updated the information quickly enough.

[NOTE: In 2006, search engine submission is not required. A more effective option is to get a link from another web page to yours. As long as that web page is already in the search index, your site will be found via that link next time the search engine spider visits]

Reporting and analysis

The final step is analysis. Web statistics allow web page owners to track how a web page is performing over time and report information such as how many people visit a web page and how they found the page. Most web hosts provide a statistics package free of charge as part of the hosting service. If yours is not set up yet, contact your web host today! Analysing web statistics shows whether your strategy is working. Key indicators are an increase in the number of referrals from search engines via keyword phrases that are ‘on topic’ and an increase in the overall number of site visitors. If this is not happening, the web statistics can help figure out why and the strategy can be adjusted for better success.

© Alicia Laing September 2004

Updated 21 October 2006