Please note: Every link on this page opens in a new window. If your "Pop-up killer" is too efficient it can also stop new windows. When this happens, please press "Ctrl" and click on the link you want.

What is a Keyword ?

When somebody wants to find something in a search engine, he/she types in one or several words in the search engine's "search bar". On average about 50% of "searchers" use only one or two search word(s), while the rest do use more words.

But that average is over all industry sectors and different sectors show different patterns. For instance on Don Pedro's Shipping Jobs Directory website most visitors from search engines (over 80%) are using 3 or more words. These words are called a "search string" or "search phrase" (when two or more words).

The search engine computer then reads and uses these same words to find relevant web pages. The search words are, in other words, used as "keys" to open the data bank and retrieve the correct information - that's why the same words are called "keywords" when the search engine uses them. The result presented consists then of a list of those webpages where those words are present.

Search for one of your webpages in a search engine. When you find your webpage on the results page click on "cache", that's the search engine's stored version of your webpage. You will see some words highlighted, these are the keywords indexed from that page.

Alternately you can look at Google's text cache - it doesn't show you specifically the keywords, but the text cache shows you what Google's spider sees on your page. This way you can check whether Google's spider can see all your keywords and key phrases or not. Type in Google's search bar:
http://www.google.com/search?q=cache: http://www.example.com/&hl=en&lr=&strip =1
"www.example.com" is of course your own website URL, although somebody has published a website under this domain ( www.example.com ).

Usually the search engine puts the pages where the "keywords" are present the greatest number of times on the top, and at the end comes those pages where may be only one of the search words is present. This practice is now (2007) changing as more and more search engines are employing semantic indexing to find the most relevant documents for each search query.

If the search words your potential visitor is using are not present at all on any one of your webpages your webpages will not be found on the results pages. It's therefore necessary to use those keywords people are really using when searching; i.e. the popular keywords. All keywords are, however, not equal - even if somebody is using them sometimes.

How to Find Keywords ?

As said above, all keywords are not equally good. You need to find a selection of keywords applicable for your website, a separate set for every webpage. Always remember the search engines are registering webpages, not websites, every single webpage succeeds or fails on its own.

This could, however, be changing as search engines start more strongly to consider topics ( themes ) for each website. That is then compared with the topics ( themes, subjects ) for each webpage included in the website. Do consider the full structure of the content on your whole website, may be you can stay ahead of some search engines.

Naturally you would have a main subject for your site, each separate webpage containing details on sub-subjects. As an example see the Site Map for this web site. It means you choose keywords specifically for each and every page, but you choose them so that taken all together they represent a family of keywords, i.e. a certain theme in which they all belong together. Don't mix "cats and dogs", except in a pet shop.

There are different ways to find keywords you then can test for efficiency. The difficulty is to pick word strings when you start optimizing. Pick word strings - two to four word sets - you get more possibilities for variation and better targeting. You can then later in your text use the words both together and/or each separately, see How to Write Internet Web Pages.

1. A brand new website

   You could start with "brain storming" together with some friends in your own sector. Try to come up with as many different suitable words and phrases as possible that you think people will use to find your website.
   
   With a brand new website the biggest problem is to come up with suitable keywords or phrases you can check if really used or not. Once you pass this hurdle you can continue as with an already running website. With one exception; you cannot expect to get to the top in search engine results pages before the website is a couple of years old. See Don Pedro's What is Google's Sandbox ?.
   
   

2. An already existing website

   Start with making a list of your competitors. Once you have that list, get the most important competing pages (highest Google PR-number) and download them on your computer. Take the most common keywords or phrases (see below how to do) and go to the major search engines in the region you plan to target. Note down rank number in the search results pages for each competing webpage.
   
   For a long time Google was up-dating the published data on the PR-number in Google's toolbar every 3-4 months. In 2007 that published number was first up-dated beginning of April and next time only in end of October. That was, in other words, with a 6 months interval, which means the published number can be up to 6 months old. The un-published number, which only Google's engineers know, is the one that's used for rankings and that one is up-dated continuously.
   
   The last step in this first analysis is to pick out those keywords and phrases that seem to be the strong ones (most popular) and subject them to optimization check as explained below.

From each search engine you check the pages on at least the first results page. If you want a bigger selection take also the 2^nd results page. Open those web pages, put your mouse on an empty space and give a right-hand-click, choose: "View source". At the very top of that code page you find the "meta tags", copy: <meta name="Keywords" content="...."> and paste it for instance on a Word page together with name (URL) of the site you have taken it from. This is for possible later reference, in case you want to check something more on certain pages.

With an already existing website you may be have collected real used keywords from your "visitor statistics". Start making a list of all keywords found. Many will be common to the different web pages, note down for each word the number of pages where it's present, together with page name.

The more technical the subject is, the longer search phrases you can expect. Three or four word search phrases means closely targeted search results, which also means less competition because of more strict targeting.

How to Choose Popular Keywords ?

As said above all keywords are not equal. I think everybody agrees, a keyword people do not use when searching for something isn't a very good keyword. So you need to start with keywords in real use. For this you can use Good keywords. This is a free down loadable program intended for finding those keywords people use when searching on the net.

The program gives also variations on those search phrases and words you want to search for together with number of searches for each search word or search phrase. Check those you earlier found to be common on several web pages and are possible for your web pages. Again copy and paste and make a print out copy. I think when sorting through several words, phrases, or concepts, most of us work best with a paper copy. Group the words and phrases for each separate webpage in a descending order depending on number of searches.

An other tool is Niche Bot, which works differently and gives different suggestions. You should get a good result if you use both, and then apply common sense.

When you get statistics on keyword use, don't look at absolute numbers. The numbers are relative to each other in the same list. Lists from different sources cannot be compared in absolute terms.

Because each keywords suggesting tool is limited to it's own data base the "best tool" would obviously be the one with the biggest data bank, which happens to be Google. Now you can use Google's AdWords keyword tool. Originally it was designed for PPC ( Pay Per Click ) advertisers but it's available to everybody for free - Google's AdWords Keyword Tool.

Now you will have a bigger and better collection of possible words to use. However, all "popular" keywords are neither equal nor efficient. Whether a keyword is efficient or not depends on how many webpages there are on the net where you can find that word. Even if the keyword choice of your heart is used by one thousand searchers daily, it's still not efficient if it can be found on two million pages. There's one more step to go.

When working with an already existing text you can usually change many instances of "it, that, this, etc.". Instead use either those popular keywords and phrases you found or may be very specific words; like "Our model XYZ gives you ...", instead of "it gives you ...".

Even if you notice some people have used misspelled words to find your webpage, don't use misspellings yourself. Most major search engines correct the spelling when searching for pages to present to the searcher. If you use misspelled words on your webpages it reduces your credibility.

If you see a menu in a restaurant spelling "spaghetti carbonada" wrong, would you ask for spaghetti at that place? How can the cook produce a proper spaghetti if he doesn't even know how to spell it?

How to Select Efficient Keywords

For this you need one more program: WebCEO, which is another free download. This program makes a lot of work for you, it helps you to:
- Research your keywords and key phrases,
- Optimize all your webpages,
- Submit your new website,
- Check your webpage rankings in the search engines, and
- Check backlinks ( incoming links ) to your webpages.

For most keywords and key phrases you will get both number of daily searches ("Demand") and number of websites ("Supply") you have to compete with. Remember, it's the searchers who decide which webpage to pick out from the search results page, i.e. they are demanding information and choosing between the available supply of webpages.

To make it easier to choose keywords and phrases each choice you check receives a relative number called KEI ( Keyword Efficiency Index ). It's calculated as Demand (DS) squared divided by Supply (C), or mathematically:
where:
DS = Number of daily World searches for that keyword or key phrase, and
C   = Number of web pages available, i.e Competition.

The higher the KEI number is, the more efficient is that keyword or key phrase:

KEI = 000 -   10 KEI = 010 - 100 KEI = 100 - 400+ Poor or not efficient keyword or key phrase Good and efficient keyword or key phrase Excellent or very good keyword or key phrase

Before you make your final selection go to keyword difficulty tool. To use it you must sign up first but it's for free, use is limited to two reports per day. The tool calculates in percent how difficult it is to rank for a certain search phrase. Best results you get when comparing two phrases with each other. After using this tool you can continue as suggested below.

May I suggest you choose the key phrases with the very highest KEI - also considering the results from the "keyword difficulty tool" - as your webpage headings, and those with slightly lesser KEI as webpage sub-headings.

The webpage file name, i.e. for instance: example.html, you should also choose from the same list, but don't take one of the very highest ones. The file name you should never change. If you change the file name you will loose all the previous incoming links you have worked for so hard as well as that webpage's position in the search engines.

The webpage headings you should subject to regular tests, may be every 6 months or so, because trends change and people start using different search words or phrases. You can do it in Google Trends, where you can follow your own keywords with a one month graph, which is updated daily. It's no problem what so ever to change a webpage title and/or heading as long as the file name is constant.

If you are targeting Europe then the German webpage promotion tool (IBP), (15.7 MB), would probably be more suitable for you - it's in English or German. The Trial version (7.9 MB) is free but stripped of certain functions. It includes, however, a validator / code checker ( lint program ). Together with the wine program the "IBP" runs also on Linux.

Whether you use both Good Keywords and WebCEO together or separately their efficiency depends very much on the sector. Some sectors (like the Maritime sector) are small, which means few searches per day, which leads to faulty statistics because of missing data. Therefore better using them together, when one doesn't give anything hopefully the other does.

Number of Competitors

Number of "competitors" you can always get manually, for instance from Google. When you get a search result, at the top you can see: Results 1 - 10 of x.xxx.xxx; i.e. total number of websites in Google with that search word or phrase present.

Example:

Effect of Longer Search Phrases - Aug. 2010   (Google) Words used Number of webpages Website design Website design job Website design job Europe Website design job West Europe Freelance website design job West Europe Freelance website design job West Europe for Croatian 399 000 000 333 000 000 13 200 000 7 810 000 147.000 11.700

BUT, just the fact that one or several key words are present on a webpage doesn't necessarily mean that's a competitor to your webpage. If your page is even a little bit optimized the number of real competitors drops considerably. So now we arrive at the last step for selecting the final efficient keywords for your webpages.

Take those of your key words you already have found out have the highest KEI number, may be the "top ten". Then you take these - one by one, or phrase by phrase - and type in the Google search bar: intitle:keyword .

Because if your important key word or phrase is present in a webpage's title it's a clear indication that webmaster has done some optimization. The less pages there are for a certain word or phrase in this search the more efficient that key word is and the less competition it has. These are the ones you should use for your own webpage.

Even this is not as straight forward as it seems. I give an example:

When you type intitle:cargo ship job, you get a total of only 27.300 webpages. But wait, this number includes all those pages where one or several of the words are present in the title. In this case most of these are stories about cargo ship job, so if your page is, for instance, a ship employment portal then every page is not a competitor.

Now when you type instead intitle:"cargo ship job", you get only those pages where the whole phrase as such is present in the title. These are now all your real competitors. In fact, this time the result is just one page only, and that's my own webpage (on the Shipping Job Links site).

Some more tips and tricks for finding competitors and key phrases.

What is the Long Tail ?

Especially for websites with a technical topic ( subject ) where you find a lot of detailed descriptions on long webpages you will also in the search engine queries find long search phrases ( 4 words or more ), each one appearing may be 2 times - very often once - only.

On my Shipping employment portal more than 50% of queries appears only once. In September this was over 80%. These are called "the long tail". From the diagram below the name itself is self-explanatory, isn't it.

A majority of websites have may be 5-10 top key phrases, each one bringing a great number of visitors. The long tail, however, have a multitude of different search terms ( phrases ) plus all random and weird variations of words / phrases that can lead to the same webpage.

A "long tail" doesn't mean long search phrases, but it means a great percentage of search phrases used once only so when these are plotted in a graph you can see the very long tail.


The "very long tail" - top 1.000 queries for one week.
( July 2009 )
The Long Tail requires a great number of possibilities ( descriptions, products, etc. ), each one wanted by few visitors / customers only. The total demand of not so popular search terms or products is often greater than the total demand of the most popular ones.

Because I moved that website in the middle of April I wasn't prepared for a very strong increase in the traffic. The statistics below were prepared manually. Because I didn't have any suitable program to do it yet the result for 2007 is somewhat faulty. Hopefully, with the program from where the diagram above comes - 10³ Bees - I will get more correct statistics for a full month.

With this online program you get (among other data):
- search queries; today, yesterday, week / month
- what pages, what search engines, suggestions for each query
- rankings for each query
- queries formulated as questions as a separate group, suggestions

The queries formulated as questions you can use at least two ways:

1. You can use the question as it is for a new webpage file name, title, and/or page name / heading - without question mark. Especially if the question points out something that currently may be is missing in your website content.
   
2. Or you can use the question as such or slightly changed as sub-heading for some new added content on an already existing webpage.

If one person is asking for some details it's very much possible somebody else will later ask for something similar. Then you have it already on your webpage. Right ? Those questions already asked are a very good source for new ideas.

The program is available in multiple languages: English, Deutsch, Español, Français, Nederlands, Norsk, Połski, Slovenskij, Türkçe.

If you would like to calculate "your own" long tail, see A practical Model for Analyzing Long Tails.

Long or Short Key Phrases ?

It's better to think about search phrases instead of a set of single key words. Below I show a table with the results from July 2006 to 2010 statistics of the site Don Pedro's Shipping Job Links.

The table shows quantity of searches as per number of words in each query in percent of total queries.

Search Phrases Used in Percent, 2006 - 2010

Number of words

July 2006 (n = 1.810)

July 2009 (n = 2.711)

July 2010 (n = 3.036)

1 word 2 words 3 words 4 words 5 words 6 words 7 or more 0.3 % 14.6 % 39.1 % 26.5 % 12.4 % 5.0 % 2.1 % 0.6 % 13.2 % 35.0 % 28.8 % 13.4 % 5.2 % 3.8 % 1.5 % 16.4 % 48.7 % 20.1 % 8.5 % 3.4 % 2.9 %

All variations of "donpedroshipping" have been excluded, because they are "navigational" queries, i.e. not "real" search queries. When counting the words, "stop words" has not been counted as the search engines usually ignore them in the queries.

"Stop words" are common words such as "the", "of", "on", etc. Even it's said the search engines ignore them, different search engines can behave differently and it's not sure which one ignores which stop words.

In February 2010 Hitwise ( US ) released a report about search terms getting longer. It's not clear from the report whether stop-words have been included or not but the trend is same as in the table above. When you compare my results with those from Hitwise, remember all website statistics is website specific.

In 2010 almost half of searchers used three words, with the result skewed to more than three. About 85% of queries were phrases of three or more words. I have seen an estimated average over the major search engines; about 31% of queries contained 4 or more words (2007). The results from my pages would give about 35% containing 4 or more words in 2010.