What Is a Search Engine Optimization?

What Is a Search Engine Optimization?

Okay, so you know the basic concept of a search engine. Type a word or phrase into a search box and click a button. Wait a few seconds, and references to thousands (or hundreds of thousands) of pages will appear. Then all you have to do is click through those pages to find what you want. But what exactly is a search engine, beyond this general concept of “seek and ye shall find”?

It’s a little complicated. On the back end, a search engine is a piece of software that uses applications to collect information about web pages. The information collected is usually key words or phrases that are possible indicators of what is contained on the web page as a whole, the URL of the page, the code that makes up the page, and links into and out of the page. That information is then indexed and stored in a database. 

On the front end, the software has a user interface where users enter a search term — a word or phrase — in an attempt to find specific information. When the user clicks a search button, an algorithm then examines the information stored in the back-end database and retrieves links to web pages that appear to match the search term the user entered.

The process of collecting information about web pages is performed by an agent called a crawler, spider, or robot. The crawler literally looks at every URL on the Web, and collects key words and phrases on each page, which are then included in the database that powers a search engine. Considering that the number of sites on the Web went over 100 million some time ago and is increasing by more than 1.5 million sites each month, that’s like your brain cataloging every single word you read, so that when you need to know something, you think of that word and every reference to it comes to mind.

Anatomy of a Search Engine

By now you probably have a fuzzy picture of how a search engine works. But there’s much more to it than just the basic overview you’ve seen so far. In fact, search engines have several parts. Unfortunately, it’s rare that you find an explanation for just how a search engine is made — and that information is vitally important to succeeding with search engine optimization (SEO).

Query interface

The query interface is what most people are familiar with, and it’s probably what comes to mind when you hear the term “search engine.” The query interface is the page that users see when they navigate to a search engine to enter a search term.

Today, many search engines on the Web have added much more personalized content in an attempt to capitalize on the real estate available to them. For example, Yahoo! Search, shown in Figure1-2, allows users to personalize their pages with a free e-mail account, weather information, news, sports, and many other elements designed to make users want to return to that site to conduct their web searches. 

One other option users have for customizing the interfaces of their search engines is a capability like the one Google offers. The Google search engine has a customizable interface to which users can add different gadgets. These gadgets allow users to add features to their customized Google search home that meet their own personal needs or tastes.

Responsive Search-Engine Template - a Collection by phpSoftware on CodePen

When it comes to search engine optimization, Google’s user interface offers the most ability for you to reach your target audience, because it does more than just optimize your site for search; if there is a useful tool or feature available on your site, you can allow users to have access to this tool or feature through the Application Programming Interface (API)made available by Google. This allows you to have your name in front of users on a daily basis. 

For example, a company called PDF24.org has a Google gadget that allows users to turn their documents into PDF files, right from their Google home page once the gadget has been added. If the point of search engine optimization is ultimately to get your name in front of as many people as possible, as often as possible, then making a gadget available for addition to Google’s personalized home page can only further that goal.

Crawlers, spiders, and robots

The query interface is the only part of a search engine that the user ever sees. Every other part of the search engine is behind the scenes, out of view of the people who use it every day. That doesn’t mean it’s not important, however. In fact, what’s in the back end is the most important part of the search engine.

If you’ve spent any time on the Internet, you may have heard a little about spiders, crawlers, and robots. These little creatures are programs that literally crawl around the Web, cataloging data so that it can be searched. In the most basic sense all three programs — crawlers, spiders, and robots — are essentially the same. They all “collect” information about each and every web URL.

This information is then cataloged according to the URL on which they’re located and are stored in a database. Then, when a user uses a search engine to locate something on the Web, the references in the database are searched and the search results are returned.


Every search engine contains or is connected to a system of databases, where data about each URL on the Web (collected by crawlers, spiders, or robots) is stored. These databases are massive storage areas that contain multiple data points about each URL. 

The data might be arranged in any number of different ways, and will be ranked according to a method of ranking and retrieval that is usually proprietary to the company that owns the search engine.


Share on Social Media: