Search Engine Optimization (SEO)

SEO is the practice of improving and promoting a website to increase the number of visitors the site receives from search engines. There are many aspects to SEO, from the words on your page to the way other sites link to you on the web.


What is Search Engine Optimization (SEO)?


SEO is a marketing discipline focused on growing visibility in organic (non-paid) search engine results. SEO encompasses both the technical and creative elements required to improve rankings, drive traffic, and increase awareness in search engines. There are many aspects to SEO, from the words on your page to the way other sites link to you on the web. Sometimes SEO is simply a matter of making sure your site is structured in a way that search engines understand.


SEO isn't just about building search engine-friendly websites. It's about making your site better for people too.


Why does my website need SEO?


The majority of web traffic is driven by the major commercial search engines, Google, Bing, and Yahoo!. Although social media and other types of traffic can generate visits to your website, search engines are the primary method of navigation for most Internet users. This is true whether your site provides content, services, products, information, or just about anything else.

Search engines are unique in that they provide targeted traffic people looking for what you offer. Search engines are the roadways that make this happen. If search engines cannot find your site or add your content to their databases, you miss out on incredible opportunities to drive traffic to your site.


Search queries—the words that users type into the search box— carry extraordinary value. Experience has shown that search engine traffic can make (or break) an organization's success. Targeted traffic to a website can provide publicity, revenue, and exposure like no other channel of marketing. Investing in SEO can have an exceptional rate of return compared to other types of marketing and promotion.



Why can't the search engines figure out my site without SEO?


Search engines are smart, but they still need help. The major engines are always working to improve their technology to crawl the web more deeply and return better results to users. However, there is a limit to how search engines can operate. Whereas the right SEO can net you thousands of visitors and increased attention, the wrong moves can hide or bury your site deep in the search results where visibility is minimal.


In addition to making content available to search engines, SEO also helps boost rankings so that content will be placed where searchers will more readily find it.


Can I do SEO for myself?


The world of SEO is complex, but most people can easily understand the basics. Even a small amount of knowledge can make a big difference. Free SEO education is widely available on the web, Combine this with a little practice and you are well on your way to becoming a guru.


Depending on your time commitment, your willingness to learn, and the complexity of your website(s), you may decide you need an expert to handle things for you. Firms that practice SEO can vary; some have a highly specialized focus, while others take a broader and more general approach.


How Search Engines Operate?


Search engines have two major functions: crawling and building an index, and providing search users with a ranked list of the websites they've determined are the most relevant.



1. Crawling and Indexing


Crawling and indexing the billions of documents, pages, files, news, videos, and media on the World Wide Web. Each stop is a unique document (usually a web page, but sometimes a PDF, JPG, or other files). The search engines need a way to “crawl” the entire city and find all the stops along the way, so they use the best path available links.



The link structure of the web serves to bind all of the pages together.


Links allow the search engines' automated robots, called "crawlers" or "spiders," to reach the many billions of interconnected documents on the web.

Once the engines find these pages, they decipher the code from them and store selected pieces in massive databases, to be recalled later when needed for a search query. To accomplish the monumental task of holding billions of pages that can be accessed in a fraction of a second, the search engine companies have constructed datacenters all over the world.


These monstrous storage facilities hold thousands of machines processing large quantities of information very quickly. When a person performs a search at any of the major engines, they demand results instantaneously; even a one- or two-second delay can cause dissatisfaction, so the engines work hard to provide answers as fast as possible.


2. Providing Answers


Providing answers to user queries, most frequently through lists of relevant pages that they've retrieved and ranked for relevancy. Search engines are answer machines. When a person performs an online search, the search engine scours its corpus of billions of documents and does two things: first, it returns only those results that are relevant or useful to the searcher's query; second, it ranks those results according to the popularity of the websites serving the information. It is both relevance and popularity that the process of SEO is meant to influence.


How do search engines determine relevance and popularity?


To a search engine, relevance means more than finding a page with the right words. In the early days of the web, search engines didn’t go much further than this simplistic step, and search results were of limited value. Over the years, smart engineers have devised better ways to match results to searchers’ queries. Today, hundreds of factors influence relevance, and we’ll discuss the most important of these in this guide.


Search engines typically assume that the more popular a site, page, or document, the more valuable the information it contains must be. This assumption has proven fairly successful in terms of user satisfaction with search results.


Popularity and relevance aren’t determined manually. Instead, the engines employ mathematical equations (algorithms) to sort the wheat from the chaff (relevance), and then to rank the wheat in order of quality (popularity).

These algorithms often comprise hundreds of variables. In the search marketing field, we refer to them as “ranking factors.”



How People Interact with Search engines?


One of the most important elements to building an online marketing strategy around SEO is empathy for your audience. Once you grasp what your target market is looking for, you can more effectively reach and keep those users.


We like to say, "Build for users, not for search engines." There are three types of search queries people generally make:

"Do" Transactional Queries: I want to do something, such as buy a plane ticket or listen to a song. 

"Know" Informational Queries: I need information, such as the name of a band or the best restaurant in New York City. 

"Go" Navigation Queries: I want to go to a particular place on the Internet, such as Facebook or the homepage of the NFL.



When visitors type a query into a search box and land on your site, will they be satisfied with what they find? This is the primary question that search engines try to answer billions of times each day. The search engines' primary responsibility is to serve relevant results to their users. So ask yourself what your target customers are looking for and make sure your site delivers it to them. It all starts with words typed into a small box.


Search Engine Tools and Services


SEOs tend to use a lot of tools. Some of the most useful are provided by the search engines themselves. Search engines want webmasters to create sites and content in accessible ways, so they provide a variety of tools, analytics, and guidance. These free resources provide data points and unique opportunities for exchanging information with the engines.



Common Search Engine Protocols


1. Sitemaps


Think of a sitemap as a list of files that give hints to the search engines on how they can crawl your website. Sitemaps help search engines find and classify content on your site that they may not have found on their own. Sitemaps also come in a variety of formats and can highlight many different types of content, including video, images, news, and mobile.


XML


Extensible Markup Language (recommended format)

This is the most widely accepted format for sitemaps. It is extremely easy for search engines to parse and can be produced by a plethora of sitemap generators. Additionally, it allows for the most granular control of page parameters.
Relatively large file sizes. Since XML requires an open tag and a close tag around each element, file sizes can get very large.
Easy to maintain. RSS sitemaps can easily be coded to automatically update when new content is added.



RSS


Really Simple Syndication or Rich Site Summary

Harder to manage. Although RSS is a dialect of XML, it is actually much harder to manage due to its updating properties.
Extremely easy. The text sitemap format is one URL per line up to 50,000 lines.
Does not provide the ability to add metadata to pages.



Txt: Text File



2. Robots.txt


The robots.txt file, a product of the Robots Exclusion Protocol, is a file stored on a website's root directory (e.g., www.google.com/robots.txt). The robots.txt file gives instructions to automated web crawlers visiting your site, including search crawlers.


By using robots.txt, webmasters can indicate to search engines which areas of a site they would like to disallow bots from crawling, as well as indicate the locations of sitemap files and crawl-delay parameters. You can read more details about this at the robots.txt Knowledge Center page.


The following commands are available:

Disallow


Prevents compliant robots from accessing specific pages or folders.


Sitemap


Indicates the location of a website’s sitemap or sitemaps.


Crawl-Delay


Indicates the speed (in milliseconds) at which a robot can crawl a server.



Measuring and Tracking Success


They say that if you can measure it, then you can improve it. In search engine optimization, measurement is critical to success. Professional SEOs track data about rankings, referrals, links, and more to help analyze their SEO strategy and create roadmaps for success.




1. Search Engine Share of Referring Visits


Every month, keep track of the contribution of each traffic source for your site, including:

Direct Navigation: Typed in traffic, bookmarks, email links without tracking codes, etc. 

Referral Traffic: From links across the web or in a trackable email, promotional, and branding campaign links Search Traffic: Queries that sent traffic from any major or minor web search engine Knowing both the percentage and exact numbers will help you identify weaknesses and give you a basis for comparison over time. For example, if you see that traffic has spiked dramatically but it comes from referral links with low relevance, it's not time to get excited. On the other hand, if search engine traffic falls dramatically, you may be in trouble. You should use this data to track your marketing efforts and plan your traffic acquisition efforts.


2. Search Engine Referrals


Three major engines make up 95%+ of all search traffic in the US: Google and the Yahoo!-Bing alliance. For most countries outside the US, 80%+ of search traffic comes solely from Google (with a few notable exceptions including Russia and China). Measuring the contribution of your search traffic from each engine is useful for several reasons:


Compare Performance vs. Market Share


Compare the volume contribution of each engine with its estimated market share.


Get Visibility Into Potential Drops


If your search traffic should drop significantly at any point, knowing the relative and exact contributions from each engine will be essential to diagnosing the issue. If all the engines drop off equally, the problem is almost certainly one of accessibility. If Google drops while the others remain at previous levels, it's more likely to be a penalty or devaluation of your SEO efforts by that singular engine.



Uncover Strategic Value


It's very likely that some efforts you undertake in SEO will have greater positive results on some engines than on others. For example, we've observed that on-page optimization tactics like better keyword inclusion and targeting reap greater benefits with Bing and Yahoo than with Google. On the other hand, gaining specific anchor text links from a large number of domains has a more positive impact on Google than the others. If you can identify the tactics that are having success with one engine, you'll better know how to focus your efforts.


3. Visits Referred by Specific Search Engine Terms and Phrases


The keywords that send traffic are another important piece of your analytics pie. You'll want to keep track of these on a regular basis to help identify new trends in keyword demand, gauge your performance on key terms, and find terms that are bringing significant traffic that you're potentially under-optimized for. You may also find value in tracking search referral counts for terms outside the top terms and phrases those that are most valuable to your business. If the trend lines are pointing in the wrong direction, you know efforts need to be undertaken to course- correct. Search traffic worldwide has consistently risen over the past 15 years, so a decline in the numbers of referrals is troubling. Check for seasonality issues (keywords that are only in demand certain times of the week/month/year) and rankings (have you dropped, or has search volume ebbed?).


4. Conversion Rate by Search Query Term/Phrase


When it comes to the bottom line for your organization, few metrics matter as much as conversion. With this information, we can now do two things:



i) Checking our rankings, we see that we only rank #4 for "SEO Tools." Working to improve this position will undoubtedly lead to more conversion.


ii) Because our analytics will also tell us what page these visitors landed on, we can focus our efforts on improving the visitor experience on that page.


The real value from this simplistic tracking comes from the low- hanging fruit: finding keywords that continually send visitors who convert to paying customers, and increasing focus on rankings and on improving the landing pages that visitors reach. While conversion rate tracking from keyword phrase referrals is certainly important, it's never the whole story.


5. Number of pages receiving at least one visit from search engines


Knowing the number of pages that receive search engine traffic is an essential metric for monitoring overall SEO performance. From this number, we can get a glimpse into indexation the number of pages from our site the engines are keeping in their indexes.


For most large websites (50,000+ pages), mere inclusion is essential to earning traffic, and this metric delivers a trackable number that's indicative of success or failure. As you work on issues like site architecture, link acquisition, XML sitemaps, and uniqueness of content and metadata, the trend line should rise, showing that more and more pages are earning their way into the engines' results. Pages receiving search traffic is, quite possibly, the best long tail metric around.


While other analytics data points are of great importance, those mentioned above should be universally applied to get the maximum value from your SEO campaigns.