This article has been excerpted from The Ultimate Guide to Search Engine Marketing by Jon Rognerud, available from EntrepreneurPress.com.

Search engine indexing is how information is collected, parsed, and stored in search engine databases to facilitate fast and accurate information retrieval. The search engine index design uses interdisciplinary theories from linguistics, cognitive psychology, mathematics, informatics, physics and computer science.

Search engine optimization is about creating a site that meets search engine criteria.

Automatic vs. Manual Submission
Search engines don't like automatic submissions. They get millions of attempted automatic submissions each day, and, as a result, go to great lengths to try to stop them, such as requiring passwords and/or fill-in details before the submission is accepted.

Automatic submission software is banned by many search engines and directories because it can't always place links in relevant categories. Whether you hire a directory submitter or use submission software yourself, you are setting your site up for failure. Getting too many incoming links too quickly or submitting the same site to the major engines repeatedly can be seen by search engines as spamming and get your site banned. There are few benefits to using a paid submission service, although the promotions tout that these services know of places to submit that you are unaware of. The best submission services take the time to educate their clients on submission with the expectation that they'll be able to do the job themselves next time.

Submitting your site manually gives you control over where you'll be listed. Automatic submitters may list your site with an engine you've never heard of outside the United States. Or, worse, they could be associating your site with sites that you find objectionable, such as pornographic sites. When you submit your own site, you decide what category to list in. You don't want your site to end up in the general directory or the miscellaneous category where it may never be seen.

Every site should be marketed differently, and submitted to different engines and directories depending on its intended audience. Automatic submission doesn't do this. Manual submission is the only way to choose the descriptions and keywords that you think will be most effective in promoting your site, as most engines and directories allow you to choose your own.

Common Myths of Search Engine Submission
Here are some common myths about submitting your site manually:

1. Search engine submission requires enormous research.
2. Almost every search engine and directory has a link labeled "Add a site" or "Submit URL," which takes you directly to their submission form. All you have to do is follow their rules and enter your information carefully.
3. Websites should be listed in all search engines.
4. If you list your website with the top-tier search engines, it will find its own way into the second tier and beyond. In fact, you don't even have to submit your site at all for it to be listed--search engine spiders are constantly combing the web and will eventually find your site on their own if your site contains links to other sites.
5. Websites should be listed on a weekly or monthly basis. Continually submitting your site can be classified as spamming and result in your site's banishment from a search engine's database. As a general rule, don't resubmit a new version of a page unless you have an unfavorable ranking.
6. Good submission is costly. The truth is you don't have to pay a submission service to submit your site to the search engines. Submission to most major search engines is free.
7. Search engines and directories you should submit your website to include:

• http://dmoz.org
• http://dir.yahoo.com
• http://www.joeant.com
• http://www.skaffe.com
• http://www.bluefind.com
• http://www.zeal.com
• http://www.goguides.org
• http://www.gimpsy.com
• http://www.wowdirectory.com
• http://www.sevenseek.com
• http://www.thisisouryear.com
• http://search.looksmart.com/Submission Service Claims

How Long Does It Take to Get Listed?
Here are estimated waiting periods for some of the more popular search engines:

• MSN up to two months
• Google up to four weeks
• AltaVista up to one week
• Fast up to two weeks
• Excite up to six weeks
• AOL up to two months
• HotBot up to two months
• iWon up to two months

How often should you resubmit? Technically, you don't have to resubmit. Once your site is indexed, simply keep adding search-friendly content and there's no need to esubmit. You should, however, be familiar with your site statistics. Look at your website statistics for robot visits.

Let the Spiders Do the Walking
Like search engines, directories can aid your web visibility. Following is a list of directories and your options for submission.

1. Yahoo! Directory. The first thing to do is submit your site to the Yahoo! Directory. This website contains a detailed compilation of websites that can be browsed by visitors through certain categories. Unlike Yahoo!'s search engine, Yahoo! Directory is powered by editors rather than robots.

What connection does the Yahoo! Directory have with the Yahoo! search engine? It's generally accepted that you have a greater chance of getting your site listed higher if you submit to the crawler-based Yahoo! search engine. This isn't a guarantee, but either way you have nothing to lose by submitting to the Yahoo! Directory.

To submit to the Yahoo! Directory, use their "standard" submission service or their "Yahoo! Directory Submit," which charges a fee. The standard submission service lets you submit your site in general categories at no extra cost. You're not allowed to submit to commercial categories. If you try to submit your site to a commercial category using the standard submission option, it won't be allowed. Instead, Yahoo! Directory will force you to upgrade to Yahoo! Directory Submit.

How does Yahoo! Directory Submit work? First, you must pay an annual fee of $299, for non-adult sites and $600 for adult sites. What you get, according to Yahoo! Directory, is an "expedited review" of your website(s)--that's right, you can submit more than one, though you have to pay $299 or $600 for each submission. You'll get a response within seven days indicating whether or not your site is accepted. If it's accepted you'll have to pay $299 or $600 once a year to keep your listing in the directory, at least if it's a commercial site (non-commercial sites aren't charged the recurring fee). The upside to the paid submission is that your site gets reviewed quickly; with the standard submission service, you may not get your site reviewed at all. The downside is that you're paying a lot of money for a service that can't guarantee you placement (although if you're promoting a business you must use the Yahoo! Directory Submit).

In terms of getting your site accepted, what matters is relative content. When it comes to the standard submission option, you want to ensure that your content is not commercial in any way. For example, if you're running an online clothing store, more likely than not this won't be acceptable with standard submission. However, if you submitted a site that talked about how to be fashionable with a certain collection of clothes, this would be accepted, even if the article provided a link to your website. Yahoo! Directory would choose the latter site because it's seen as being informative and more helpful than the actual store.

If you do use the Yahoo! Directory Submit and you choose not to pay the renewal fee (if it applies to you), your listing will be dropped by Yahoo! Directory. However, this may not have a negative impact on search engine crawlers. If you get an initial boost from the Yahoo! Directory, there's greater potential that others will want to link to you. This is even more true if you follow the link-building specifications laid out in this book. This may be enough to keep your high ranking with the other search engines, including Yahoo!. If it isn't, you can always submit your site again through Yahoo! Directory Submit.

To submit a site to the Yahoo! Directory, you need to visit dir.yahoo.com. If you want to use standard submission, select the "Suggest a Site" link at the top right corner of the non-commercial category page you want your site to be indexed to. Note: You'll have to look closely, since "Suggest a Site" is in a smaller font. You could use the same method to use Yahoo! Submit, or you could use the home page. The category you choose doesn't matter as much, because the editors working for Yahoo! Directory choose the category for you.

2. Yahoo! Search Engine. With your site submitted to the Yahoo! Directory, you're ready to submit your site to the Yahoo! search engine. If you performed your quality link- building work, Yahoo! should be able to pick you up automatically. Otherwise, you can use their URL submission form, which can be accessed by going to search.yahoo.com. Enter the URL you want to submit. You'll probably want to submit only your home page, though you can submit individual subtopic pages as well. When you submit the web page, be sure to include the whole URL including the "http://www." If you don't include http://www, the page will prompt you to enter the proper URL. Also, make sure that you don't submit your site twice, as you may get penalized.

Yahoo! also offers Search Submit. With Search Submit you must pay an annual subscription fee in addition to payment per click. Don't confuse this program with Yahoo! Sponsor Search, which lists your site in certain commercial categories. The monthly fee for Sponsor Search ranges from $5 to $30. These are pay-per-click programs that show ads on the right side of search listing results. You pay when visitors click on your ads. With Yahoo! Search Submit, your site is shown just like other normal search engine listings. However, you do have to pay for it. If you run out of money, you might lose your listing, though sometimes you won't.

Is it worth using Search Submit? It is in the sense that you can get your site indexed immediately. However, it offers no benefit when it comes to your ranking. The payment structure for this program is not as attractive as other pay-per-click programs, since in addition to PPC you also have a subscription fee. For this reason you may prefer to spend your money on Yahoo! Search or Google AdWords. In fact, these are a great way to test the advertising value of your keyword, at least if you want to see results before your site gets indexed.

3. Ask. Ask Jeeves doesn't offer a free URL submission page nor does it offer paid submission. However, the search engine periodically crawls the web, indexing sites on the basis of how many times they appear on other sites. So, again, there's value in building valid links.

4. MSN. MSN Search, also called Microsoft Live Search, doesn't offer free URL submission or paid submission. It uses its bot, MSNBot, to crawl the web in search of relevant sites.

5. AOL. AOL Search is powered by Google's search engine technology. So once you submit your site to Google (which we talk about in the next section), you also get your results shown in the AOL Search listings.

6. The Open Directory (DMOZ). The Open Directory is a special directory built by volunteers that serves as a guide to the internet. The Open Directory is now owned by Google. Many of the most popular search engines, including Google, pull results from and point to the Open Directory, so trying to make your website a part of it definitely doesn't hurt. Plus it's free. The only major problem with the Open Directory is that you won't get a guarantee that your site is accepted, nor can you estimate how long it will take to get a response (if you get one).

To submit to the Open Directory, you need to select the category you want your page to show up on. Use the "Suggest URL" link; it can be found on the category page near the top. You are then directed to a form that requires you to provide your site URL, title of the site, site description, and your e-mail address. It takes about three weeks for your site to show up in the directory, and if it doesn't you can resubmit.

Google's Guidelines
Meet Google, the "coolest kid" on the cyberblock. Google is popular, and popularity means it may be tough to get in initially. Even if you do everything right, it could take months to see results, at least if you use their URL submission page. However, there is hope. There's a method to get indexed in 24 hours, so don't even bother submitting through the URL page. But before you get to that, you should know about the guidelines you must follow to ensure that not only does your site get listed, but that it also doesn't get banned. Plus, you should learn about elements of your website that Google won't look at.

1.How to Get Google to Read Your Keywords First. Google's bots read your web pages from the topmost left corner of your site to the bottom right. However, most sites are designed with all of the links on the left side, and the content on the right. In fact, earlier in this book you learned that this is the recommended website design you should use. Yet the problem with this design is instead of seeing your content first, Google sees the links first. Your links may not be seen to be as optimized as your content.

One solution is to use three panes rather than two. Keep the normal left and right panes, but add an extra pane at the top left of the layout. Don't put keywords in this extra pane. With this area "blank" when the Google bots read the site, rather than going for the links as they normally would, the bots see that a portion of where the links are is "blank." This then forces it to read the content first, which is more keyword-rich than the links.

Note that not all search engines read sites this way, which is why this guideline was provided in this special section dedicated to optimizing for Google. You could be on the safe side and use the layout anyway, especially if you do plan to submit to Google, which you should. It doesn't take away from the look of the site, and by using it you ensure that your content gets read first. If you don't use it, you aren't giving yourself the best opportunity to rank highly in Google search engine listings. Making tables isn't very hard to do. Most word processors and even WYSIWYG HTML editors provide them, so take advantage of it.

2. Things That Google Ignores. There are some HTML attributes that Google pays no attention to when it goes through its crawling process. While you won't get penalized if you use these attributes, why waste your time with them if they're not going to count anyway?

Of course, there are exceptions to these rules, as noted in the numbered list below. There are also some elements listed that you will choose not to include.

1. The keywords and description attributes of the meta tag. The keywords and description attributes are read by other search engines. However, the boost you get from having them isn't as much as if you follow the other techniques, such as proper link building. If you submit to Google only, you may not want to include the keywords attribute, but focus on a smart "upsell" or "positioning," "branding" of your message in the description attribute. Other search engines use them, so you should go on and include them.
2. The tag. The comments tag is an optional tag designed more for the website designer than for search engines or browsers. You use it to make personal notes related to what the upcoming coding does. It's especially useful if other webmasters are working on web pages that have been started by someone else. Still, it isn't a necessary tag, so you can omit it if you want.