With online content becoming more dynamic, are search engines able to categorize and serve up dynamic content?

Question

Are search engines like google and yahoo able to categorize a page that doesn't exist until variables are introduced? How does a spider work within a dynamic site?

Answer 1

They are able to do so.

Some search engines may shy away from sites that have variables in their URLs, such as http://mysite.com?page=1&stuff=4

You can use mod_rewrite in Apache to trick engines into crawling sites that have variables in thier URLs. For example, the above line would be rewritten as http://mysite.com/page/1/stuff/4.

Answer 2

Dynamic content is delivered to the Web browser in a different form than it exists on the server, while static content is stored on the Web server in the same format that is delivered to the Web browser. Dynamic site pages are generated from a database "on the fly" as users request them.

You can often tell when you are looking at a dynamically generated page, because dynamic URLs contain one or more "query strings," or question marks (?) while static URLs do not, but there are exceptions to this rule, which we shall discuss below.

Search engines have a hard time with dynamic URLs. Dynamic URLs may cause search engines to mistake a small site for a very large one because an unlimited number of URLs can be used to provide essentially the same content. This can cause search engine spiders to avoid such sites for fear of falling into "dynamic spider traps," crawling through thousands of URLs when only a few are needed to represent the available content.

Answer 3

Meta tags are not required if there is any way to reach the title of the dvd in text in the page or by using links in the page to get to a page that has the title. If you can only reach them by entering a search term, you will need meta tags. Might be a good idea to generate a few pages of titles automatically. Some search engines may downgrade your page if you have words in your metas that do not appear in the page.