The internet crawlers are referred to as bots or spiders. The search engine requires the internet pages it crawls and downloads them. In addition it discovers new articles mainly because it indexes new webpages and web-sites. The objective of Search engine optimisation marketing would be to be