crawler

crawler is a program used by search engine to make way through the internet to secure data, which is later indexed. The major search engines on the Web all have such a program, which is also known as a “spider” or a “bot.”

Leave a Reply

Your email address will not be published.