The name given to automated harevsting program that scans web sites, searching for and feeding back links and information to search engines. This technique is also employed by Press Agencies to discover who is writing about their clients.
The "Spider" anaology comes from the fact that a
Spider crawls on a web (the web in this case being the
www )