Skip to content

dpsearch-4.43-07092006

CrawlDelay command has been added. Use it to specify default pause in seconds between consecutive fetches from same server. This is similar to crawl-delay command used in robots.txt file, but can specified in indexer.conf file on per server basis. If no crawl-delay value is specified in robots.txt, the value of CrawlDelay is used. If crawl-delay is specified in robots.txt, then the maximum of CrawlDelay and crawl-delay is used as interval between consecutive fetches.

The Neo PopRank can now be calculated using several indexer threads (ex.: "indexer -TRN4"). This allow speed-up Neo PopRank calculation if several simulateneous connections to SQL server are allowed.

Leave a Reply

Your email address will not be published.