ParallelCrawlerEngine » Developer.Team

AbotX 2.1.3 Ultimate - A powerful C# web crawler

AbotX 2.1.3 Ultimate - A powerful C# web crawler
AbotX 2.1.3 Ultimate - A powerful C# web crawler


A powerful C# web crawler that makes advanced crawling features easy to use. AbotX builds upon Abot C# Web Crawler Framework by providing a powerful set of wrappers and extensions.

Features

Crawl multiple sites concurrently (ParallelCrawlerEngine)
Pause/resume live crawls (CrawlerX & ParallelCrawlerEngine)
Render jаvascript before processing (CrawlerX & ParallelCrawlerEngine)
Simplified pluggability/extensibility (CrawlerX & ParallelCrawlerEngine)
Avoid getting blocked by sites (AutoThrottling)
Automatically tune speed/concurrency (AutoTuning)
Read more