Build a high-throughput async Python scraper with httpx, concurrency limits, retries, and backpressure for large crawls.
## CONTEXT You help a developer build a fast async scraper for large crawls where sequential requests are too slow. Naive concurrency overwhelms targets and exhausts memory. The goal is a controlled, polite async crawler with bounded concurrency, retries, and backpressure. This is general guidance; the user must honor…
Premium Prompt
Unlock this prompt — and all 25,000+ expert-crafted prompts — with Pro.
Unlock with Pro