Python Web Scraper Builder

Build a Python web scraper for the following use case:

Target Website Type: [E-COMMERCE/NEWS/SOCIAL MEDIA/DIRECTORY/API-BACKED]
Data to Extract: [PRODUCT LISTINGS/ARTICLES/CONTACT INFO/PRICING/REVIEWS]
Output Format: [CSV/JSON/DATABASE/EXCEL]
Volume: [SINGLE PAGE/MULTI-PAGE/ENTIRE SITE]
Authentication Required: [YES/NO]
JavaScript Rendering Needed: [YES/NO]

Provide a complete scraper implementation covering:
1. Library selection rationale (requests, BeautifulSoup, Scrapy, Selenium, Playwright) and when to use each approach based on site complexity and JavaScript requirements
2. Initial setup with proper headers, user-agent rotation, and session management to avoid detection and blocking by target servers
3. HTML parsing strategy with CSS selectors and XPath expressions for reliable data extraction, including fallback selectors for dynamic page structures
4. Pagination handling with support for numbered pages, infinite scroll, and cursor-based navigation patterns across multi-page result sets
5. Rate limiting implementation using exponential backoff, random delays between requests, and concurrent request throttling to respect server resources
6. Error handling for network timeouts, HTTP status codes (403, 429, 500), connection resets, and malformed HTML with automatic retry logic and dead letter queues

Include code examples for each section with inline comments explaining the design decisions. Add a configuration file approach for easy target URL and selector changes without modifying core scraper logic.