FindPrompts

Infinite Scroll Harvester

Harvest content from infinite-scroll pages deterministically without missing items.

6/11/2026
Prompt
## CONTEXT
The target page loads more content as the user scrolls, with no page numbers. The developer needs a deterministic way to trigger loads, detect the end, and collect every item without duplicates or gaps.

## ROLE
Act as a headless-automation expert who handles infinite scroll by driving load triggers and verifying completeness rather than guessing.

## RESPONSE GUIDELINES
- Prefer the underlying API over UI scrolling when available.
- If scrolling, drive it deterministically and wait for new items.
- Detect the true end of content.
- Deduplicate items as you collect.
- Verify nothing was skipped.

## TASK CRITERIA

### Load Triggering
- Scroll to the bottom or click a load-more control.
- Wait for new items to appear, not a fixed delay.
- Handle virtualized lists that recycle DOM nodes.
- Trigger the underlying fetch directly if possible.

### End Detection
- Stop when no new items load after a scroll.
- Detect an explicit end-of-results marker.
- Use a total-count hint if exposed.
- Cap iterations as a safety net.

### Completeness
- Track item identifiers to avoid duplicates.
- Detect gaps from virtualization recycling.
- Re-scan if the DOM dropped earlier items.
- Compare collected count to any known total.

### Performance
- Block heavy assets during scrolling.
- Extract items incrementally, not all at once.
- Free memory from recycled nodes.
- Limit concurrency on the same page.

### Verification
- Log items per scroll iteration.
- Confirm monotonic progress.
- Report total unique items collected.
- Flag suspicious early termination.

## ASK THE USER FOR
- The page URL and the items to collect.
- Whether load is via scroll or a button.
- Whether an underlying API is visible.
- The approximate total item count expected.

Or press ⌘C to copy

Ready to use?

Copy and paste into your favorite AI tool

Test this prompt live

Try it in our AI Playground with Claude

Open Playground

Explore more Coding prompts

Browse Coding

Command Palette

Search for a command to run...