Pagination selector

Execution model

The Pagination selector is recursive. It continues discovering pagination targets, including links that become visible only on later pages. Place listing extraction selectors beneath it so they run on every discovered page.

Pagination types

  • Auto - Automatically selects a pagination method. Use this first and configure a specific type if detection is incorrect.
  • Link - Reads the pagination destination from an anchor href.
  • Scripted link - Reads a URL from JavaScript in an href or onclick attribute.
  • Attribute link - Reads the destination from another HTML attribute.
  • Text link - Reads the destination from text.
  • Link from any script - Reads a destination from page script.
  • Click multiple times on next/more button - Clicks a Next or Load more control until no new records are scraped.
  • Click once on multiple buttons - Clicks each unique matched pagination control once.

Recommended structure

_root
└── pagination
    └── item
        ├── name
        └── price

If every paginated page also loads records through scrolling, use a scrolling Element selector as the child of the Pagination selector. Because pagination is recursive, the child selector and its scrolling behavior execute on each discovered page.

_root
└── pagination
    └── items (Element, scrolling enabled)
        ├── name
        └── price

Fig. 1: Sitemap with Pagination selector

Validate the pagination selector on a later page

Always validate a pagination selector after moving to the second page. A control that is selected correctly on the first page may no longer be unique once the pagination state changes.

  1. Create the Pagination selector on the first page.
  2. Navigate manually to the second page.
  3. Preview or highlight the Pagination selector again.
  4. Confirm that it still matches the intended Next or arrow control and does not match Previous or another pagination element.

This is especially important when Next and Previous controls share a generic class. On the first page, only Next may be present, so a broad selector can appear correct. On the second page, the Previous control may use the same class and become the first matching element. The scraper can then follow the wrong control, revisit an earlier page, or fail to traverse pagination correctly.

If the selector matches Previous on page 2

Reselect the Next or arrow control while you are on the second page. With both Previous and Next controls present, the point-and-click selector tool has more page context and can create a selector that distinguishes the intended Next control from the Previous control.

After reselecting, preview the selector again on page 2 and, where practical, on another later page before running the sitemap.

Validation checklist

  • The selector matches the intended Next, arrow, numbered, or Load more control on later pages.
  • It does not switch to a Previous control when that control appears.
  • Later pages return records not present on the first page.
  • The current page is not rediscovered indefinitely.
  • Pagination reaches controls that are not visible initially.
  • Repeated clicking stops when no new records are added.

Related