JavaScript content does not load
Confirm how the content loads
Reload the page and check whether the target content is available immediately or only appears after JavaScript, scrolling, or clicking.
Cloud driver
Use the FullJS driver when the target content is not present in the initial HTML. The Fast driver does not execute page JavaScript.
Delay
Web Scraper waits for network activity by default, but only for requests to the same domain. If content is loaded from an external domain, a delay may be needed. Otherwise, keep the default delay settings.
Interaction
- Enable Element selector scrolling for infinite-scroll records.
- Use a Pagination selector for load more controls.
- Use an Element click selector for tabs, variants and buttons.
Check the page source
Some values may already exist in the HTML even if they only become visible after an interaction, such as a click. Check whether the value can be extracted directly before adding extra interactions to the sitemap.