Data export
Automatic data export
Connect a supported destination, configure Data Export on the sitemap, and Web Scraper Cloud sends the complete dataset after each scraping job finishes. Automatic export is not continuous while the job is running.
Supported destinations are:
- Google Sheets
- Google Drive
- Dropbox
- Google Cloud
- Microsoft Azure
- Amazon S3
Configure automatic export
- Open the sitemap's Data Export configuration.
- Select the destination.
- Connect or synchronize the required account.
- Configure the destination and available export options.
- Save the configuration.
- Run a job once and confirm that the dataset arrives in the expected location.
Scheduler and automatic Data Export can be combined for unattended recurring delivery.
Manual download
Scraping-job data can also be downloaded manually. If a download is created while a job is still running, it contains only records completed at the time of the download.
Supported data formats
| Environment | Formats |
|---|---|
| Web Scraper browser extension | CSV, XLSX |
| Web Scraper Cloud | CSV, XLSX, newline-delimited JSON |
| Cloud API download | CSV, JSON |
Automatic-export formats depend on the selected integration.
Parser and exported data
When Parser rules are configured, validate the final parsed schema before relying on automatic exports. Check column names, removed and Virtual columns, transformed values and required date/number formats.
File format notes
XLSX
- Cells are limited to 32,767 characters; longer content is truncated.
- Worksheets are limited to 1,000,000 rows; larger datasets are split across worksheets.
JSON
Cloud JSON output is newline-delimited JSON: each line contains one JSON object. Newline characters inside field values are escaped.
CSV
CSV output uses standard escaping rules and fields can contain line breaks. When processing CSV programmatically, use a CSV parser rather than splitting the file by text lines.
