Data Collector API

  • IDE - API Documentation

    Available commands navigate Navigate the browser to a URL. A 404 status code will throw a dead_page error by default. Use opt.allow_status to override this. Examples navigate(<url>); navigate(input.url); navigate('https://example....

  • Getting started

    The Data Collector product offers a JSON API to control and inspect your data collectors programmatically. Getting started: Signup (or login) to our control panel Insert target URL (website to collect data from) or choose a ready-...

  • Trigger a collector for batch collection method

    API endpoint: POST /dca/trigger?collector={ID_COLLECTOR} Parameters : collector={ID_COLLECTOR} - A unique identification of collector version=dev - (Optional) Trigger the development version of the collector name={human_name} - (...

  • Receive batch data

    API endpoint: GET /dca/dataset?id={id_dataset} Parameters : id={id_dataset} – An unique identification of data set. Payload : Header: Authorization Bearer Key -H "Authorization: Bearer API_TOKEN" Sample Response : Waitin...

  • Trigger a collector for real-time collection

    API endpoint: POST /dca/trigger_immediate?collector={ID_COLLECTOR} Parameters : collector={ID_COLLECTOR} - A unique identification of collector version=dev - (Optional) Trigger the development version of the collector name={human...

  • Receive data from real-time work collector

    API endpoint: GET /dca/get_result?response_id={ID_RESPONSE} Parameters : response_id={ID_RESPONSE} – An unique identification of response Payload : Header: Authorization Bearer Key -H "Authorization: Bearer API_TOKEN" Sam...

  • Choose a delivery type on request level

    Instead of creating duplicate collectors for each delivery type, you can choose a delivery type per job using API.   Step 1. Ensure that your collector is updated to the latest version to improve success rate.   Step 2. Set a del...

  • Errors

    – Request not found. one of your inputs is incorrect { "error": "request_not_found", "message": "This request wasn't found in our system. Either it's not a valid ID or the stored result has expired" }