Outscraper API Docs

/universal-scraper

Extracts the data you need from any web page by using AI.

Other Services
↗ Open in API Explorer
GET

Universal AI-Powered Scraper

Other Services

Extracts the data you need from any web page by using AI.

Request

Endpoint: GET https://api.outscraper.cloud/universal-scraper

Parameters

NameInTypeRequiredDescriptionExampleConstraints
query query array<string> Yes Links to web pages (e.g., https://www.apple.com/iphone/). It supports batching by sending arrays with up to 1000 queries (e.g., query=text1&query=text2&query=text3). It allows multiple queries to be sent in one request and to save on network latency time. - -
attributes query array<string> No The parameter defines attributes to parse from a web page (e.g., attributes=name,price,discount price,description). - -
limit query integer No The parameter specifies the limit of pages to return from one query. - default 1
fields query string No The parameter defines which fields you want to include with each item returned in the response. By default, it returns all fields. Use &fields=query,name to return only the specific ones. - -
async query boolean Yes The parameter defines the way you want to submit your task to Outscraper. It can be set to false to open an HTTP connection and keep it open until you got your results, or true (default) to just submit your requests to Outscraper and retrieve them later (usually within 1-3 minutes) with the Request Results endpoint. Each response is available for 4 hours after a request has been completed. A good practice is to send async requests and start checking the results after the estimated execution time. Check out this Python implementation as an example. As most of the requests take some time to be executed the async=true option is preferred to avoid HTTP requests timeouts. - default true
ui query boolean No The parameter defines whether a task will be executed as a UI task. This is commonly used when you want to create a regular platform task with API. Using this parameter overwrites the async parameter to true. - default false
format query array<string enum> No The parameter defines the format of the output data. Using this parameter overwrites the ui parameter to true. - -
webhook query string No The parameter defines the URL address (callback) to which Outscraper will create a POST request with a JSON body once a task/request is finished. Using this parameter overwrites the webhook from integrations. - -

Responses

200

application/json

The response contains the status of the request and data. Data is an array where each element represents a response for a single query from the request.

{
  "id": "your-request-id",
  "status": "Success",
  "data": [
    {
      "query": "https://www.apple.com/iphone/",
      "name": "iPhone",
      "price": "$799",
      "description": "The ultimate smartphone from Apple, with advanced features, powerful performance, and a sleek design."
    }
  ]
}

202

application/json

The response contains a request ID that can be used to fetch results by using Request Results endpoint. Each response is available for 4 hours after a request has been completed.

{
  "id": "your-request-id",
  "status": "Pending",
  "results_location": "https://api.outscraper.cloud/requests/your-request-id"
}

401

application/json

Wrong or missing API Key (token).

{
  "error": true,
  "errorMessage": "401 Unauthorized: The server could not verify that you are authorized to access the URL requested. You either supplied the wrong credentials (e.g. a bad password), or your browser doesn't understand how to supply the credentials required."
}

402

application/json

Past due invoices or a payment method not connected.

{
  "error": true,
  "errorMessage": "Please close past due invoices or verify your card information is correct."
}

422

application/json

Wrong query url parameters.

{
  "error": true,
  "errorMessage": "422 Unprocessable Entity: The request was well-formed but was unable to be followed due to semantic errors."
}

Code Samples

cURL

curl -X GET "https://api.outscraper.cloud/universal-scraper?query=https://www.apple.com/iphone/&attributes=name,price,description&async=false" -H  "X-API-KEY: YOUR-API-KEY"

cURL (async)

curl -X GET "https://api.outscraper.cloud/universal-scraper?query=https://www.apple.com/iphone/&attributes=name,price,description" -H  "X-API-KEY: YOUR-API-KEY"

Related Endpoints