Getting data from HTML with AgentQL's REST API

AgentQL’s REST API enables powerful, flexible data retrieval from HTML files in a structured format, ready for seamless integration into your workflow.

Overview

This guide shows you how to use the REST API to extract data from raw HTML and retrieve structured data in JSON format with AgentQL queries.

Defining the REST API request structure

The following fields outline the high-level structure of a data querying request:

html: The raw HTML to query data from.
query: The AgentQL query to execute. Learn more about how to write an AgentQL query in the docs.
params: (Optional) Additional settings for enhanced data retrieval, such as enabling screenshots or scrolling. See the API Reference for more details about params.

The REST API also accepts url, which you can use to scrape a live web page.

Constructing the API request

To perform a basic data scraping request, start by defining the url of the desired webpage and the query to specify the data you want to retrieve in the request body.

Example REST API Request

Below is an example request body structure:

request_body

json

{
  "query": "{ page_title }",
  "html": "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n    <title>Simple Web Page</title>\n</head>\n<body>\n    <h1>Main Page</h1>\n</body>\n</html>"
}

Setting Request Headers

Before making the API request, include the necessary headers for authentication and content type. These headers authorize the request and specify the data format to send.

X-API-Key: this header should have your AgentQL API key for authentication.
Content-Type: set it to application/json to indicate that the request body is in JSON format, allowing the server to interpret the data correctly.

Making the API Request

Using your preferred HTTP client (like curl, Postman, or an HTTP library in Python or your preferred language), you can make a POST request to the AgentQL REST API endpoint.

terminal

curl -X POST https://api.agentql.com/v1/query-data \
  -H "X-API-Key: $AGENTQL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "{ page_title }",
    "html": "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n    <title>Simple Web Page</title>\n</head>\n<body>\n    <h1>Main Page</h1>\n</body>\n</html>",
  }'

note

Make sure to replace $AGENTQL_API_KEY with your actual API key.

Reviewing the API Response

If the request is successful, the API will return a JSON response with the extracted data.

Example Response

response

json

{
  "page_title": "Simple Web Page"
}

You can read more about the response structure and metadata fields in the API Reference.

note

To get more familiar with the AgentQL's REST API and other params options, check out the API Reference.