Why AgentQL?
AgentQL is a reliable, flexible, and intuitive query language for locating web elements. Developers can specify web elements easily with its natural language-like query syntax without diving into complex DOM structures or writing fragile XPath expressions. AgentQL can return data or web elements with queries and prompts.
This example demonstrates an AgentQL query that can be used locate a search box and its button on a web page.
You can pass an AgentQL query to one of two query methods: query_elements
returns elements, and query_data
, which returns the data in those elements.
AgentQL can also return an element from a prompt using its get_by_prompt
method:
Advantages over traditional methods
XPath and CSS selectors have been the go-to solutions for web scraping and automation, but they come with drawbacks:
- Fragility: Traditional selectors break easily when websites update their structure.
- Complexity: XPath and complex CSS selectors can be difficult to write and maintain.
- Lack of Context: Traditional selectors don't understand the meaning or purpose of elements.
AgentQL lets you spend less time updating broken selectors and more time extracting data and building automations by solving for these problems:
- Robustness: AgentQL adapts to changes in website structure automatically
- Simplicity: AgentQL queries use natural language descriptions instead of rigid syntax
- Context-aware: AgentQL uses AI to understand the context and meaning of web elements
Features
Semantic Selectors
AgentQL uses AI to build a semantic understanding of the context surrounding the web elements on a page. Elements are found based on their meaning and context, not just their position in the DOM, making them more:
- Stable even when website layouts change over time.
- Reusable across sites, standardizing outputs.
- Intuitive for both developers and non-technical users to write
Natural Language Queries
With AgentQL, you describe what you're looking for in plain English and can even pass entire prompts to explain in great detail what you’re looking for on a page. This makes queries more readable across teams and time and more maintainable.
Controlled Output
Your query defines the structure of your output data, eliminating post-processing steps.
Deterministic Results
AgentQL not only allows you to define an exact response structure, but also provides consistent and reliable results. You will get the same output for the same input, every time. This lets you confidently automate processes and tests.
Primary Use Cases
AgentQL is useful for data scraping and extraction, web automation, and testing.
Data Scraping
AgentQL can extract structured data from websites:
- Gather pricing details from multiple storefronts
- Collect social media metrics
- Aggregate news and articles from multiple sources
Web Automation
Streamline repetitive tasks and complex workflows:
- Automate form submissions
- Interact with web applications programmatically
- Create powerful web-based bots and agents
End-to-End Testing
Build more robust and maintainable test suites:
- Write tests that are resistant to UI changes
- Reduce test flakiness and maintenance overhead
Conclusion
AgentQL can change how you interact with web content. If you’d like to dive in, check these out: