Apify·Async

AI Web Scraper

Name: AI Web Scraper
Brand: APIFY

Knowledge & Data$0.025 / Page extraction

What it does

AI-first web scraper that extracts structured data from any website using natural-language prompts. No programming knowledge required. No hard-coded logic that breaks when a website changes.

Returns: AI-first web scraper that extracts structured data from any website using natural-language prompts. No programming knowledge required. No hard-coded logic that breaks when a website changes.

Parameters

Parameter	Type	Required	Description
`prompt`	textarea	No	Natural language description of which items to find and what to extract from each. If empty, the Actor just returns each start URL's content as Markdown (no crawling).
`startUrls`	array	Yes	URLs to start crawling from.
`maxCrawlDepth`	number	No	How many link hops away from the Start URL the crawler may follow.
`extractionMode`	select	No	Single page: Extracts data relevant to the prompt only from Start URLs. Links are not followed, maximum crawl depth is ignored. Best when you already have the exact pages and just want structured data. Scout: Uses sitemaps to find pages relevant to your prompt (up to Max pages to visit) and returns their markdown content. Maximum crawl depth is ignored. Fails if it can’t get the sitemap. Agentic: Discovers pages relevant to your prompt and extracts the most suitable structured content. Supports custom data structures, complex navigation and extraction prompts. Best when the pages you want are buried deep in the site.
`maxPagesToVisit`	number	No	Applied per Start URL. So the overall cap of pages to visit is this number × the number of Start URLs (e.g. 10 Start URLs × 5 = up to 50 pages). Ignored in Single page mode. Number of relevant results may be smaller.

Endpoints

GET/v1/agents/resources/inspect/apify.apify.ai-web-scraperPublic

POST/v1/agents/resources/estimate/apify.apify.ai-web-scraperPublic

POST/v1/agents/resources/run/apify.apify.ai-web-scraperAgent key

GET/v1/agents/resources/runs/{run_id}Agent key

Quick start

Run it with one Agent One Key over the OpenAI-compatible resource rail.

curl -X POST https://api.deva.me/v1/agents/resources/run/apify.apify.ai-web-scraper \
  -H "Authorization: Bearer deva_your_key_here" \
  -H "Content-Type: application/json" \
  --data '{
    "params": {}
  }'

Related in Knowledge & Data

Zillow Detail Scraper

Get details of Zillow properties from URLs. This Actor can be easily integrated with other Zillow Scrapers.

Linkedin Jobs Scraper

Scrape Linkedin jobs from jobs search results using search URLs or filters. Get full job details, job poster and company details.

🔥 LinkedIn Jobs Scraper

ℹ️ Designed for both personal and professional use, simply enter your desired job title and location to receive a tailored list of job opportunities. Try it today!

Posts Search Scraper for LinkedIn | No Cookies

Scrape LinkedIn posts by keyword without login. Get post content, reactions, author details, and media. Sort by relevance or date. Perfect for research, analysis, and monitoring trends.

Google News Scraper

Scrape Google News by keyword, topic or date range, past the 100-result limit. Get headlines, sources, dates, real publisher URLs and images as JSON, CSV or Excel, or via API, MCP or n8n. For news monitoring and alerts, brand tracking, market research, stock trading and AI agents.

Indeed Scraper

Scrape jobs posted on Indeed. Get detailed information from this job portal about saved and sponsored jobs. Specify the search based on location with the output attributes position, location, and description.