Home
  • /
  • Developer documentation

Crawl API

Identify the technologies used on any public website in real-time. Results are always guaranteed up-to-date and delivered within minutes.

This API is asynchronous: results are sent to a callback URL after the domain has been indexed.

Endpoint

GET https://api.wappalyzer.com/crawl/v1/

Properties

Property Description
Execution Asynchronous
Request timeout 6s
Rate limit 1 request / second

Parameters

Name Required Description Example
url Yes URL of the first web page to analyze https://example.com
callback_url Yes A POST request will be made to the callback URL upon completion of the request https://example.com

Examples

Example request

curl -H "x-api-key: <your api key>" "https://api.wappalyzer.com/crawl/v1/?url=https://example.com&callback_url=https://example.com"

Example response (202)

{
  "status": "received"
}

Example callback response (success)

The callback URL will receive a POST request when results become available.

{
  "url": "https://example.com",
  "applications": [
    {
      "name": "Craft CMS",
      "confidence": "100",
      "version": "",
      "icon": "CraftCMS.svg",
      "website": "https://craftcms.com",
      "categories": {
        "1": "CMS"
      }
    }
  ],
  "status": "success"
}

Example callback response (error)

An error type and message is returned if no URLs could be fetched.

{
  "url": "https://example.com",
  "applications": [],
  "status": "error",
  "errors": [
    {
      "type": "RESPONSE_NOT_OK",
      "message": "Response was not ok"
    }
  ]
}

Subscribe to receive occasional product updates.