Map API

The Map API discovers all accessible URLs within a website, creating a comprehensive sitemap of your target domain. Perfect for SEO audits, content inventory, migration planning, and understanding website architecture.

Authentication

Authorization

string

required

Bearer token using your API key: Bearer YOUR_API_KEY

Request Body

url

string

required

The root URL to begin mapping. Must be a valid HTTP/HTTPS URL.

options

object

Configuration options for mapping behavior

Show options properties

options.maxDepth

number

default:"2"

Maximum crawling depth from the starting URL (1-5)

Depth 1: Only links directly on the starting page
Depth 2: Links found on pages from depth 1
Higher depths discover more pages but take longer

options.maxPages

number

default:"100"

Maximum number of pages to discover (1-1000)Stops mapping once this limit is reached

options.sameDomain

boolean

default:"true"

Only map pages within the same domain as the starting URLWhen true, external links are ignored

options.includePatterns

array

URL patterns to include (supports wildcards)Example: ["/blog/*", "/products/*"]

options.excludePatterns

array

URL patterns to exclude (supports wildcards)Example: ["/admin/*", "*.pdf", "/api/*"]

options.respectRobots

boolean

default:"true"

Respect robots.txt rules when mappingWhen true, pages disallowed in robots.txt are skipped

options.search

string

Search query to filter discovered URLsOnly URLs containing this string will be included in results

options.timeout

number

default:"30000"

Maximum time to wait for each page load in milliseconds (1000-60000)Pages that don’t load within this time are skipped

options.engine

string

default:"auto"

Scraping engine to use for URL discovery

auto - Automatically selects best engine (default)
playwright - Full browser automation with JavaScript
lightweight - Fast HTTP-based discovery (no JS)

options.includeMetadata

boolean

default:"false"

Extract title and description metadata for discovered URLsWhen true, performs lightweight HTTP requests to fetch page metadata (title and meta description tags) for each discovered URL. This provides richer information about each page but slightly increases processing time and credit cost.Credit Impact: No additional credits - metadata extraction is included in base mapping costPerformance: Adds ~200-500ms per URL with concurrent processing of 3 URLs at a timeBest for: Content inventory, SEO audits, and comprehensive site documentation

options.metadataLimit

number

default:"10"

Maximum number of URLs to extract metadata for (1-50)When includeMetadata is true, this limits how many URLs receive full metadata extraction. Useful for large sites where you only need detailed information for the most important pages.Example: Set to 20 to get metadata only for the first 20 discovered URLsRemaining URLs: Still included in results but with title and description as undefined

Response

success

boolean

Indicates if mapping was successful

data

object

Mapping results and discovered URLs

Show data properties

domain

string

The domain that was mapped

rootUrl

string

The starting URL used for mapping

totalPages

number

Total number of unique URLs discovered

maxDepthReached

number

Maximum crawl depth reached during mapping

urls

array

Array of discovered URLs with metadata

Show url properties

url

string

The discovered URL

depth

number

Crawl depth at which this URL was found (0 = starting page)

statusCode

number

HTTP status code (200, 301, 404, etc.)

title

string

Page title if availableAlways present when includeMetadata: true, otherwise undefined

description

string

Meta description extracted from the pageOnly present when includeMetadata: true is specified in options. Provides a brief summary of the page content extracted from meta description tags (including og:description and twitter:description). Truncated to 200 characters for consistency.

discoveredAt

string

ISO timestamp when URL was discovered

parentUrl

string

URL of the page where this link was found

summary

object

Mapping statistics and summary

Show summary properties

totalUrls

number

Total URLs discovered

successfulUrls

number

URLs that returned 200 status

failedUrls

number

URLs that returned error status

redirectUrls

number

URLs that returned redirect status

mappingTime

number

Total time taken for mapping in seconds

creditsUsed

number

Credits consumed for this mapping operation

Examples

Basic Website Mapping

curl -X POST "https://api.whizo.ai/v1/map" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "options": {
      "maxDepth": 2,
      "maxPages": 100,
      "sameDomain": true
    }
  }'

{
  "success": true,
  "data": {
    "domain": "example.com",
    "rootUrl": "https://example.com",
    "totalPages": 45,
    "maxDepthReached": 2,
    "urls": [
      {
        "url": "https://example.com",
        "depth": 0,
        "statusCode": 200,
        "title": "Example Domain - Home",
        "discoveredAt": "2025-01-15T10:00:00Z",
        "parentUrl": null
      },
      {
        "url": "https://example.com/about",
        "depth": 1,
        "statusCode": 200,
        "title": "About Us",
        "discoveredAt": "2025-01-15T10:00:05Z",
        "parentUrl": "https://example.com"
      },
      {
        "url": "https://example.com/contact",
        "depth": 1,
        "statusCode": 200,
        "title": "Contact Us",
        "discoveredAt": "2025-01-15T10:00:06Z",
        "parentUrl": "https://example.com"
      }
    ],
    "summary": {
      "totalUrls": 45,
      "successfulUrls": 42,
      "failedUrls": 2,
      "redirectUrls": 1,
      "mappingTime": 12.5,
      "creditsUsed": 5
    }
  }
}

Advanced Mapping with Pattern Filtering

curl -X POST "https://api.whizo.ai/v1/map" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://blog.example.com",
    "options": {
      "maxDepth": 3,
      "maxPages": 200,
      "includePatterns": ["/blog/*", "/articles/*"],
      "excludePatterns": ["/admin/*", "*.pdf", "/api/*"],
      "respectRobots": true,
      "engine": "playwright"
    }
  }'

{
  "success": true,
  "data": {
    "domain": "blog.example.com",
    "rootUrl": "https://blog.example.com",
    "totalPages": 87,
    "maxDepthReached": 3,
    "urls": [
      {
        "url": "https://blog.example.com",
        "depth": 0,
        "statusCode": 200,
        "title": "Blog Home",
        "discoveredAt": "2025-01-15T10:00:00Z"
      },
      {
        "url": "https://blog.example.com/blog/getting-started",
        "depth": 1,
        "statusCode": 200,
        "title": "Getting Started Guide",
        "discoveredAt": "2025-01-15T10:00:02Z",
        "parentUrl": "https://blog.example.com"
      },
      {
        "url": "https://blog.example.com/articles/web-scraping-101",
        "depth": 1,
        "statusCode": 200,
        "title": "Web Scraping 101",
        "discoveredAt": "2025-01-15T10:00:03Z",
        "parentUrl": "https://blog.example.com"
      }
    ],
    "summary": {
      "totalUrls": 87,
      "successfulUrls": 85,
      "failedUrls": 1,
      "redirectUrls": 1,
      "mappingTime": 34.2,
      "creditsUsed": 9
    }
  }
}

curl -X POST "https://api.whizo.ai/v1/map" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "options": {
      "maxDepth": 3,
      "search": "product",
      "maxPages": 50
    }
  }'

Mapping with Metadata Extraction

Extract rich metadata (title + description) for discovered URLs:

curl -X POST "https://api.whizo.ai/v1/map" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "options": {
      "maxDepth": 2,
      "maxPages": 50,
      "includeMetadata": true,
      "metadataLimit": 20
    }
  }'

{
  "success": true,
  "data": {
    "domain": "example.com",
    "rootUrl": "https://example.com",
    "totalPages": 35,
    "maxDepthReached": 2,
    "urls": [
      {
        "url": "https://example.com",
        "depth": 0,
        "statusCode": 200,
        "title": "Example Domain - Leading Provider of Web Services",
        "description": "Example Domain offers comprehensive web hosting, domain registration, and cloud solutions for businesses of all sizes. Get started today!",
        "discoveredAt": "2025-01-15T10:00:00Z",
        "parentUrl": null
      },
      {
        "url": "https://example.com/about",
        "depth": 1,
        "statusCode": 200,
        "title": "About Us - Our Story and Mission",
        "description": "Learn about Example Domain's 10-year journey in providing reliable web hosting services to over 50,000 customers worldwide.",
        "discoveredAt": "2025-01-15T10:00:05Z",
        "parentUrl": "https://example.com"
      },
      {
        "url": "https://example.com/services",
        "depth": 1,
        "statusCode": 200,
        "title": "Our Services - Web Hosting & Cloud Solutions",
        "description": "Explore our range of web hosting plans, cloud storage options, and managed services designed to help your business thrive online.",
        "discoveredAt": "2025-01-15T10:00:06Z",
        "parentUrl": "https://example.com"
      }
    ],
    "summary": {
      "totalUrls": 35,
      "successfulUrls": 33,
      "failedUrls": 1,
      "redirectUrls": 1,
      "mappingTime": 15.8,
      "creditsUsed": 4
    }
  }
}

Mapping Strategies

Shallow vs Deep Mapping

Shallow Mapping (depth 1-2)

Fast discovery of main pages
Good for homepage and primary navigation
Lower credit cost
Recommended for initial site exploration

{
  "url": "https://example.com",
  "options": {
    "maxDepth": 1,
    "maxPages": 50
  }
}

Deep Mapping (depth 3-5)

Comprehensive site discovery
Finds nested content and deep pages
Higher credit cost
Recommended for full site audits

{
  "url": "https://example.com",
  "options": {
    "maxDepth": 4,
    "maxPages": 500
  }
}

Pattern-Based Mapping

Use patterns to focus on specific sections:

{
  "url": "https://ecommerce.com",
  "options": {
    "includePatterns": [
      "/products/*",
      "/categories/*",
      "/collections/*"
    ],
    "excludePatterns": [
      "/cart",
      "/checkout",
      "/account/*"
    ]
  }
}

Credit Costs

Mapping costs vary based on pages discovered:

Feature	Cost
Basic mapping (per page)	0.1 credits
JavaScript rendering	+0.05 credits per page
Deep crawling (depth 4-5)	+0.02 credits per page

Examples:

Mapping 100 pages at depth 2: 10 credits
Mapping 500 pages with JavaScript: 75 credits
Deep mapping (depth 5) of 200 pages: 24 credits

Error Responses

error

object

Show error properties

code

string

Error code identifier

message

string

Human-readable error message

details

object

Additional error context

Common Errors

Status Code	Error Code	Description
400	`invalid_url`	The starting URL is invalid or unreachable
400	`invalid_options`	Mapping parameters are invalid
401	`unauthorized`	Invalid or missing API key
402	`insufficient_credits`	Not enough credits for mapping
429	`rate_limited`	Rate limit exceeded
500	`mapping_failed`	Mapping process failed

{
  "success": false,
  "error": {
    "code": "invalid_url",
    "message": "The starting URL is not accessible",
    "details": {
      "url": "https://invalid-site.com",
      "statusCode": 404,
      "reason": "Page not found"
    }
  }
}

Rate Limits

Map API rate limits by plan:

Free: 2 mappings per hour, 10 per day
Starter: 10 mappings per hour, 50 per day
Pro: 50 mappings per hour, 200 per day
Enterprise: Custom limits

Use Cases

SEO Audits

Discover all pages for comprehensive SEO analysis:

Find all URLs for indexing status check
Identify orphaned pages without internal links
Detect broken links and 404 errors
Map site structure for optimization

Content Inventory

Create complete content inventories:

List all blog posts and articles
Catalog product pages
Document landing pages
Track content growth over time

Migration Planning

Plan website migrations and redirects:

Map existing URL structure
Create redirect mapping
Identify pages to migrate
Plan new site architecture

Competitive Analysis

Understand competitor site structures:

Discover competitor pages
Analyze site organization
Identify content strategies
Find competitive opportunities

Security Audits

Identify all accessible pages:

Find all public endpoints
Discover admin pages
Locate sensitive URLs
Map attack surface

Best Practices

Start shallow, go deeper if needed - Begin with depth 1-2 for quick overview
Use pattern filtering - Focus on relevant sections with includePatterns
Respect robots.txt - Keep respectRobots true for ethical scraping
Set reasonable limits - Use maxPages to control costs and runtime
Choose appropriate engine - Use lightweight for fast discovery, playwright for JavaScript-heavy sites
Monitor progress - Check summary statistics in response for insights
Use search filters - Narrow results with search parameter when looking for specific content
Handle external links - Set sameDomain based on your needs

Performance Tips

Optimal Settings by Site Size

Site Size	maxDepth	maxPages	Engine	Est. Time
Small (< 50 pages)	3	50	lightweight	10-20 sec
Medium (50-200 pages)	2-3	200	lightweight	30-60 sec
Large (200-1000 pages)	2	500	lightweight	2-5 min
Enterprise (1000+ pages)	2	1000	playwright	5-15 min

Speeding Up Mapping

Use engine: 'lightweight' for fastest discovery
Reduce maxDepth to discover fewer levels
Use includePatterns to focus on specific sections
Set lower timeout for faster failure handling
Avoid JavaScript rendering when not needed

Scrape API - Scrape individual pages
Crawl API - Crawl and extract content from multiple pages
Search API - Search and scrape results
Jobs API - Monitor mapping job progress

Core APIs

Job Management

User Management

Advanced Features

Authentication

Request Body

Response

Examples

Basic Website Mapping

Advanced Mapping with Pattern Filtering

Mapping with Search Filter

Mapping with Metadata Extraction

Mapping Strategies

Shallow vs Deep Mapping

Pattern-Based Mapping

Credit Costs

Error Responses

Common Errors

Rate Limits

Use Cases

Best Practices

Performance Tips

Optimal Settings by Site Size

Speeding Up Mapping

Core APIs

Job Management

User Management

Advanced Features

​Authentication

​Request Body

​Response

​Examples

​Basic Website Mapping

​Advanced Mapping with Pattern Filtering

​Mapping with Search Filter

​Mapping with Metadata Extraction

​Mapping Strategies

​Shallow vs Deep Mapping

​Pattern-Based Mapping

​Credit Costs

​Error Responses

​Common Errors

​Rate Limits

​Use Cases

​Best Practices

​Performance Tips

​Optimal Settings by Site Size

​Speeding Up Mapping

​Related Endpoints

Authentication

Request Body

Response

Examples

Basic Website Mapping

Advanced Mapping with Pattern Filtering

Mapping with Search Filter

Mapping with Metadata Extraction

Mapping Strategies

Shallow vs Deep Mapping

Pattern-Based Mapping

Credit Costs

Error Responses

Common Errors

Rate Limits

Use Cases

Best Practices

Performance Tips

Optimal Settings by Site Size

Speeding Up Mapping

Related Endpoints