CrawlioCrawlio Docs

Method Mode

The problem

Code mode gives your AI the right primitives. Your AI still composes them wrong.

A page analysis requires three HTTP calls: trigger a WebKit capture, poll for enrichment data with backoff, and check crawl status. Every invocation reinvents this composition. Some models get the polling right. Some sleep for 30 seconds and hope for the best. Some forget to check whether enrichment data arrived.

This is the anti-pattern:

trigger_capture({ url: "..." })     // POST /capture -> 202
// ... ad hoc sleep ...
get_enrichment({ url: "..." })      // GET /enrichment -> maybe empty
// ... retry? give up?

Method mode fixes this.

What method mode adds

Method mode is a layer above code mode. It does not replace code mode. The raw tools, the search/execute surface, and all existing primitives remain available. Method mode adds composite tools that collapse multi-step workflows into single calls with normalized output.

Layer What it does Behavioral variance
Code Mode Token-efficient search + execute Medium (your AI picks endpoints, composes ad hoc)
Method Mode Composite tools with normalized output Low (proper polling, structured results)
Evidence Mode Typed findings, gap tracking Minimal (enforced schema, confidence propagation)

Composite tools

analyze_page

One call. One evidence record.

Field Value
Input { url: string }
Output { url, timestamp, captureTriggered, enrichment, enrichmentStatus, crawlStatus }
Timeout 60s

Internal steps:

  1. POST /capture triggers WebKit runtime capture
  2. Polls GET /enrichment?url= with exponential backoff (500ms, 1s, 2s, 4s, 4s, 4s)
  3. GET /crawled-urls?url=&limit=1 checks crawl status
  4. Composes a unified evidence record

The enrichmentStatus field is always present: "ok" when enrichment data arrived, "timeout" when polling exhausted without a response. Check this field before treating enrichment data as complete.

compare_pages

Side-by-side comparison of two pages.

Field Value
Input { urlA: string, urlB: string }
Output { siteA, siteB, comparisonSummary }
Timeout 120s

Runs analyze_page sequentially on both URLs. Returns both evidence records plus a typed comparison summary:

{
  "comparisonReadiness": "ready",
  "symmetric": true,
  "degradationNotes": [],
  "timingDelta": {
    "totalMsDelta": 1234.5,
    "captureA": 500.0,
    "captureB": 600.0
  },
  "evidenceIdA": "obs_a1b2c3d4",
  "evidenceIdB": "obs_e5f6g7h8"
}

The summary provides:

  • comparisonReadiness: ready (both complete), cautious (one side partial), unreliable (either side degraded)
  • symmetric: whether both sides have the same gap profile
  • degradationNotes: what data is missing and why
  • evidenceId values for round-trip verification via get_observation

synthesize_openapi

Generates an OpenAPI 3.0.3 spec from intercepted network traffic.

Field Value
Input { exhaustDir?, title?, serverURL? }
Output OpenAPI YAML
Timeout 120s

Chains traffic analysis, schema extraction, and OpenAPI export into one call.

When to use composite tools vs raw calls

Scenario Use
Analyze a page's framework, performance, and errors analyze_page
Compare two sites side-by-side compare_pages
Generate an API spec from traffic synthesize_openapi
Start a crawl execute_api or start_crawl (raw)
Update settings execute_api or update_settings (raw)
Quick status check execute_api or get_crawl_status (raw)

Use composite tools when you need multi-step evidence acquisition with proper polling. Use raw tools for single-step operations.

Browser-side methods

On the browser side, the smart object provides higher-order methods that compose CDP browser commands into reliable evidence acquisition primitives.

Method What it replaces What it adds
extractPage() Individual capture + manual perf/security/meta calls 7 parallel operations, typed gap records on failure
comparePages(urlA, urlB) Two-navigate-two-extract with inconsistent output Sequential extraction, symmetric output
scrollCapture() Manual scroll + sleep + screenshot loops Bottom detection, stuck scroll detection, auto-reset
waitForIdle() Blind sleep() calls MutationObserver with 500ms quiet window, 15s cap
diffSnapshots() Manual before/after tree comparison Accessibility tree diff with structured delta

extractPage() runs seven operations in parallel: page capture, performance metrics, security state, font detection, meta extraction, accessibility tree, and mobile-readiness checks. If a supplementary call fails, the field returns as null and a typed CoverageGap record documents what data is missing and why.

Two substrates, one methodology

Aspect Crawlio App (HTTP) Browser (CDP)
Evidence source WebKit capture + enrichment store DOM + JS execution + devtools hooks
Parallel extraction Sequential HTTP Promise.all (7 sources)
Scroll/viewport No viewport Yes (scrollCapture, waitForIdle)
Framework detection WebKit runtime capture JS runtime injection (17 namespaces)
Evidence tracking evidenceId chain via get_observation Typed Finding with confidence propagation

Crawlio App composite tools do not mirror browser methods where no viewport exists. No scrollCapture, no waitForIdle, no JS sandbox.

Next steps

  • Code Mode: the search-and-execute pattern that method mode extends
  • Evidence Mode: typed findings and confidence propagation
  • Tool Reference: all 49 full-mode tools with parameters
  • JIT Context: how the aggregator handles multi-pillar orchestration
© 2026 Crawlio. All rights reserved.