AI Skills

AI Skills are structured Markdown files that encode domain judgment for web crawling. They follow the Agent Skills open standard and ship as a plugin from Crawlio-app/crawlio-plugin.

Skills tell your AI agent when to use which settings, how to interpret observations, and what constitutes a finding. The MCP server handles the mechanics; skills handle the judgment.

Install

Install using the Agent Skills CLI:

npx skills add Crawlio-app/crawlio-plugin

This copies the 7 skill files and 1 agent into your project's .claude/skills/ directory (or equivalent for your AI tool).

Install as a Claude Code plugin:

claude plugin add Crawlio-app/crawlio-plugin

Or install directly from the marketplace:

claude install crawlio

Clone the plugin repo and copy the skills into your project:

git clone https://github.com/Crawlio-app/crawlio-plugin.git
cp -r crawlio-plugin/skills/ .claude/skills/crawlio/
cp -r crawlio-plugin/agents/ .claude/agents/

ℹ️

All installation methods require the Crawlio macOS app running and the CrawlioMCP server available. See MCP Setup for details.

Skills

7 skills covering the full crawl-analyze-export workflow plus MCP orchestration and web research:

crawlio-mcp

Canonical MCP reference: 37 full-mode tools, 6 code-mode tools, 4 resources, 4 prompts.

crawl-site

Crawl a website with intelligent configuration, progress monitoring, and failure retry.

extract-and-export

Complete pipeline: crawl, extract structured content, and export in 7 formats.

observe

Query the observation log, the append-only timeline of everything Crawlio saw during a crawl.

finding

Create and query evidence-backed findings with observation IDs as evidence.

audit-site

Six-phase site audit: configure, crawl, capture enrichment, analyze, create findings, report.

web-research

Acquire → Normalize → Analyze protocol for multi-source research and competitive comparison.

Agent

The plugin includes one agent, site-auditor, a systematic analysis agent that follows a structured protocol:

Reconnaissance. Detect site type, configure crawl settings
Crawl. Download with monitoring and failure retry
Analysis. 4 passes: structure, errors, enrichment, synthesis
Report. Evidence-backed findings with prioritized recommendations

The agent uses Sonnet and has access to Bash, Read, Glob, Grep, WebFetch, and WebSearch alongside all Crawlio MCP tools.

How it works

AI Agent (Claude, Gemini, Cursor, etc.)

reads skill Markdown

▼

Skill (domain judgment)

invokes MCP tools

▼

CrawlioMCP (stdio)

HTTP

▼

Crawlio App (macOS, 127.0.0.1)

Skills encode judgment: when to use which settings, how to interpret observations, what constitutes a finding.

MCP server handles mechanics: HTTP calls, file reads, protocol bridging.

This separation is what makes the plugin forkable: swap the judgment layer for your domain, keep the same mechanics.

Forking

The plugin is designed to be forked for domain-specific use cases. Replace the skill Markdown with your own judgment layer:

SEO Auditor. Meta tags, heading hierarchy, structured data, internal linking
Security Scanner. HTTPS enforcement, security headers, exposed endpoints
Competitive Analysis. Multi-site framework comparison, third-party services
Content Migration. URL mapping, redirect chains, content volume

See FORKING.md in the plugin repo for a full guide.

Next steps

Skill Reference

Detailed parameters, workflows, and examples for all 7 skills.

MCP Setup

Configure the CrawlioMCP server that skills communicate through.

Connect AI

Connect Claude, ChatGPT, Cursor, or Windsurf to Crawlio.

PreviousTool Reference NextSkill Reference