File Locations

Overview

Crawlio writes several files to disk that MCP tools, the CLI, and external scripts can read. All files use standard macOS locations.

State and control files

control.sock

Unix Domain Socket for the HTTP API. This is the primary transport used by the CLI and MCP server. Permissions are set to 0600 (owner only).

curl --unix-socket ~/Library/Logs/Crawlio/control.sock http://localhost/status

control.port

Contains a single integer: the HTTP port number. Written on app launch, deleted on quit.

PORT=$(cat ~/Library/Logs/Crawlio/control.port)
curl http://localhost:$PORT/status

Finding the port programmatically: Read ~/Library/Logs/Crawlio/control.port. If the file does not exist, the app is not running. If the file exists but the port is not responding, the app quit without cleanup. Delete the stale file and relaunch.

state.json

Full snapshot of the current crawl. Updated every 500ms while crawling. Cleared when the app quits.

{
  "engineState": "crawling",
  "url": "https://example.com",
  "progress": {
    "discovered": 150,
    "downloaded": 85,
    "failed": 2,
    "queued": 63,
    "localized": 80
  },
  "items": [
    {
      "url": "https://example.com/index.html",
      "status": "completed",
      "size": 15234,
      "contentType": "text/html",
      "localPath": "/Users/you/Downloads/Crawlio/example.com/index.html",
      "startTime": "2026-02-14T10:30:00Z",
      "endTime": "2026-02-14T10:30:01Z"
    }
  ]
}

crawl.jsonl

One JSON object per line. Written continuously during a crawl.

{"timestamp":"2026-02-14T10:30:00.123Z","category":"engine","level":"info","message":"Crawl started for https://example.com"}
{"timestamp":"2026-02-14T10:30:01.456Z","category":"download","level":"info","message":"Downloaded https://example.com/index.html (15.2 KB)"}

Categories: engine, download, parser, localizer, network

Levels: debug, info, notice, warning, error, fault

Log rotation

Log files rotate at 10 MB. Up to 3 rotated files are kept. Files older than 7 days are deleted automatically.

Download locations

Default

Downloaded files are saved to:

~/Downloads/Crawlio/{domain}/

For example, downloading https://example.com creates:

~/Downloads/Crawlio/example.com/
  index.html
  about/
    index.html
  css/
    style.css
  images/
    logo.png

Custom destination

Set a custom destination per-crawl through:

• The project settings panel in the app
• The --dest flag in the CLI
• The destinationPath field in POST /start (HTTP API)
• The destination parameter in start_crawl (MCP)

Application support

Checkpoints

Crawl resume data with 3-checkpoint rotation and atomic writes. Crawls survive app crashes and system restarts. Resume a crawl by relaunching the app.

Enrichment

Runtime capture data (framework detection, network requests, console logs, DOM snapshots) persisted as JSON. Scoped per-project.

Browser extension bridge

{browser} is one of: Google/Chrome, Chromium, BraveSoftware/Brave-Browser, Microsoft Edge.

MCP and CLI files

MCP server binary

launchd service (portal mode)

MCP client config files

The crawlio-mcp init command writes to detected client config files:

Skills

Homebrew binaries

Vault

Vault sessions are stored in the macOS Keychain, not on disk.

Next steps

• See the HTTP API for using the control socket and port
• Check Troubleshooting for stale port files and connection issues
• See MCP Tools for the tool reference