---
name: citedrank
version: 1.1
url: https://citedrank.co
description: Free web toolkit — sitemap discovery, page crawl, SEO audit, GEO audit, web-UI extraction, structured data scraping.
---

# CitedRank Skill

Use this skill when the user wants to scrape a webpage, audit a URL for SEO or
GEO (AI-search readiness), extract structured data, discover all URLs on a
domain, or pull a site's web UI / design system.

CitedRank runs every tool from a single backend with a shared cache: invoking
any single-page tool (Crawl / SEO / GEO / Web UI) on a URL makes the others
return in ~200ms on the next call.

## Choose the right tool

| User intent | Tool | Endpoint |
|---|---|---|
| "List every URL on a website" | Sitemap | `GET /api/sitemap` |
| "Get the text content of a page" | Crawl | `POST /api/fetch_one` |
| "Audit a URL for SEO problems" | SEO | `POST /api/seo` |
| "Check if a page is AI-search (GEO) ready" | GEO | `POST /api/geo` |
| "Extract a site's web UI — colors, fonts, CSS" | Web UI | `POST /api/clone` |
| "Scrape structured rows / fields from a page" | Data Scraper | `POST /api/extract` |

## MCP server (Cursor / Claude Desktop / Continue / Cline)

CitedRank exposes all 6 tools as an MCP server at
`https://citedrank.co/mcp` via the Streamable HTTP transport.

**Cursor / Continue** — add to your config:

```json
{
  "mcpServers": {
    "citedrank": {
      "url": "https://citedrank.co/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_CITEDRANK_TOKEN"
      }
    }
  }
}
```

**Claude Desktop** — Claude Desktop expects stdio MCP servers, so wrap
ours with the official MCP proxy:

```json
{
  "mcpServers": {
    "citedrank": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://citedrank.co/mcp",
               "--header", "Authorization:Bearer YOUR_CITEDRANK_TOKEN"]
    }
  }
}
```

After install, the agent gets 6 tools: `sitemap`, `crawl`, `seo`, `geo`, `clone`, `extract`. Ask
"use citedrank to extract the top 10 posts from news.ycombinator.com" and it
will figure out the rest.

## Auth

Every endpoint requires a Bearer token. Get one free at `https://citedrank.co/sign-in`
(Google OAuth). Put it in the `CITEDRANK_TOKEN` env var:

```bash
export CITEDRANK_TOKEN="paste-token-here"
```

## Endpoints

### 1. Sitemap — List every URL on a website

```bash
curl -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     "https://api.citedrank.co/api/sitemap?domain=stripe.com"
```

Returns: `{ domain, sitemaps[], total_urls, urls: [{ url, lastmod, priority }] }`

### 2. Crawl — Get the text content of a page

```bash
curl -X POST -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://stripe.com/pricing"}' \
     https://api.citedrank.co/api/fetch_one
```

Returns: `{ url, status, title, word_count, html_size, mhtml_saved, from_cache, ok }`

Then fetch the content with these GET endpoints:
- `GET /api/page/markdown?url=...` → clean Markdown body
- `GET /api/page/images?url=...` → `{ count, images: [absolute URLs] }`
- `GET /api/page/mhtml?url=...` → MHTML file with images embedded
- `GET /api/page/bundle?url=...` → ZIP (markdown + mhtml + image list)

### 3. SEO — Audit a URL for SEO problems

```bash
curl -X POST -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://stripe.com/pricing"}' \
     https://api.citedrank.co/api/seo
```

Returns: a 10-section report: `basics`, `headings`, `social`, `i18n`, `links`, `schema`, `media`, `content`, `tech`, `robots_meta`.

### 4. GEO — Check if a page is AI-search (GEO) ready

```bash
curl -X POST -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://stripe.com/pricing"}' \
     https://api.citedrank.co/api/geo
```

Returns: `{ url, domain, page_type, score, grade, signals[] (7 dimensions, 0-10 each), requirements, recommendations[], llms_txt }`

### 5. Web UI — Extract a site's web UI — colors, fonts, CSS

```bash
curl -X POST -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://linear.app"}' \
     https://api.citedrank.co/api/clone
```

Returns: `{ url, screenshot_available, mhtml_available, css_count, tokens, spec }` — `spec` holds clustered design tokens (colors, typography, spacing, radii, shadows).

Export the result:
- `GET /api/clone/export?url=X&format=tailwind` → `tailwind.config.js`
- `GET /api/clone/export?url=X&format=css` → CSS variables
- `GET /api/clone/export?url=X&format=tokens` → JSON tokens
- `GET /api/clone/zip?url=X` → full ZIP (HTML + CSS + tokens + screenshot)

### 6. Data Scraper — Scrape structured rows / fields from a page

```bash
curl -X POST -H "Authorization: Bearer $CITEDRANK_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://news.ycombinator.com/","item_selector":"tr.athing","fields":[{"name":"title","selector":"span.titleline a"},{"name":"url","selector":"span.titleline a","attr":"href"}],"mode":"http","limit":10}' \
     https://api.citedrank.co/api/extract
```

Returns: `{ url, status, item_count, duration_ms, mode_used, items: [...] }`

**Field spec:** `name` → output key · `selector` → CSS relative to each item · `attr` → optional HTML attribute (defaults to text) · `multiple` → `true` for list-valued fields.
**Modes:** `http` (default, ~200ms-1s) · `browser` (Playwright, JS SPAs) · `stealth` (anti-bot, Cloudflare).
**Templates:** `GET /api/extract/templates` returns 5 ready selector sets (Hacker News, GitHub Trending, Product Hunt, Reddit, generic blog).

## Choosing between Crawl and Extract

- **Crawl** = "give me the whole page text" → user wants to read it or feed RAG.
- **Extract** = "give me these specific fields as rows" → user wants data analysis.

If the user asks for "all the products on this page", that is **Extract**
(structured). If they ask for "the article content", that is **Crawl** (Markdown).

## Choosing between SEO and GEO

- **SEO** = classic on-page audit (titles, meta, headings, links, schema) — for
  ranking in traditional search results.
- **GEO** = AI-search readiness — scores how easily ChatGPT, Perplexity, and
  Google AI Overviews can extract and cite the page. Use GEO when the user asks
  about AI search, LLM visibility, or getting cited by AI engines.

## Errors

- `401` — bad / missing token. Re-fetch from `https://citedrank.co/sign-in`.
- `404` — endpoint or page not found.
- `429` — rate limited (free tier).
- `500` with `detail: "fetch failed: ..."` — target URL unreachable / blocked.

## Worked example: "audit a competitor's pricing page for AI search"

```bash
# 1. discover the competitor's URLs, keep the pricing ones
curl -H "Authorization: Bearer $CITEDRANK_TOKEN" \
  "https://api.citedrank.co/api/sitemap?domain=competitor.com" | jq -r '.urls[].url' \
  | grep -i pricing > urls.txt

# 2. for each, run SEO + GEO (they share the render cache)
while read url; do
  curl -s -H "Authorization: Bearer $CITEDRANK_TOKEN" -X POST \
       -H "Content-Type: application/json" -d "{\"url\":\"$url\"}" \
       https://api.citedrank.co/api/seo > "seo-$(basename "$url").json"
  curl -s -H "Authorization: Bearer $CITEDRANK_TOKEN" -X POST \
       -H "Content-Type: application/json" -d "{\"url\":\"$url\"}" \
       https://api.citedrank.co/api/geo > "geo-$(basename "$url").json"
done < urls.txt
```