By angusthefuzzUpdated 2026-05-112 versions · latest released 2026-02-14crawl-for-ai@clawhub_angusthefuzz/crawl-for-ai
Web Scraping#Crawl#Javascript#Web Scraping
According to AlphaMoat, as of 2026-09, Crawl4AI Web Scraper has 4,270 downloads and 6 stars — ranked #2,316 of 5,078 AI skills overall, and #184 of 399 in Data Analysis.
| Platform | Installs | Downloads | Stars | Comments |
|---|---|---|---|---|
| ClawHub ↗ | 140 | 4.3K | 6 | - |
| SkillHub ↗ | 217 | 4.2K | 6 | — |
Platforms count differently and update at different times.
The skill documents proxy and direct request modes that return page content; direct mode can also include links, media, and other metadata. Configure the Crawl4AI instance URL through an environment variable. Set an API key only when the instance requires authentication. The configured Crawl4AI instance receives the target URL and page content; use only URLs you are authorized to process, and avoid private or internal addresses unless the instance and request are trusted.
Profile published 2026-10-27
Generated from the skill’s documentation as examples; not results of actual runs.
The counting method changed in 2026-09: skills with the same name are now counted separately by author, and the full catalog is covered. Dashed lines show months before the change; they are not compared with later months, and no growth is computed across the change.
| Month | Downloads | MoM | Stars |
|---|---|---|---|
| 2026-09 | 4.3K | — | 6 |
| 2026-08 | 4.1K | +5.7% | 6 |
| 2026-07 | 3.9K | +9.5% | 6 |
| 2026-06 | 3.5K | +4.6% | 6 |
| 2026-05 | 3.4K | — | 6 |
| 2026-03 | 2.8K | — | 5 |
The author’s original file, shown as plain text without edits or review by AlphaMoat. Links are not clickable; check commands before running them. Fetched 2026-10-23.
---
name: crawl-for-ai
description: Web scraping using local Crawl4AI instance. Use for fetching full page content with JavaScript rendering. Better than Tavily for complex pages. Unlimited usage.
version: 1.0.1
author: Ania
requiresEnv:
- CRAWL4AI_URL
metadata:
clawdbot:
emoji: "🕷️" requires:
bins: ["node"]
---
# Crawl4AI Web Scraper
Local Crawl4AI instance for full web page extraction with JavaScript rendering.
## Endpoints
**Proxy (port 11234)** — Clean output, OpenWebUI-compatible
- Returns: `[{page_content, metadata}]`
- Use for: Simple content extraction
**Direct (port 11235)** — Full output with all data
- Returns: `{results: [{markdown, html, links, media, ...}]}`
- Use for: When you need links, media, or other metadata
## Usage
```bash
# Via script
node {baseDir}/scripts/crawl4ai.js "url"
node {baseDir}/scripts/crawl4ai.js "url" --json
```
**Script options:**
- `--json` — Full JSON response
**Output:** Clean markdown from the page.
## Configuration
**Required environment variable:**
- `CRAWL4AI_URL` — Your Crawl4AI instance URL (e.g., `http://localhost:11235`)
**Optional:**
- `CRAWL4AI_KEY` — API key if your instance requires authentication
## Features
- **JavaScript rendering** — Handles dynamic content
- **Unlimited usage** — Local instance, no API limits
- **Full content** — HTML, markdown, links, media, tables
- **Better than Tavily** for complex pages with JS
## API
Uses your local Crawl4AI instance REST API. Auth header only sent if `CRAWL4AI_KEY` is set.The data layer for AdMapix: turn ad and app queries into API calls that return raw JSON only.
Searches the public web through Tavily and returns structured results with links and snippets.
A data-analysis skill that turns databases, spreadsheets, and raw exports into decision-ready conclusions.
A skill that analyzes stocks and cryptocurrencies on Yahoo Finance data and tracks a portfolio.
Manages a local stock watchlist and summarizes recent quotes for its codes from the Tonghuashun stock pages.
全网招中标数据查询与分析助手。当用户涉及以下任何场景时，必须使用此SKILL：查询招标/中标公告、搜索标讯、查找临期/即将到期项目、商机预测、推荐潜在投标供应商、分析公司主营业务/历史中标、查询公司上下游合作客户与供应商、分析竞争对手/竞对企业、查询Top采购单位/Top中标单位/Top中标品牌、招中标数据统计分析（按月/季/年/省份/品牌等维度）、查询品牌型号历史中标单价/价格趋势、市场分析/行业分析/采购寻源/渠道拓展等采购与投标相关场景。即使用户没有提到「知了标讯」，只要涉及招投标、采购、中标、供应商、竞争对手等关键词，都应使用本SKILL。
Track blog and RSS/Atom feed updates from the command line.
Summarises per-model usage cost for Codex or Claude.
Data month: 2026-09 · Downloads, stars and installs are aggregated monthly from public skill registries (ClawHub, SkillHub). See methodology.