据 AlphaMoat 统计,截至 2026-10,Web Scraper as a Service 累计下载 5,909 次、获 7 个星标,按累计下载量在本次参与排名的 10,914 个 AI 技能中排名第 4,891 位,在「数据分析」分类(915 个)中排名第 450 位。
列表默认按累计下载量排序,也可按星标排序;详情页排名按累计下载量计算。
| 平台 | 累计下载 | 星标 | 评论 |
|---|---|---|---|
| ClawHub ↗ | 5,040 | 3 | - |
| SkillHub ↗ | 5,909 | 7 | — |
各平台的计数口径不同,统计时间不一致。
根据抓取需求创建面向交付的爬虫项目,涵盖目标页面分析、脚本生成、数据清理和交付资料整理。文档还列出对速率限制、错误恢复、数据校验和结果报告的要求。只应抓取获准访问的网站;开始前检查 robots.txt 与服务条款,审阅生成的代码,并控制对目标站点的请求频率。
资料发布于 2026-10-27
以下是根据技能文档生成的示例,不代表实际运行结果。
2026-09 起统计方法调整:同名技能按作者分别统计,并覆盖全部技能。图中虚线为调整前的月份,与之后的数据不直接比较,也不计算跨越调整的增长。
| 月份 | 累计下载 | 环比 | 星标 |
|---|---|---|---|
| 2026-10 | 5,909 | — | 7 |
| 2026-09 | 5,152 | — | 7 |
| 2026-08 | 4,766 | +3.0% | 7 |
| 2026-07 | 4,625 | +6.7% | 7 |
| 2026-06 | 4,336 | +6.2% | 3 |
| 2026-05 | 4,081 | — | 5 |
| 2026-03 | 2,744 | — | 3 |
以下是作者发布的原始文件,按纯文本显示,本站未做修改或审核;其中的链接不可点击,命令请核实后再运行。抓取于 2026-10-23。
--- name: web-scraper-as-a-service description: Build client-ready web scrapers with clean data output. Use when creating scrapers for clients, extracting data from websites, or delivering scraping projects. argument-hint: "[target-url-or-brief]" allowed-tools: Read, Write, Edit, Grep, Glob, Bash, WebFetch, WebSearch --- # Web Scraper as a Service Turn scraping briefs into deliverable scraping projects. Generates the scraper, runs it, cleans the data, and packages everything for the client.
## How to Use
```
/web-scraper-as-a-service "Scrape all products from example-store.com — need name, price, description, images. CSV output."
/web-scraper-as-a-service https://example.com --fields "title,price,rating,url" --format csv
/web-scraper-as-a-service brief.txt
```
## Scraper Generation Pipeline
### Step 1: Analyze the Target
Before writing any code:
1. **Fetch the target URL** to understand the page structure
2. **Identify**:
- Is the site server-rendered (static HTML) or client-rendered (JavaScript/SPA)?
- What anti-scraping measures are visible? (Cloudflare, CAPTCHAs, rate limits)
- Pagination pattern (URL params, infinite scroll, load more button)
- Data structure (product cards, table rows, list items)
- Total estimated volume (number of pages/items)
3. **Choose the right tool**:
- Static HTML → Python + `requests` + `BeautifulSoup`
- JavaScript-rendered → Python + `playwright`
- API available → Direct API calls (check network tab patterns)
### Step 2: Build the Scraper
Generate a complete Python script in `scraper/` directory:
```
scraper/
scrape.py # Main scraper script
requirements.txt # Dependencies
config.json # Target URLs, fields, settings
README.md # Setup and usage instructions for client
```
**`scrape.py` must include**:
```python
# Required features in every scraper:
# 1. Configuration
import json
config = json.load(open('config.json'))
# 2. Rate limiting (ALWAYS — be respectful)
import time
DELAY_BETWEEN_REQUESTS = 2 # seconds, adjustable in config
# 3. Retry logic
MAX_RETRIES = 3
RETRY_DELAY = 5
# 4. User-Agent rotation
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36...",
# ... at least 5 user agents
]
# 5. Progress tracking
print(f"Scraping page {current}/{total} — {items_collected} items collected")
# 6. Error handling
# - Log errors but don't crash on individual page failures
# - Save progress incrementally (don't lose data on crash)
# - Write errors to error_log.txt
# 7. Output
# - Save data incrementally (append to file, don't hold in memory)
# - Support CSV and JSON output
# - Clean and normalize data before saving
# 8. Resume capability
# - Track last successfully scraped page/URL
# - Can resume from where it left off if interrupted
```
### Step 3: Data Cleaning
After scraping, clean the data:
1. **Remove duplicates** (by unique identifier or composite key)
2. **Normalize text** (strip extra whitespace, fix encoding issues, consistent capitalization)
3. **Validate data** (no empty required fields, prices are numbers, URLs are valid)
4. **Standardize formats** (dates to ISO 8601, currency to numbers, consistent units)
5. **Generate data quality report**:
```
Data Quality Report
───────────────────
Total records: 2,487
Duplicates removed: 13
Empty fields filled: 0
Fields with issues: price (3 records had non-numeric values — cleaned)
Completeness: 99.5%
```
### Step 4: Client Deliverable Package
Generate a complete deliverable:
```
delivery/
data.csv # Clean data in requested format
data.json # JSON alternative
data-quality-report.md # Quality metrics
scraper-documentation.md # How the scraper works
README.md # Quick start guide
```
**`scraper-documentation.md`** includes:
- What was scraped and from where
- How many records collected
- Data fields and their descriptions
- How to re-run the scraper
- Known limitations
- Date of scraping
### Step 5: Output to User
Present:
1. **Summary**: X records scraped from Y pages, Z% data quality
2. **Sample data**: First 5 rows of the output
3. **File locations**: Where the deliverables are saved
4. **Client handoff notes**: What to tell the client about the data
## Scraper Templates
Based on the target type, use the appropriate template:
### E-commerce Product Scraper
Fields: name, price, original_price, discount, description, images, category, sku, rating, review_count, availability, url
### Real Estate Listings
Fields: address, price, bedrooms, bathrooms, sqft, lot_size, listing_type, agent, description, images, url
### Job Listings
Fields: title, company, location, salary, job_type, description, requirements, posted_date, url
### Directory/Business Listings
Fields: business_name, address, phone, website, category, rating, review_count, hours, description
### News/Blog Articles
Fields: title, author, date, content, tags, url, image
## Ethical Scraping Rules
1. **Always respect robots.txt** — check before scraping
2. **Rate limit** — minimum 2 second delay between requests
3. **Identify yourself** — use realistic but honest User-Agent
4. **Don't scrape personal data** (emails, phone numbers) unless explicitly authorized by the client AND the data is publicly displayed
5. **Cache responses** — don't re-scrape pages unnecessarily
6. **Check ToS** — note if the site's terms prohibit scraping and inform the client
中国金融机构数据爬取与分析综合 Skill v12.0.0。2334家/43大类机构名单、60+财经媒体源舆情、 A股报告/公告/研报、60+种查询多源金融数据(自动降级+缓存)、爬取调度与网页操作增强、 实时数据中枢、24 技术指标引擎、多源证据融合、110 MCP 工具 + 10 Resource + 14 Prompt。 🆕 v12.0.0 四条线(信息获取 / 打包下载 / 数据分析 / 登录): 编排:gather_info() 一次跑完"路由→抓取→去占位→质量分→能力边界→回执→打包", 8 类意图各带问法、输入输出、典型耗时与 not_for(说清哪件事做不了)。 诚实:反爬占位条目一律剔除并记账,空结果报 E_SOURCE_EMPTY,不把假数据判成 B 级成功。 交付:包内自带 manifest.json(逐文件 sha256),verify 查得出改动/缺失/多塞, 批量下载单条失败不中断整批,失败清单可直接只重下那几条。 登录:Cookie 由用户导出或扫码接管,档案化 + 健康度四档,对外输出全脱敏(值永不回显)。 何时触发:爬取机构、舆情爬取、东方财富、公告下载、实时行情、技术指标、统一查询、 交付包、批量重下、我已经登录了、登录态还有效吗(完整触发词见下方 auto_trigger)。
大宗商品价格数据查询技能。用于查询大宗商品期现货价格、宏观经济指标、产业链供需数据、进出口库存数据、金融市场数据等。当用户询问:(1) 期货/现货价格查询,如"螺纹钢价格"、"铜期货收盘价"; (2) 历史行情或趋势分析;(3) 区域或企业价格对比;(4) 宏观经济指标,如GDP、CPI;(5) 产业链上下游数据;(6) 多市场金融数据(期权、股票、债券、外汇);(7) 结算确认单生成时触发此技能。
32种图表,3 个主题,2步生成: 1.上传数据 — 将 CSV / Excel / JSON 等格式的数据文件拖入对话框; 2.查看结果 — 交互式图表(HTML),可保存图表为图片.
AdMapix 的取数层:把广告与应用相关的查询转成接口调用,只回原始 JSON。
论文配图一键生成:一条命令从 JSON/CSV/Excel 生成顶刊级学术图表,科研绘图 与期刊配图全场景覆盖。40 种图表(森林图/KM 生存曲线(自动风险表)/竞争风险累计发生率/ROC/校准 曲线/决策曲线 DCA/列线图/RCS 非线性剂量反应/亚组分析森林图/RMST 限制平均生存时间/Galbraith 径向图/火山图/雨云图/山脊图/哑铃图/游泳图(逐例时长+里程碑)/ 肿瘤缓解瀑布图/UpSet 交集/PRISMA 流程图/漏斗/BA 一致性/PCA/热图/箱线/小提琴/ 组合图/Cox 多因素回归森林图等),9 套配色(含色盲安全 Okabe-Ito 与期刊色板),9 种期刊投稿预设, YAML 流水线一条命令出全套论文图,一次渲染多格式导出(TIFF/PNG/PDF), PDF 重叠门禁,600dpi。纯本地运行,数据不出机,中文零配置。适用:写论文/ 投稿要配图、把数据变成可投稿图表、批量重出全套图。不适用:文本润色/翻译/ 改写(请用写作类工具);商业报表/仪表盘/海报装饰;无数据支撑的纯示意图。
Intelligent chart generation and data analysis skill. Reads user-supplied data files (CSV/Excel/JSON), analyzes data characteristics with LLM assistance, auto-recommends and generates interactive ECharts visualizations. Use when the user asks to analyze data, generate charts, create visualizations, or work with tabular data files.
全球智能投研工具(stock-researcher)v13.0:多源数据 + 概率预测 + 可导出报告。 核心能力: - 多源数据路由:按市场/类别选源,逐级降级;每个结果标注来源与时效等级;源不可用时明确标 unavailable - 全资产量化:股票/ETF/场外基金/指数/期货(期限结构、基差、持仓、折溢价)多因子分析 - 概率预测:1d/5d/1M/3M/6M 经验分布 + 分位数 + 方向概率 + 区间,kNN 邻域收缩融合全局分布; 样本不足自动弃权,可按历史命中样本(同桶)做概率校准 - 证据链简报:每条结论绑定来源、时间与数据质量,冲突信号保留而不抹平 - 报告导出:Word/PPT/PDF/Markdown/JSON/纯文本,纯标准库渲染 + 可选第三方引擎 - MCP Server 24 工具,Claude Code 可直接调用;知识库 12 适配器双向同步 三条铁律: - 数据不装:拿不到就报不可用,不拿 0 价、不拿空表冒充 - 预测不装:概率来自历史相似条件的经验分布,样本不足就说不足,不硬给方向 - 评分≠盈利保证:任何结论都带免责声明
走 Tavily 检索公开网页,返回带链接与摘要的结构化结果。
数据月份:2026-10 · 下载/星标数聚合自公开技能注册库(ClawHub、SkillHub),按月更新。口径说明见数据方法。