據 AlphaMoat 統計,截至 2026-10,Web Scraper as a Service 累計下載 5,909 次、獲 7 個星標,按累計下載量在本次參與排名的 10,914 個 AI 技能中排名第 4,891 位,在「資料分析」分類(915 個)中排名第 450 位。
列表預設按累計下載量排序,也可按星標排序;詳情頁排名按累計下載量計算。
| 平台 | 累計下載 | 星標 | 評論 |
|---|---|---|---|
| ClawHub ↗ | 5040 | 3 | - |
| SkillHub ↗ | 5909 | 7 | — |
各平台的計數口徑不同,統計時間不一致。
根據抓取需求建立面向交付的爬蟲專案,涵蓋目標頁面分析、指令碼生成、資料清理和交付資料整理。文件還列出對速率限制、錯誤恢復、資料校驗和結果報告的要求。只應抓取獲准訪問的網站;開始前檢查 robots.txt 與服務條款,審閱生成的程式碼,並控制對目標站點的請求頻率。
資料釋出於 2026-10-27
以下是根據技能文件生成的示例,不代表實際執行結果。
2026-09 起統計方法調整:同名技能按作者分別統計,並覆蓋全部技能。圖中虛線為調整前的月份,與之後的資料不直接比較,也不計算跨越調整的增長。
| 月份 | 累計下載 | 環比 | 星標 |
|---|---|---|---|
| 2026-10 | 5909 | — | 7 |
| 2026-09 | 5152 | — | 7 |
| 2026-08 | 4766 | +3.0% | 7 |
| 2026-07 | 4625 | +6.7% | 7 |
| 2026-06 | 4336 | +6.2% | 3 |
| 2026-05 | 4081 | — | 5 |
| 2026-03 | 2744 | — | 3 |
以下是作者釋出的原始檔案,按純文字顯示,本站未做修改或稽核;其中的連結不可點選,命令請核實後再執行。抓取於 2026-10-23。
--- name: web-scraper-as-a-service description: Build client-ready web scrapers with clean data output. Use when creating scrapers for clients, extracting data from websites, or delivering scraping projects. argument-hint: "[target-url-or-brief]" allowed-tools: Read, Write, Edit, Grep, Glob, Bash, WebFetch, WebSearch --- # Web Scraper as a Service Turn scraping briefs into deliverable scraping projects. Generates the scraper, runs it, cleans the data, and packages everything for the client.
## How to Use
```
/web-scraper-as-a-service "Scrape all products from example-store.com — need name, price, description, images. CSV output."
/web-scraper-as-a-service https://example.com --fields "title,price,rating,url" --format csv
/web-scraper-as-a-service brief.txt
```
## Scraper Generation Pipeline
### Step 1: Analyze the Target
Before writing any code:
1. **Fetch the target URL** to understand the page structure
2. **Identify**:
- Is the site server-rendered (static HTML) or client-rendered (JavaScript/SPA)?
- What anti-scraping measures are visible? (Cloudflare, CAPTCHAs, rate limits)
- Pagination pattern (URL params, infinite scroll, load more button)
- Data structure (product cards, table rows, list items)
- Total estimated volume (number of pages/items)
3. **Choose the right tool**:
- Static HTML → Python + `requests` + `BeautifulSoup`
- JavaScript-rendered → Python + `playwright`
- API available → Direct API calls (check network tab patterns)
### Step 2: Build the Scraper
Generate a complete Python script in `scraper/` directory:
```
scraper/
scrape.py # Main scraper script
requirements.txt # Dependencies
config.json # Target URLs, fields, settings
README.md # Setup and usage instructions for client
```
**`scrape.py` must include**:
```python
# Required features in every scraper:
# 1. Configuration
import json
config = json.load(open('config.json'))
# 2. Rate limiting (ALWAYS — be respectful)
import time
DELAY_BETWEEN_REQUESTS = 2 # seconds, adjustable in config
# 3. Retry logic
MAX_RETRIES = 3
RETRY_DELAY = 5
# 4. User-Agent rotation
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36...",
# ... at least 5 user agents
]
# 5. Progress tracking
print(f"Scraping page {current}/{total} — {items_collected} items collected")
# 6. Error handling
# - Log errors but don't crash on individual page failures
# - Save progress incrementally (don't lose data on crash)
# - Write errors to error_log.txt
# 7. Output
# - Save data incrementally (append to file, don't hold in memory)
# - Support CSV and JSON output
# - Clean and normalize data before saving
# 8. Resume capability
# - Track last successfully scraped page/URL
# - Can resume from where it left off if interrupted
```
### Step 3: Data Cleaning
After scraping, clean the data:
1. **Remove duplicates** (by unique identifier or composite key)
2. **Normalize text** (strip extra whitespace, fix encoding issues, consistent capitalization)
3. **Validate data** (no empty required fields, prices are numbers, URLs are valid)
4. **Standardize formats** (dates to ISO 8601, currency to numbers, consistent units)
5. **Generate data quality report**:
```
Data Quality Report
───────────────────
Total records: 2,487
Duplicates removed: 13
Empty fields filled: 0
Fields with issues: price (3 records had non-numeric values — cleaned)
Completeness: 99.5%
```
### Step 4: Client Deliverable Package
Generate a complete deliverable:
```
delivery/
data.csv # Clean data in requested format
data.json # JSON alternative
data-quality-report.md # Quality metrics
scraper-documentation.md # How the scraper works
README.md # Quick start guide
```
**`scraper-documentation.md`** includes:
- What was scraped and from where
- How many records collected
- Data fields and their descriptions
- How to re-run the scraper
- Known limitations
- Date of scraping
### Step 5: Output to User
Present:
1. **Summary**: X records scraped from Y pages, Z% data quality
2. **Sample data**: First 5 rows of the output
3. **File locations**: Where the deliverables are saved
4. **Client handoff notes**: What to tell the client about the data
## Scraper Templates
Based on the target type, use the appropriate template:
### E-commerce Product Scraper
Fields: name, price, original_price, discount, description, images, category, sku, rating, review_count, availability, url
### Real Estate Listings
Fields: address, price, bedrooms, bathrooms, sqft, lot_size, listing_type, agent, description, images, url
### Job Listings
Fields: title, company, location, salary, job_type, description, requirements, posted_date, url
### Directory/Business Listings
Fields: business_name, address, phone, website, category, rating, review_count, hours, description
### News/Blog Articles
Fields: title, author, date, content, tags, url, image
## Ethical Scraping Rules
1. **Always respect robots.txt** — check before scraping
2. **Rate limit** — minimum 2 second delay between requests
3. **Identify yourself** — use realistic but honest User-Agent
4. **Don't scrape personal data** (emails, phone numbers) unless explicitly authorized by the client AND the data is publicly displayed
5. **Cache responses** — don't re-scrape pages unnecessarily
6. **Check ToS** — note if the site's terms prohibit scraping and inform the client
中國金融機構資料爬取與分析綜合 Skill v12.0.0。2334家/43大類機構名單、60+財經媒體源輿情、 A股報告/公告/研報、60+種查詢多源金融資料(自動降級+快取)、爬取排程與網頁操作增強、 即時資料中樞、24 技術指標引擎、多源證據融合、110 MCP 工具 + 10 Resource + 14 Prompt。 🆕 v12.0.0 四條線(資訊獲取 / 打包下載 / 資料分析 / 登入): 編排:gather_info() 一次跑完"路由→抓取→去佔位→質量分→能力邊界→回執→打包", 8 類意圖各帶問法、輸入輸出、典型耗時與 not_for(說清哪件事做不了)。 誠實:反爬佔位條目一律剔除並記賬,空結果報 E_SOURCE_EMPTY,不把假資料判成 B 級成功。 交付:包內自帶 manifest.json(逐檔案 sha256),verify 查得出改動/缺失/多塞, 批次下載單條失敗不中斷整批,失敗清單可直接只重下那幾條。 登入:Cookie 由使用者匯出或掃碼接管,檔案化 + 健康度四檔,對外輸出全脫敏(值永不回顯)。 何時觸發:爬取機構、輿情爬取、東方財富、公告下載、即時行情、技術指標、統一查詢、 交付包、批次重下、我已經登入了、登入態還有效嗎(完整觸發詞見下方 auto_trigger)。
大宗商品價格資料查詢技能。用於查詢大宗商品期現貨價格、宏觀經濟指標、產業鏈供需資料、進出口庫存資料、金融市場資料等。當用戶詢問:(1) 期貨/現貨價格查詢,如"螺紋鋼價格"、"銅期貨收盤價"; (2) 歷史行情或趨勢分析;(3) 區域或企業價格對比;(4) 宏觀經濟指標,如GDP、CPI;(5) 產業鏈上下游資料;(6) 多市場金融資料(期權、股票、債券、外匯);(7) 結算確認單生成時觸發此技能。
32種圖表,3 個主題,2步生成: 1.上傳資料 — 將 CSV / Excel / JSON 等格式的資料檔案拖入對話方塊; 2.檢視結果 — 互動式圖表(HTML),可儲存圖表為圖片.
AdMapix 的取數層:把廣告與應用相關的查詢轉成介面呼叫,只回原始 JSON。
論文配圖一鍵生成:一條命令從 JSON/CSV/Excel 生成頂刊級學術圖表,科研繪圖 與期刊配圖全場景覆蓋。40 種圖表(森林圖/KM 生存曲線(自動風險表)/競爭風險累計發生率/ROC/校準 曲線/決策曲線 DCA/列線圖/RCS 非線性劑量反應/亞組分析森林圖/RMST 限制平均生存時間/Galbraith 徑向圖/火山圖/雨雲圖/山脊圖/啞鈴圖/游泳圖(逐例時長+里程碑)/ 腫瘤緩解瀑布圖/UpSet 交集/PRISMA 流程圖/漏斗/BA 一致性/PCA/熱圖/箱線/小提琴/ 組合圖/Cox 多因素迴歸森林圖等),9 套配色(含色盲安全 Okabe-Ito 與期刊色板),9 種期刊投稿預設, YAML 流水線一條命令出全套論文圖,一次渲染多格式匯出(TIFF/PNG/PDF), PDF 重疊門禁,600dpi。純本地執行,資料不出機,中文零配置。適用:寫論文/ 投稿要配圖、把資料變成可投稿圖表、批次重出全套圖。不適用:文字潤色/翻譯/ 改寫(請用寫作類工具);商業報表/儀表盤/海報裝飾;無資料支撐的純示意圖。
Intelligent chart generation and data analysis skill. Reads user-supplied data files (CSV/Excel/JSON), analyzes data characteristics with LLM assistance, auto-recommends and generates interactive ECharts visualizations. Use when the user asks to analyze data, generate charts, create visualizations, or work with tabular data files.
全球智慧投研工具(stock-researcher)v13.0:多源資料 + 機率預測 + 可匯出報告。 核心能力: - 多源資料路由:按市場/類別選源,逐級降級;每個結果標註來源與時效等級;源不可用時明確標 unavailable - 全資產量化:股票/ETF/場外基金/指數/期貨(期限結構、基差、持倉、折溢價)多因子分析 - 機率預測:1d/5d/1M/3M/6M 經驗分佈 + 分位數 + 方向機率 + 區間,kNN 鄰域收縮融合全域性分佈; 樣本不足自動棄權,可按歷史命中樣本(同桶)做機率校準 - 證據鏈簡報:每條結論繫結來源、時間與資料質量,衝突訊號保留而不抹平 - 報告匯出:Word/PPT/PDF/Markdown/JSON/純文字,純標準庫渲染 + 可選第三方引擎 - MCP Server 24 工具,Claude Code 可直接呼叫;知識庫 12 介面卡雙向同步 三條鐵律: - 資料不裝:拿不到就報不可用,不拿 0 價、不拿空表冒充 - 預測不裝:機率來自歷史相似條件的經驗分佈,樣本不足就說不足,不硬給方向 - 評分≠盈利保證:任何結論都帶免責宣告
透過 Tavily 搜尋公開網頁,並回傳含連結與摘要的結構化結果。
資料月份:2026-10 · 下載/星標數聚合自公開技能註冊庫(ClawHub、SkillHub),按月更新。口徑說明見資料方法。