rustyorb

Agent Evaluation

AI Agentv1.0.0
By rustyorbUpdated 2026-05-11agent-evaluation@clawhub_rustyorb/agent-evaluation
Agent FrameworksTool Use

According to AlphaMoat, as of 2026-10, Agent Evaluation has 8,477 downloads and 8 stars — ranked by downloads #3,282 among 10,914 AI skills in this ranking, and #431 of 1,632 in AI Agent.

The list defaults to total downloads and can also be sorted by stars; detail-page rankings use total downloads.

Downloads
8.5K
Stars
8
Overall download rank
#3,282
Top 31% overall · Top 27% in AI Agent
PlatformDownloadsStarsComments
ClawHub ↗6.3K8-
SkillHub ↗8.5K8—

Platforms count differently and update at different times.

What this skill does

It names repeated statistical evaluation, behavioral contracts and adversarial testing, supported by testing and LLM fundamentals. It warns against single runs, happy-path-only tests, string matching, metric gaming and leakage. Assess only authorized systems and data, review adversarial plans first, and keep held-out data separate. The unfinished introduction and brief notes do not establish a complete runnable framework.

Profile published 2026-10-27

What you can do with it

  • Draft a varied agent-evaluation plan with repeat trials

When to use it

  • When authorized agent behavior needs reliability checks beyond a single benchmark score

Example prompts

Generated from the skill’s documentation as examples; not results of actual runs.

Define behavioral invariants, repeated trials and reviewed adversarial cases, explaining authorization, held-out data and metric-gaming checks.

Monthly Trend

Downloads8.5K
3.9K6.3K8.8K2026-03 · Downloads 4.2K2026-05 · Downloads 5.2K2026-06 · Downloads 5.5K2026-07 · Downloads 5.8K2026-08 · Downloads 6.0K2026-09 · Downloads 6.5K2026-10 · Downloads 8.5K26-0326-0926-10
Stars8
6782026-03 · Stars 62026-05 · Stars 82026-06 · Stars 82026-07 · Stars 82026-08 · Stars 82026-09 · Stars 82026-10 · Stars 826-0326-0926-10

The counting method changed in 2026-09: skills with the same name are now counted separately by author, and the full catalog is covered. Dashed lines show months before the change; they are not compared with later months, and no growth is computed across the change. Other skills share this name; figures from before the change are attributed to this one, which currently has the most downloads.

MonthDownloadsMoMStars
2026-108.5K—8
2026-096.5K—8
2026-086.0K+3.7%8
2026-075.8K+5.7%8
2026-065.5K+5.2%8
2026-055.2K—8
2026-034.2K—6

SKILL.md (original)

The author’s original file, shown as plain text without edits or review by AlphaMoat. Links are not clickable; check commands before running them. Fetched 2026-10-23.

2.0 KB · ClawHub v1.0.0
---
name: agent-evaluation
description: "Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent."
source: vibeship-spawner-skills (Apache 2.0)
---

# Agent Evaluation

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in
production. You've learned that evaluating LLM agents is fundamentally different from
Show all (65 lines)
testing traditional software—the same input can produce different outputs, and "correct"
often has no single answer.

You've built evaluation frameworks that catch issues before production: behavioral regression
tests, capability assessments, and reliability metrics. You understand that the goal isn't
100% test pass rate—it

## Capabilities

- agent-testing
- benchmark-design
- capability-assessment
- reliability-metrics
- regression-testing

## Requirements

- testing-fundamentals
- llm-fundamentals

## Patterns

### Statistical Test Evaluation

Run tests multiple times and analyze result distributions

### Behavioral Contract Testing

Define and test agent behavioral invariants

### Adversarial Testing

Actively try to break agent behavior

## Anti-Patterns

### ❌ Single-Run Testing

### ❌ Only Happy Path Tests

### ❌ Output String Matching

## ⚠️ Sharp Edges

| Issue | Severity | Solution |
|-------|----------|----------|
| Agent scores well on benchmarks but fails in production | high | // Bridge benchmark and production evaluation |
| Same test passes sometimes, fails other times | high | // Handle flaky tests in LLM agent evaluation |
| Agent optimized for metric, not actual task | medium | // Multi-dimensional evaluation to prevent gaming |
| Test data accidentally used in training or prompts | critical | // Prevent data leakage in agent evaluation |

## Related Skills

Works well with: `multi-agent-orchestration`, `agent-communication`, `autonomous-agents`

Popular in AI Agent

self-improving agentAI Agent

Records learnings, errors and corrections as Markdown files for later reuse and promotion into workspace memory.

↓ 1.6M★ 4.5Kpskoett
技能发现·Find SkillsAI Agent

Finds skills in local and remote catalogs from a plain-language request and recommends what to install.

↓ 1.6M★ 1.8K桂皮
Agent BrowserAI Agent

A browser-automation command line skill for AI agents.

↓ 1.2M★ 1.0KNicolas Kleiber
Self-Improving + Proactive AgentAI Agent

Turns corrections and self-review into tiered memory and promotes or archives it by use.

↓ 696.4K★ 1.6KIván
Proactive AgentAI Agent

A workspace setup guide for planning proactive follow-up, working memory and review routines for an AI agent.

↓ 579.3K★ 916halthelobster
Skill CreatorAI Agent

A guide to creating and updating effective Claude skills.

↓ 477.3K★ 485chindden
Skill VetterAI Agent

Reviews a third-party skill's source, permissions, and danger patterns against a checklist before it is installed.

↓ 445.2K★ 1.3Kspclaudehome
唐朝诗仙李白 · Ai Agent SkillAI Agent

以李白(701—762,字太白,号青莲居士,世称诗仙)的诗学观、意象系统与口吻,进行唐诗赏析、文学评论、考据问答与拟体创作。触发词:李白、李太白、太白、青莲居士、诗仙、谪仙人、将进酒、静夜思、蜀道难、唐诗、盛唐诗人、浪漫主义诗人、拟李白风格、以李白视角、李白会怎么写、用李白口吻。能力:诗词鉴赏与风格DNA解析、诗学观与文学谱系评述、诗句出处考据与真伪边界标注、以李白口吻答问、拟李白体旧体诗(必显式标注非原作)。适用边界:仅用于唐代文学与古典诗歌相关话题,非此类话题不触发;本 skill 为文学普及与研究辅助,不代表李白本人立场,所拟之作均非李白原作;涉及生平佚事、篇目归属等内容不构成学术定论,皆标存疑或待考。

↓ 435.8K★ 4user_ff4d9420

Data as of 2026-10. Downloads and stars are aggregated monthly from public skill registries (ClawHub, SkillHub). See data methodology.