🤖 AI Agent 研究Research
现有的多代理基准主要在竞争环境中进行测试,在20个步骤下进行短期互动,或者只是汇总个人绩效,未能隔离和突出基于LLM的代理的真正协作能力。我们引入了AgentWorld ,这是一个包含100个人工注释任务(包含100个增强变体)的基准,用于评估长期多Agent协作。
Existing multi-agent benchmarks primarily test in competitive settings, short-horizon interactions under 20 steps, or simply aggregate individual performance, failing to isolate and highlight genuine collaboration capabilities of LLM-based agents. We introduce AgentWorld, a benchmark of 100 human-annotated tasks (with 100 augmented variants) for evaluating long-horizon, multi-agent collaboration.
时间序列客服代表通过调用外部工具回答分析问题,他们携带的工具由人员在客服代表运行之前决定。但是,我们在此设置中发现了两个故障。Human-Agent Tool Misalignment :由21个专家精心策划的工具组成的库可帮助完成某些任务,而对其他任务造成伤害,从而降低我们测试的每个骨干的异常准确性。
Time series agents answer analytical questions by calling external tools, and which tools they carry is decided by people before the agent runs. However, we identify two failures in this setup. Human-Agent Tool Misalignment: a library of 21 expert-curated tools helps on some tasks and hurts on others, dropping anomaly accuracy under every backbone we test.
工具调用代理产生异构输出,将结构化工具调用与面向用户的自然语言摘要交织在一起。这种输出异构性在标准的策略上强化学习( RL )中呈现出一种结构性故障模式:像GRPO这样的算法不加区分地向所有代币广播均匀轨迹级标量优势。
Tool-calling agents produce heterogeneous outputs, interleaving structured tool invocations with user-facing natural language summaries. This output heterogeneity presents a structural failure mode in standard on-policy Reinforcement Learning (RL): algorithms like GRPO indiscriminately broadcast a homogeneous trajectory-level scalar advantage to all tokens.
我们推出了Kaggle Game Arena ,这是一个开放且不断扩展的平台,通过竞技游戏评估大型语言模型(LLM)。与静态基准不同,游戏竞技场使模型能够在结构化环境中进行正面交锋,其中游戏强度随着模型的发展而自然增加,从而防止性能饱和。
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation.
⭐ GitHub 热门项目GitHub Trending
【GitHub】支持MCP的多语言CAD/3D设计服务器。使用Python和Java后端控制Claude的FreeCAD和其他设计工具。使用AI-dri创建、编辑、分析和协作3D模型(⭐ 0 )
【GitHub】Multi-language CAD/3D design server with MCP support. Control FreeCAD and other design tools from Claude using Python and Java backends. Create, edit, analyze, and collaborate on 3D models with AI-dri (⭐ 0)
【GitHub】模型上下文协议(MCP)服务器🛡️的安全审计CLI —扫描AI代理配置,了解工具中毒、地毯拉动、硬编码秘密、命令注入和供应链风险。纯Python , SARIF + CI就绪。
【GitHub】🛡️ Security audit CLI for Model Context Protocol (MCP) servers — scan AI agent configs for tool poisoning, rug pulls, hardcoded secrets, command injection & supply-chain risks. Pure Python, SARIF + CI ready.
【GitHub】一个围绕Claude Code的开源包装器,专注于上下文管理、连续性和个人助理风格的工程工作流程。(⭐ 1 )
【GitHub】An open source wrapper around Claude Code focused on context management, continuity, and personal-assistant-style engineering workflows. (⭐ 1)
【GitHub】免费的自托管X ( Twitter )刮刀,没有API密钥,没有积分。通过剧作家使用您自己的浏览器会话。将推文、搜索结果和时间轴导出为LLM就绪的JSON/CSV/Markdown。为OpenClaw和Hermes运送人工智能特工技能。
【GitHub】Free, self-hosted X (Twitter) scraper no API key, no credits. Uses your own browser session via Playwright. Export tweets, search results & timelines as LLM-ready JSON/CSV/Markdown. Ships an AI agent Skill for OpenClaw & Hermes.
【GitHub】为Claude、Gemini CLI、Codex和其他AI代理提供全面的徽标设计技能:原则、流程、SVG工艺、测试工具和1,400多个徽标参考库。
【GitHub】A comprehensive logo-design skill for Claude, Gemini CLI, Codex and other AI agents: principles, process, SVG craft, testing tools and a 1,400+ logo reference library.
🚀 模型与行业动态Models & Industry
人工智能实验室的一位高级管理人员告诉《华尔街日报》,该模型在服从命令方面表现不佳。
A top executive at the AI lab told the Wall Street Journal that the model in question had displayed a poor aptitude for following orders.
此次收购将使世界实验室创始人李飞飞加入AMD ,担任执行副总裁兼首席科学家。
The acquisition will see World Labs founder Fei-Fei Li join AMD as executive vice president and chief scientist.
Shopify正在将WebMCP支持扩展到结账,允许基于浏览器的人工智能代理在买方授权下更新订单详细信息并完成购买。
Shopify is expanding WebMCP support to checkout, allowing browser-based AI agents to update order details and complete purchases with a buyer’s authorization.
🔥 社区热议Community
【Lobsters】热度: 106↑ | 9 评论 | 标签: practices, vibecoding
【Lobsters】热度: 106↑ | 9 评论 | 标签: practices, vibecoding
【Lobsters】热度: 44↑ | 6 评论 | 标签: a11y, design, vibecoding
【Lobsters】热度: 44↑ | 6 评论 | 标签: a11y, design, vibecoding
【HN】热度: 278 分 | 115 评论
【HN】热度: 278 分 | 115 评论
【HN】热度: 6 分 | 1 评论
【HN】热度: 6 分 | 1 评论