找三位 data scientist 實測 Devin：20 題只過 3 題
案例摘要
Answer.AI 三位 data scientist（Hamel Husain、Isaac Flath、Johno Whitaker）給 Devin 出 20 道題：Devin 在 Slack 接指令，在 Docker 裡有終端、瀏覽器、編輯器、規劃器。早期題目含 Notion 搬到 Sheets、做行星位置追蹤器。結果：只過 3 題（Notion→Sheets、行星追蹤、Discord bot 研究），3 題 inconclusive、14 題失敗；看似簡單的題常耗數天，且無法預測哪題會過。
實際結果
3/20 通過；簡單題也可能耗數天。
台灣／大陸場景對照
🇹🇼 台灣
經典的期望管理案例：agent 工程能力被高估時，拿這篇給老闆看；台灣團隊導入前先做小規模 PoC。
🇨🇳 大陸
大陸團隊選型同理：別看 demo，看第三方實測；先定義驗收標準再談導入。
英文原文
Test Devin on pulling Notion data into Sheets
Three Answer.AI data scientists (Hamel Husain, Isaac Flath, Johno Whitaker) gave Devin 20 tasks. Devin takes commands through Slack and works in a Docker container with a terminal, browser, code editor and planner. Early tasks included pulling data from a Notion database into Google Sheets and building a planet tracker for checking historical Jupiter and Saturn positions.
Devin finished 3 of 20 tasks: the Notion-to-Sheets move, the planet tracker, and a Discord bot research task. Three were inconclusive and 14 failed. Simple-looking tasks often took days, and it was hard to predict which would work.
來源
https://agentusecases.ai/use-case/test-devin-on-pulling-notion-data-into-sheets
本站為策展彙整，原始內容版權歸原作者所有。