Compare research agents on filling gaps in a public dataset in two languages
Human-rights researcher Roya Pakzad gave Meta Muse, Claude Cowork and ChatGPT the same job through their regular web apps: fill missing fields in the World Bank's public procurement database from official sources, for the US in English and for Iran in Farsi, then register and upload the changes. She recorded every run and checked the data herself. All three produced spreadsheets; for the US they filled roughly 40 to 50% of empty fields, mostly from official sites, but for Iran coverage was far thinner and sources drifted to Telegram channels and blogs. Permission prompts and portal handling differed a lot between agents. A useful template for testing agents on non-English research.
Tools used
- Meta Muse
- Claude Cowork
- ChatGPT
- Claude Code
Sources (2)
- Originalhttps://royapakzad.substack.com/p/multilingual-ai-agents (opens in a new tab) ↗
Roya Pakzad · royapakzad.substack.com
- Found via