carlos

Excited about our work on SWE-bench Multimodal! The 2 🔑 insights about coding agents: - The importance of visual reasoning in coding tasks - Enabling LMs to interact with powerful environments is better than building LMs into pre-made solution patterns Great work by the team!

RT @jyangballin: We're launching SWE-bench Multimodal to eval agents' ability to solve visual GitHub issues. - 617 *brand new* tasks from 1…