U1 Pro: An engineering breakthrough in long-horizon task closure, or just another multimodal agent show?

I recently ran the GAIA validation set and noticed an interesting phenomenon: multimodal LLMs generally exceed 90% accuracy on single-step instructions, but once the task chain exceeds 5 steps, success rates plummet off a cliff to below 25%. GPT-4V barely holds up to 8 steps, Claude 3.5 Opus starts frequently losing context around 10 steps, and Gemini 2.0 Flash often messes up parameters during tool calls. Behind this data lies an industry consensus: closing the loop on long-horizon tasks is currently the hardest bottleneck for multimodal agents.