Funnel tests AI agents: GLM repeats same SQL query 254 times: what are the key facts?

Funnel ran a randomized A/B test of GLM-5.2 and Claude Opus 4.8 to compare their data-agent performance.

Funnel tests AI agents: GLM repeats same SQL query 254 times: what are the key facts?

GLM averaged 7.8 seconds per model call versus 18.1 seconds for Opus, making GLM faster.

Funnel tests AI agents: GLM repeats same SQL query 254 times: what are the key facts?

GLM averaged 19 tool calls, 66% more than Opus's 11, yet end-to-end task time was 14% longer.

Funnel tests AI agents: GLM repeats same SQL query 254 times: what are the key facts?

In the two test runs, GLM executed the same successful SQL query 113 times and 254 times, respectively.

Funnel tests AI agents: GLM repeats same SQL query 254 times: what are the key facts?

Funnel notes that the agent framework matters as much as model intelligence, and offers recommendations on tool contracts and behavioral boundaries.

Physix Frontier · News Briefing Card (enbrief · Oct 29, 2026)

Funnel tests AI agents: GLM repeats same SQL query 254 times

KEY FACTS

Funnel ran a randomized A/B test of GLM-5.2 and Claude Opus 4.8 to compare their data-agent performance.
GLM averaged 7.8 seconds per model call versus 18.1 seconds for Opus, making GLM faster.
GLM averaged 19 tool calls, 66% more than Opus's 11, yet end-to-end task time was 14% longer.
In the two test runs, GLM executed the same successful SQL query 113 times and 254 times, respectively.
Funnel notes that the agent framework matters as much as model intelligence, and offers recommendations on tool contracts and behavioral boundaries.

KEY DATA

7.8sGLM average call latency

18.1sOpus average call latency

19GLM average tool calls

11Opus average tool calls

PHYSIX OBSERVATION

A fast model does not mean a fast task; GLM's repeated calls expose the agent framework's shortcomings. Swapping an API endpoint for an open-source model is easy, but vague tool contracts and the lack of protection against repeated calls can zero out the efficiency advantage. Engineering teams should prioritize tightening framework constraints rather than fixating on model benchmark scores.

Source: enbrief original report ↗