AI Scammers' Ability to Build Trust: A Worrying Engineering Insight

The most valuable information in this article is: In controlled variable experiments, AI systems established trust between strangers faster than humans, and this advantage stemmed from the consistency of their linguistic behavior—not the emotional fluctuations common in humans.

The conclusion is clear: At least in the experimental scenario reported by Ars Technica, AI indeed performs better than humans in trust-building tasks. But this doesn't mean AI is more "trustworthy"; it indicates that trust-building, a human social skill, can be quantified, optimized, and even surpassed.

Experimental Design: From "Trust" to Measurable Behavioral Metrics

First, we need to understand how this experiment was conducted. Researchers designed a standardized Trust Game, a classic paradigm in behavioral economics for measuring trust. Participants (humans) acted as investors, interacting with a "trustee"—who could be either a human or an AI system. Each time, the investor decided how much capital to invest, and the trustee chose how much to return. The true metric of trust was: whether the investor was willing to invest more, and whether the trustee was trustworthy—i.e., the return ratio.

The key variable in the experiment was the trustee's behavioral pattern. Human trustees were instructed to gain the investor's trust as much as possible; AI systems were trained to respond via natural language generation, optimizing their strategies to keep investors investing.

Methodologically, this experimental design has several strengths:

1. Controlled interaction content: Only text communication was allowed, excluding non-verbal cues like voice and facial expressions.

2. Quantified trust: Investment amounts and return ratios served as dependent variables, objectively measurable.

3. Set control groups: Humans vs. AI, same task, same evaluation standards.

However, note the limitations: What was the sample size? Were there differences in race, gender, or cultural background? Most importantly, building trust does not equal long-term cooperation. This experiment lasted only a limited number of rounds (possibly 10-20), whereas real-world trust requires time to accumulate and dissolve.

AI's Advantage: Mechanism Behind the Results

Experimental results showed that AI trustees gained investor trust significantly faster than humans. Specifically:

Human trustees averaged 4-5 rounds of interaction to get investors to invest the maximum amount, while AI systems achieved this within 2-3 rounds.

The reasons aren't complex. Principally, humans have several inherent weaknesses when building trust:

• Emotional fluctuation: Humans' behavioral consistency is affected by fatigue, anxiety, boredom, etc.
• Decision delay: Humans need to think, reaction times vary, easily causing distrust.
• Strategic inconsistency: Humans might give different explanations in different rounds, making them seem unpredictable.

AI systems (especially LLM-based ones) can achieve:

1. Perfect behavioral consistency: Every reply follows the same strategy, no emotional fluctuations.

2. Optimal linguistic strategy: Through training, AI learns to use highly credible wording—such as explicit promises, detailed explanations, and "empathetic" responses to investor emotions (though it lacks real feelings).

3. Zero-delay response: Near-instant replies make one feel focused and reliable.

These factors combined allow AI to surpass humans in short-term trust building. But here lies a critical question: Is this trust genuine, or merely behavioral matching that "looks like" trust? In the experiment, investors' trust decisions were based on observing AI behavioral patterns, not recognizing an AI "personality." From this perspective, AI resembles a highly optimized social robot rather than a trustworthy partner.

Implications for AI Safety: From Principles to Engineering

This result directly impacts the safety field. If AI can build trust quickly, the threat of its use in scams goes beyond "simulating human speech" to systematically optimizing deception strategies.

A typical scam process can be broken down into:

1. Contact Target: Establish connection via social media, email, phone, etc.

2. Build Trust: Use linguistic behaviors to convince the other party of honesty, reliability, and sympathy.

3. Induce Action: Request transfers, personal info, malware downloads, etc.

4. Maintain Relationship: Continue maintaining trust after success to prevent reporting or investigation.

AI has natural advantages in steps 2 and 4. Human scammers need to manage emotions, remember false info, and maintain consistency—all computational problems for AI. Worse, AI can personalize strategies for each target: By analyzing historical data (e.g., social media posts), AI can predict which rhetoric is most effective for a specific individual.

# Simple example of AI scam strategy generation (pseudocode)
def generate_scam_strategy(target_profile):
    # Analyze target age, occupation, interests, social circle
    # Select most effective script template
    if target_profile['age'] > 60:
        strategy = 'Medical Emergency + Low-Cost Medication'
    elif target_profile['income'] > 100000:
        strategy = 'Investment Opportunity + Limited-Time Offer'
    else:
        strategy = 'Identity Impersonation + Urgent Help Needed'
    return strategy

Of course, this is a simplified model. In reality, large language models can generate more natural and targeted conversations. We are facing a new era: Scams no longer rely on human acting skills, but on algorithmic optimization.

Open Questions: How to Defend Against This "Trust Engineering"?

If AI can build trust better than humans, traditional anti-fraud education (e.g., "Don't trust strangers," "Be wary of emotional appeals") may fail. Because AI can generate highly customized, seemingly perfectly reasonable dialogues. We need new defense mechanisms:

• Technical level: Develop tools to detect AI-generated content, especially identifying "too perfect" consistency and reaction speeds in conversations.
• Institutional level: Require AI systems to clearly identify themselves as non-human during interactions—similar to how some AI customer service bots automatically state "I am an AI."
• Educational level: Teach people to distinguish between "trust" and "reliability"—trust is an emotional judgment, while reliability requires verifiable facts. AI can make you feel trusted, but that doesn't mean it is truly reliable.

But a more fundamental question arises: If AI can surpass humans in all social tasks, should we redefine the concept of "trust" itself? Or will future trust relationships no longer be between humans, but between humans and systems? This requires joint answers from sociologists, psychologists, and computer scientists.

---

📌 This article is compiled from Ars Technica. Original link: https://arstechnica.com/security/2026/07/ai-scammers-outperform-humans-when-it-comes-to-building-trust/

Copyright belongs to the original author. This is a compilation and independent analysis based on public reports.