Cloned voices: whether it sounds real isn't really the point

Let me say something. Last week I tried voice cloning with Fish Audio, fiddled around for about forty minutes total: first upload a sample, wait for the model to finish, then have the same text read once by the cloned voice and once by a built-in voice, and compare by ear.

First impression is speed. The sample doesn't need to be long, tens of seconds to a minute or so is enough; this threshold is lower than I expected. But after hearing the result, the problem isn't whether it sounds alike. It does. The tone, the rhythm of pauses, the habit of dropping at sentence ends—all captured.

The problem is elsewhere.

I ran a mixed Chinese-English text with numbers and English abbreviations through it; English read fine, but the numbers started getting swallowed. I changed a few punctuation marks, swapped commas for periods, split long sentences shorter, and the swallowing eased somewhat but wasn't cured. The sample also mentioned this flaw, saying the noise might be deliberately added to prevent abuse. I didn't hear obvious noise on my end, though maybe my sample was too clean. Not sure, maybe I misunderstood.

Another thing made me pause more. The cloning threshold is so low that half of my forty minutes was spent thinking about permissions. Who can upload, where it's stored after upload, whether it can be deleted, whether there's a rate limit on the API side. The product page says ultra-low latency, REST and SDK; those are all fine, everything engineering-wise that should be given is given. But voice is different from other data; once a dry vocal goes out, the cost of getting it back is higher than you'd imagine.

If you're doing real-time conversation, podcast dubbing and the like, Fish Audio is worth trying; latency and emotion control at this tier measured sufficient on my end. But don't connect it to production right away; first clarify sample authorization and deletion, then talk integration. Whether the voice sounds alike is, in fact, the last thing you should care about.