What is voice cloning?
Voice cloning is the process of training a text-to-speech (TTS) model on a short audio sample of a specific human voice so the model can then read arbitrary text in that voice. Modern systems need 15 to 30 seconds of clean audio. The output is indistinguishable from the source for most listeners.
Definition
Traditional TTS uses generic voices recorded by voice actors in a studio. The output sounds professional but generic. Voice cloning lets a business use its own brand voice: the founder, an in-house spokesperson, an actor under contract. AI calls then sound like the company rather than "default Polly voice 7".
The technical path: extract a speaker embedding from the sample audio, condition a TTS model (typically a diffusion or autoregressive model) on that embedding, generate spectrograms, convert to waveform with a vocoder. The pretrained model has heard thousands of voices and learns to interpolate quickly.
How message.com uses it
Voice cloning is the differentiator for the Smart Hold and AI Receptionist features. Customers upload a 15-second sample at POST /voice-profiles and get a cloned voice they can assign to any phone number. See voice profiles API.
Our TTS pipeline is neural text-to-speech with voice cloning, run on infrastructure we operate. We picked it for low latency, an open-weight backbone, and the ability to self-host when scale justifies it.
Consent and ethics
Voice cloning has obvious abuse vectors. Three policies that mitigate them:
- Consent attestation. Uploading a sample requires the customer to confirm they have rights to the voice.
- Disclosure. AI calls always identify themselves as AI within the first few seconds, regardless of how human the voice sounds. This is a hard product rule.
- Audit log. Every voice profile creation is recorded in audit log. Mass cloning attempts hit rate limits and human review.
Personality tags
A cloned voice is not enough by itself. The same voice can sound warm, energetic, calm, or professional depending on prosody. Voice profiles in message.com carry a personalityTag: warm, professional, energetic, calm, or custom. The tag biases the prosody model so the same brand voice can play different roles for different scenarios.
Alternatives to cloning
Customers who do not want to clone can pick from a curated set of preset voices. The trade-off is generic-sounding output in exchange for zero setup. Preset voices are free; cloned voices are also free in message.com (this is a deliberate competitive choice; competitors charge for cloning).