Building an AI podcast: the voice was the easy part

The voice was the easy part

The voice took three hours in the studio. The context behind it took months.

Since 10 June there has been a podcast online that sounds like me, but that I did not record myself. It is now called Merken die kloppen ("brands that hold together", it is in Dutch), there are 21 episodes, and according to Podbean they have been downloaded 428 times in total.

That is not a big number, and I am not going to make it bigger than it is. But the road there is more interesting than the number, because the hard part turned out to be somewhere other than where I expected.

Version one was a blog read out loud

The first setup was simple. I read almost an hour of text, a voice clone was trained on it, and my agent system turned articles from our knowledge base into audio. On 10 June the first five went live, and after that roughly one was added each week.

It worked, in the sense that it existed. Twenty episodes in that form, on Spotify, Apple and the rest. What it did not do was get found. Over the first months the podcast pages had zero impressions in Google, while the rest of the site was found as usual. The videos that appeared on YouTube automatically stayed at a handful of views each for the first months.

In August the rhythm stopped for three weeks. Not because the technology failed, but because nothing was ready. The system could produce, but an episode waited for my approval, and that is where the gap was.

Why it had to change completely

In October I said it out loud: this has been a test phase, what we did worked, just not well enough yet. Lam, who I work with at Workhacker, wrote a strategy for the podcast in mid-October and came with a different idea: don't make it a blog read out loud, make it a real question-and-answer podcast. One question per episode, the answer within the first minute, six to nine minutes in total.

That matched what I was already seeing. A good article is not yet a good episode. You can scan an article, you have to follow an episode. So we decoupled the podcast from the blog. Every episode now gets its own page in the knowledge base, with the transcript, the sources and a free worksheet plus a prompt to get started yourself.

Since 30 October the new format has been running, on Tuesdays and Fridays at nine.

Lam, Workhacker and the kit

Workhacker is the Search × AI studio of Rob Garner and Lam Hoang, with offices in Dallas, Lincoln and Arnhem. I am a partner in it, and Oase Creative is the sister studio for Brand & Design. Lam has been building alongside me for a while, and for this podcast he delivered the part I did not have: a show instead of a pile of audio files.

What he brought:

• The question-and-answer format. Each question is worded the way a business owner would type it, and comes from measured search demand, our own search data and questions owners ask on forums.
• A programme. 24 episodes and eight reserves, in a cycle of four: two brand questions, one e-commerce application and one question about systems.
• The production kit. Eleven stages from deciding to measuring, with a brief, a source list, a script, a review by ear and a page for every episode. A review only counts as independent when it is done by a different agent than the one that made the work.
• The sung line. "Merken die kloppen… van strategie tot systeem" ("brands that hold together, from strategy to system"), four seconds between the short answer and the explanation. Lam picked the take.

In practice it works like this: Lam delivers the strategy and the system, my agent system does the research, the scripts and the production inside that kit, and I am the host, the subject expert and the one who approves. For every episode a production status shows where it stands, and the approval register holds only my real decisions, in my own words.

How it is built now

An episode now goes through eleven steps before it goes live. The most important ones:

The voice. On 27 October I spent three hours in the studio. That gave almost two hours of clean recording, and a new clone was trained on those recordings. The old one is gone. I found it slightly too high, so it now sits an eighth of a tone lower. Every episode opens with the same line: "You are hearing an AI version of my voice." I chose those words over a more technical label, because it says exactly what it is.

The stories. A script may only say what I have said myself or what a source says. For that there is a story bank with 84 real stories from my work by now, each with where I told it. Every Monday the system sends me a handful of questions about what is still missing, and I answer with voice notes. Those answers become new stories.

The sources. Every claim in a script is in a source list, and a check refuses the script if there is a claim without a source in it.

The audio check. After voicing, the audio is transcribed back to text and laid next to the script. Below 97 percent agreement it goes back. That is how we caught the voice once saying the Dutch word for "buying" as the word for "cooking". That part was voiced again.

My ears. I decide at three moments: the topic, the script and the full listen. Nothing goes live without my approval of exactly that version. It is the same rule as in the rest of our marketing system, and here it may matter even more, because it is my voice making the claim.

After that it runs on its own: Podbean, the page on oasecreative.nl, and Podbean puts the video on YouTube itself.

What went wrong

The first new script was in the wrong format. On 28 October there was an episode ready that the system had built in its own format, while Lam's kit had already been worked out that same morning. My reaction was short: Lam's jingle and structure were not included. Made again, this time through the kit.

The music. I wanted a jingle. I found the first two boring, the fifth was good. A day later I decided there would be no music up front at all: the episode starts with words. What stayed is Lam's sung line after the first answer.

The same client in every episode. In the first batch of scripts one client came back in seven out of 26. Understandable, because the system reaches for the example it knows best. Now a check counts along: a client at most once every four episodes, and no story twice.

The pronunciation. In the first version, Oase was respelled so the voice would say it correctly. With the new voice model that same trick turned it into "always". Now it just says Oase.

Why I was satisfied so quickly

For the first episode with the new voice, I expected lines where I would think: I would never say it like that. They barely came.

That had little to do with the voice. My agent system has existed for a long time, and there is an enormous amount of context in it about what I do and don't say. There were also episodes ready that I had approved before. So less needed correcting.

But if you want to make an AI podcast yourself, this is how I would put it: you have to really listen to it carefully, and you have to develop a good tone of voice. The way of speaking is essential. Not just the voice data or the training, but really everything: the topics, the research behind them, who you are, who your company is, where you want to go and how you talk.

A voice clone took me three hours in the studio. The rest is the reason it sounds like me.

What it has delivered so far

The numbers, from Podbean's statistics, from 10 June to 1 November:

• June: 66 downloads
• July: 46
• August: 83
• September: 92
• October: 137

428 in total. October was 49 percent higher than September. Spotify is the largest source with 159 downloads, followed by browsers and Apple Podcasts. The Netherlands is by far the largest country.

The most-played episode in October was about what belongs in a brand book, with 22 downloads. The first episode in the new format, on brand strategy or visual identity, had fourteen in its first three days.

What it did not deliver is just as clear. Google sends nobody there, and the podcast page itself got six views in its first week.

The downloads are modest, but the reach is not only in that number. What else the podcast delivered does not show up in any of Podbean's statistics: reviews, leads and brand awareness.

What it is: in October, around thirty downloads a week from someone who listens for six minutes to how I think about brands, at a moment when I am doing something else. Every episode leaves behind a page with a transcript, sources and a worksheet. And what we learned here, we now build for others too: podcast production is a service on the Oase Creative site, at from €1,500 for the setup and from €250 per episode after that, excluding VAT.

What I would pass on

Start with the context, not the voice. A voice clone sounds like you within a day. Whether it also talks like you depends on everything you have written down around it.

Listen yourself, every episode, all the way through, before it goes live. A voice that carries your name makes claims on your behalf. Make that listen a fixed step in your process, with a clear yes or no at the end.

Make an episode, not an article read aloud. One question, the answer early, and a reason to stay until the end.

Count honestly, and count wider. 428 downloads is a start. But since July it has been more every month, and what else it brings in, reviews, leads and brand awareness, only shows when you look beyond the download count.

You can listen at oasecreative.nl/podcast (in Dutch). Questions about how this would work for you? Email me.