A Chinese–English Homophone Joke Generator

This article was originally written in Chinese and translated into English by AI.

duck bubi, babe wuchi... Here is how to manufacture an endless supply of comedian Li Dan's favorite jokes. The cover was inspired by the show I Can I BB.

Estimated reading time: approximately 9 minutes.

Comedy: ★★

Knowledge: ★★★

What you will learn: string similarity.

A homophone joke creates humor by replacing part of a Chinese expression with a similar-sounding English word. Examples include book siyi, Tony daishui, and tansheng pass.

Compared with other jokes, homophone puns are relatively easy to create. Without much thought, you can produce something delightfully entertaining. Naturally, telling one in front of Li Dan delivers twice the delight.

If homophone jokes are so much fun, can a program generate them automatically? In this article, we will build a Homophone Joke Generator, also known as Li Dan's Source of Happiness.

First, define the goal. Given a four-character Chinese idiom such as 有备而来 (you bei er lai, “come prepared”), the generator should replace two characters with a similar-sounding English word, producing something like 有bear来 (you bear lai).

With the goal established, the generator's overall workflow is:

Step 1: Convert Chinese Characters to Pinyin

First, convert the Chinese idiom 有备而来 into pinyin: you-bei-er-lai.

Many ready-made libraries already solve this problem, so we can call one directly—for example, xpinyin in Python.

Step 2: Prepare an English Dictionary

Next, we need to prepare an English dictionary.

To simplify the computation, consider only the most common elementary-school vocabulary and keep words containing three to five letters.

This ensures that most readers will recognize words such as “green,” “blue,” and “time,” while reducing the amount of string matching required later. Extremely long words such as supercali... will never be useful here.

Step 3: Calculate String Similarity and Find Matches

We now have the idiom's pinyin, YouBeiErLai, and an English dictionary containing thousands of simple words. The next step is string matching.

We are not looking for an English word exactly identical to the pinyin. Instead, we calculate the similarity between the idiom's pinyin and every English word, then perform a replacement.

This leads to the most important question:

How do we define the similarity between two strings?

The following sections introduce several common algorithms. Readers who are not interested in the algorithms can skip to Step 4.

1. Hamming Distance

First, consider the simplest case: two strings of equal length.

Hamming distance is the number of characters that must be replaced to transform one string into another. Put simply, compare whether the characters at each position are equal.

Strings A and B differ in two positions—the third and fifth—so their Hamming distance is 2.

Similarity is generally expressed between 0 and 1, with values closer to 1 indicating greater similarity. We can therefore define it as the number of matching characters divided by the total string length. In the diagram above, that is 3/5, or 0.6.

Another way to understand Hamming distance is to XOR the two strings and count the number of 1s in the result.

2. Edit Distance

Next, consider two strings with different lengths.

Hamming distance counts character substitutions. When lengths differ, we can instead count insertions, deletions, and substitutions.

Edit distance is the minimum number of character substitutions, insertions, and deletions required to transform one string into another.

In this example, delete the second and third characters of string A, then replace its fifth character “a” with “d.” These three operations produce string B, so the edit distance is 3.

We could also delete all five characters from A and all three from B, making both equal to the empty string in eight operations. But eight is not the minimum number of operations required to transform A into B, so the edit distance is not 8.

The following formula calculates similarity between A and B. In this example, it is (8-3)/8 = 0.625.

Edit distance sounds simple, but implementing it requires some thought. It is a classic dynamic-programming exercise. Interested readers can try it here:

leetcode-cn.com/problems/edit-distance

3. Cosine Similarity

Cosine similarity is one of the most common ways to compare two vectors.

We can convert each string into a vector and then use the cosine to calculate similarity between the vectors.

How do we turn a string into a vector? With a bag-of-words model.

Suppose strings A and B are as follows:

Together, the two strings contain four basic characters: “a,” “b,” “c,” and “d.”

Count the occurrences of these four characters in A and B to construct a pair of vectors.

A={2,2,1,0}, B={1,0,1,1}. String A contains two “a” characters, two “b” characters, one “c,” and no “d.” String B is encoded in the same way. 

Notice that a bag-of-words model ignores character order. Both “aabbc” and “cabab” can be encoded as {2,2,1,0}.

Once we have the vectors corresponding to strings A and B, use the cosine formula to calculate their similarity.

Cosine similarity can also compare two documents.

4. Other Measures

Besides the three methods above, other similarity measures include:

Jaccard similarity

Dice similarity

Jaccard and Dice are well suited to comparing documents. In that case, sets A and B represent the documents, and the elements in each set are words.

All of these similarity measures are useful not only for string matching, but also in fields such as object detection and face recognition.

After learning five similarity measures, return to our earlier question: how do we define similarity between two strings?

The answer depends on the situation. For equal-length strings, Hamming distance is the simplest and most direct. To compare documents, Jaccard, Dice, or cosine similarity may be better. If...

As with loss functions in deep learning, we are not limited to famous formulas. We can define a custom similarity formula for the particular problem. Understanding the principle behind an algorithm is far more important than applying it mechanically.

Step 4: Final Results and Further Improvements

Now combine the preceding steps.

Start with 有备而来 and convert it to the pinyin “You Bei Er Lai.”

Prepare a dictionary containing simple English words.

Select two characters from the four-character idiom, such as 备而. This selection can be made manually, or the program can traverse every possible combination and choose the best result.

The pinyin for 备而 is “beier.” Use one of the algorithms from Step 3—edit distance, for example—to calculate the similarity between “beier” and every word in the English dictionary, from “able,” “above,” and “act” through “zoo.”

We find that “beer” is the closest word, with a similarity of 0.88888. Under the edit-distance definition in Step 3, the distance is 1.

Replace “beier” with “beer,” turning 有备而来 into 有beer来.

Likewise, 欢天喜地 becomes “hunt喜地.” 

This method works well in some cases, but it still has minor flaws.

Consider the first two characters of 不可思议, pronounced “buke.” The similar-sounding English word should be “book,” but by edit distance, “buke” and “book” are four operations apart: delete “u” and “e,” then insert two “o” characters. “buke” is only one operation from “bake” and “bike,” so the output becomes “bake思议.”

The result is usable, but there is still room for improvement.

Is there a better approach?

Yes: use phonetic transcription!

Pinyin records the pronunciation of Chinese characters, while phonetic transcription records the pronunciation of English words. They provide a natural bridge between the two languages.

The new algorithm follows this workflow:

The overall idea remains unchanged. Begin with 有备而来, convert it to “YouBeiErLai,” and select “BeiEr” for matching.

The new algorithm changes how English words are represented. First, look up each word's phonetic transcription in an English dictionary: “beer” corresponds approximately to “ber,” while “sky” corresponds to “skaɪ.”

Next, manually map every phonetic symbol to an approximate pinyin sound. For example, “aɪ” maps to “ai,” “ð” resembles the Chinese pinyin “zhe,” and the stress mark “ˈ” can simply be ignored.

Finally, write a function that converts phonetic transcription into approximate pinyin.

After conversion, the phonetic form “ber” for “beer” remains “ber,” while “skaɪ” for “sky” becomes “skai.”

Now match “BeiEr” from 有备而来 against these approximate pinyin forms. “ber,” derived from “beer,” is the closest match, so “beer” replaces 备而.

The new method performs a little better than comparing pinyin directly with English spelling, but not dramatically better.

With the old method, “buke” from 不可思议 finds only “bake” and “bike.” The new method can find “book,” but it also returns less relevant words such as “box” and “block.”

The code from this and earlier articles, including the Homophone Joke Generator,

has been uploaded to GitHub:

github.com/DrMofu/MLab_wechat

If you enjoyed this article, follow the Technical Miscellany Shop public account, and like and share the post.