How did counting letters crack secret codes for centuries?

Secret messages that scramble every letter into a different symbol can still be cracked just by counting. In any long stretch of a language, some letters turn up far more often than others: in English, E, T, A and O are the most common, Z, Q, X and J are rare, and pairs like TH, ER, ON and AN show up constantly. A simple substitution cipher, where a given plaintext letter always becomes the same ciphertext symbol, cannot erase that pattern. So a codebreaker who counts the symbols in a scrambled message and matches the most frequent ones to the most frequent real letters can start prying it open, one guess at a time.

This trick has a documented origin. The first known recorded explanation of frequency analysis, and in fact of any kind of cryptanalysis, was given in the 9th century by the Arab polymath Al-Kindi, in a work called A Manuscript on Deciphering Cryptographic Messages. Some scholars suggest that close study of the Qur'an first revealed that Arabic writing has its own characteristic letter frequencies. Al-Kindi, who also oversaw the translation of Greek texts into Arabic in Baghdad's House of Wisdom, built on earlier work by al-Khalil; his manuscript is credited with the earliest known use of statistical inference. The idea later spread into Europe, and by 1474 the Italian statesman Cicco Simonetta had written his own manual on breaking Latin and Italian ciphers.

Codemakers fought back by muddying the pattern. Homophonic substitution gave a common letter several different disguises; polyalphabetic substitution, credited first to Leone Alberti, rotated between several alphabets; polygraphic substitution, used in the Playfair cipher, scrambled pairs of letters as a single unit. Each trick made frequency analysis harder, but also made encrypting correctly harder, which invites mistakes.

Counting letters stayed useful astonishingly long. In World War II, the British and Americans recruited codebreakers through newspaper crossword contests, and some Axis ciphers, including Japanese consular codes, still fell to simple counting. It took rotor machines like Enigma to finally defeat straightforward frequency analysis. Today a computer finishes the count in seconds.