Researchers have used state-of-the-art machine translation software — and some old-fashioned hunches — to crack the code used by a secret society in Germany three centuries ago. The results shed light on the tricks of the cryptographic process as well as on the bizarre history of such societies, which were all the rage in the 18th century.

It turns out that the 105-page, 75,000-character manuscript, known as the Copiale Cipher, provided a detailed description for setting up initiation ceremonies — including the techniques used to throw a scare into the initiates. It also revealed the methods that members used to identify each other in the outside world, and delved into the comparisons and rivalries surrounding Masonic-like rites in different countries.

"This opens up a window for people who study the history of ideas and the history of secret societies," Kevin Knight, a computer scientist at the University of Southern California's Information Sciences Institute, said in a news release issued today. "Historians believe that secret societies have had a role in revolutions, but all that is yet to be worked out, and a big part of the reason is because so many documents are enciphered."

Knight and his colleagues are now turning their attention to other, better-known cryptographic puzzles — such as the brain-teasing Kryptos sculpture on the CIA's grounds, the cipher used by the Zodiac Killer in 1969, and the totally baffling 15th-century Voynich Manuscript. But veteran code-breakers say those puzzles will be far tougher to solve. "Generally, that type of decryption has already been tried on those ciphers," said Elonka Dunin, whose website keeps tab on the world's top cryptological puzzles.

Knight said the work could eventually lead to better translation tools for non-Latin languages such as Pashto, Arabic, Chinese, Japanese and Korean, "which have been a big challenge for machines."

How the code was cracked
Tracking down the handwritten Copiale manuscript (which gets its name from one of the two readable words on the pages) was the first challenge facing Knight and two colleagues from Sweden's Uppsala University, Beata Megyesi and Christiane Schaefer. The book, bound in green and gold paper, turned up in the East Berlin Academy after the Cold War and is now in a private collection.

The researchers transcribed a machine-readable version of the coded text and put it through computerized statistical analysis. The software looked for patterns in the different combinations of coded characters, including Roman and Greek letters as well as abstract symbols.

At first, Knight and his colleagues focused on the Roman and Greek characters and tried to match them up with words from 80 different languages. "It took quite a long time, and resulted in complete failure," Knight said.

Then they played a hunch: Maybe those characters were actually meaningless "nulls," and the true code was contained in the abstract symbols. When they ran the symbols through statistical analysis, they came up with a German text titled "Ceremonie der Aufnahme" ... "Ceremonies of Initiation." Soon they had pages and pages of deciphered lore.

What the manuscript says
The text, apparently written in the 1760-1780 time frame, is "obviously related to an 18th-century secret society, namely the 'oculist order,'" the researchers say. The volume is inscribed "Phillipp 1866," perhaps suggesting that it passed into the hands of an owner named Phillipp in that year.

The manuscript, available in several formats from Uppsala University's website, describes the procedure for initiating new members of the society. At one point, candidates are asked to read the writing on a blank piece of paper. When they can't, they're told to put on eyeglasses, and then they undergo an "operation" that involves plucking a hair from the eyebrow. After the operation, the blank paper is replaced by a document laying out "the entire teaching for the apprentices."

Later, "the left part of the chest and the right knee get uncovered, the eyes are being tied, and all sorts of words of comfort are spoken, which raise even more fear." The candidates are told, "Prepare yourself to die" — but that's just a scare tactic.  No injuries are inflicted in the course of the ceremony.

USC / Uppsala University

The Copiale Cipher, used in an 18th-century book on secret society practices, used Roman and Greek characters as well as abstract symbols. The Roman and Greek characters proved to mere place-holders.

Another section of the book describes how members can recognize each other. When one member asks how "Hans" is, the other should respond by mentioning a name that begins with the second letter of the first name — for example, "He's with Anton."

Other passages discuss how much members at various levels of the secret society should know about the codes and customs. The manuscript notes that secret societies were established in England during the reign of Queen Elizabeth, "but because they practiced more evil than good, they have been destroyed." In Germany, societies in different cities are associated with different hand signs: a forefinger on the mouth for Berlin; the middle finger on the right eye and a thumb on the ear for Frankfurt; a forefinger on the chin for Marburg.

Some passages even take on political issues, referring to a three-headed monster as symbolizing "rule and governance which, by means of power and perfidy, deprive man of his natural freedom and enjoyment of the timely things and [that which] we human beings need." Such passages could help historians trace the influence of secret societies on the political movements of the time, which were notable for their focus on natural rights. The natural-rights concept set the stage for the American Revolution as well as the French Revolution.

What next?
Knight wants to use his machine-translation software on the Kryptos, Zodiac Killer and Voynich ciphers, but the cryptographers who have been working on those puzzles for years suspect that machines alone can't crack the code. Nick Pelling, an expert on the Voynich Manuscript and other ciphers, pointed out that human intuition played a big role in figuring out the Copiale Cipher.

"The story they outline in the paper is a classic hunch-based cipher-cracking sequence," Pelling told me. "They guessed one way, and then it turned out to be the other way. These are great hunches, and they tell a great story about how they followed these hunches and got to the end of the line."

He doubted that the work done on the Copiale Cipher could be adapted easily for the Voynich Manuscript. "It's pretty clear that it's a different type of cipher from the Copiale Cipher," he said. In fact, he suspects the manuscript, whose content is completely unknown, may be a combination of ciphers and idiosyncratic abbreviations that would be devilishly hard to untangle.

Dunin, who is the co-leader of a group trying to crack the Kryptos code, was similarly pessimistic about the researchers' chances for success. "They're welcome to try, but many machines have already been pointed at Kryptos," she told me.

Klaus Schmeh, a German crypto expert, said that even though the Copiale Cipher has been around for 250 years or so, it hadn't gotten much attention in the past. "In my view, this cipher wasn't known at all to the public," Schmeh said. He saluted the researchers for their work, but echoed Pelling's view that the effort fit the standard pattern for breaking secret codes.

"It's pretty much the way cryptography is done," he said. "It was certainly not an easy puzzle, but I'm sure that other cryptographers would have solved it."

Update for 6:55 p.m. ET: Knight responded via email to a few follow-up questions I sent him:

Cosmic Log: The Daily Mail suggests that the cipher was solved using the Google Translate software, but I'm assuming that it was a more specialized program.

Knight: The Daily Mail made a mistake.  Anyway, we used a bunch of software derived from our own statistical language translation algorithms.  We apply those original algorithms to the translation of Chinese and Arabic into English.

Q: Was the Copiale Cipher a straight substitution cipher, or was it something more complex?

A: It was a substitution cipher, but not a simple one-for-one type.  The cipher alphabet has many more than 26 letters.  So there are many ways to encode "E," for example.  Also, sometimes whole sequences of plaintext letters, for example "SCH," are encoded with a single cipher letter.  Lastly, there are some "logograms," cipher letters that stand for whole words, such as the name of the secret society.

Q: How could this method be applied to Voynich, Kryptos and other ciphers? Are there any wider applications for military code-making and code-breaking? Are there particular types of ciphers that the machine translation software is best suited for?

A: When you think about language translation, you can think about substituting a word in one language (like "boy" in English) with a word in another language (like "nanhaizi" in Chinese).  But sometimes whole phrases are substituted for whole phrases.  Also, there is reordering -- "transposition," in cryptographic jargon.  We pretend Chinese is a code for English -- a substitution/transposition cipher.  So there is a deep connection between translation and classical cryptography.  Of course, modern militaries use new cipher systems based on number theory now, so a lot of the classical work is not relevant anymore to them.  But it's super-relevant to us working on more accurate language translation algorithms.

Q: It sounds as if humans still played a key role...

A: Yes, it was a human/machine collaboration.  The machine has incredible patience, but it only looks for what you tell it.  We could tell it to decipher against 80 possible plaintext languages (Latin, English, German, etc.), and it had a slight preference for German, but it didn't know, for example, that a single cipher letter could stand for a sequence of three plaintext letters ("SCH"), because we didn't tell it that could happen.  But as a human, you are very flexible and can spot what is happening.

More secret messages:

• FBI can't crack murder code — can you?
• Computer program helps decode ancient texts
• Voynich Manuscript dated to the 15th century
• CIA sculpture still baffles cryptologists
• Has amateur cracked Zodiac Killer code?

Connect with the Cosmic Log community by "liking" the log's Facebook page, following @b0yle on Twitter or adding me to your Google+ circle. You can also check out "The Case for Pluto," my book about the controversial dwarf planet and the search for other worlds.