Randomness: A highly valuable, and surprisingly elusive, resource

Good randomness is hard to find. Computers aren’t great at generating it; people are even worse. And that’s a problem for the industries that count it as a crucial resource. The multi-billion-dollar online security industry would be ineffective without random values. At its core, hacker-proof encryption requires a system that can generate reliable randomness. Such numbers are needed to keep elections honest, ensure the lottery is fair, produce scientifically accurate simulations, and to secure cryptocurrency. But the problem with randomness is that it’s not all created equal. In fact, some randomness is more random than others. Randomness is a lack of pattern, and the degree to which there is no pattern is known as entropy. A high-entropy source is completely chaotic: true randomness. Randomness that has some degree of order, even if tiny, is called pseudo-randomness. In this story, true randomness is the holy grail. And like the divine chalice itself, it’s tough to find. 🐦 [Tweet this!]( 🌐 [View this email on the web]( [Quartz Obsession] Randomness September 23, 2019 The quest for random --------------------------------------------------------------- Good randomness is hard to find. Computers aren’t great at generating it; people are even worse. And that’s a problem for the industries that count it as a crucial resource. The multi-billion-dollar online security industry would be ineffective without random values. At its core, hacker-proof encryption requires a system that can generate reliable randomness. Such numbers are needed to keep elections honest, ensure the lottery is fair, produce scientifically accurate simulations, and to secure cryptocurrency. But the problem with randomness is that it’s not all created equal. In fact, some randomness is more random than others. Randomness is a lack of pattern, and the degree to which there is no pattern is known as entropy. A high-entropy source is completely chaotic: true randomness. Randomness that has some degree of order, even if tiny, is called pseudo-randomness. In this story, true randomness is the holy grail. And like the divine chalice itself, it’s tough to find. 🐦 [Tweet this!]( 🌐 [View this email on the web]( Giphy By the digits [10%:]( Share of the internet that Cloudflare keeps safe using randomness [20:]( Number, from 1-49, that came up least in the first 20 years of Britain’s national lottery, 204 times [23:]( Number that came up most, 266 times [10,231:]( “Heads-up” coin tosses that landed on heads, out of 20,000 flips by a University of California, Berkeley student [10,014:]( “Tails-up” coin tosses that landed on tails, out of 20,000 flips by a different Berkeley student [8,300:]( “Randonauts” on Reddit, who are attempting to harness randomness to travel the multiverse by converting random numbers into location data in the hopes of creating “reality tunnels” [92:]( Consecutive times a flipped coin lands on heads in the opening scene of Tom Stoppard’s Rosencrantz and Guildenstern Are Dead [£15,000 ($27,300):]( Amount the band Wilco paid to settle a copyright claim over the [“numbers station”]( sample that gave its 2002 album Yankee Hotel Foxtrot its name Explain it like I’m 5! Harnessing nature’s chaos --------------------------------------------------------------- Computers tend to produce pseudo-random numbers, and they do so via a random number generator (RNG). RNGs are valuable because they can spit out many, many numbers quickly and cheaply. But ultimately they’re just algorithms, meaning that they’re deterministic; their behavior is set by their initial state and inputs. In other words, the same input will always produce the same output. In order to begin producing random numbers, an RNG needs a starting input, a value called a “seed.” [If you figure out the seed]( you can figure out all the random numbers it produces. Not very random or secure. So you need a random, unpredictable seed as well. Since neither computers [nor humans]( can produce this reliable randomness, we need to turn to nature. There are a handful of natural processes that defy order. It is impossible to predict radioactive decay, the motion of a double pendulum, the swirl of turbulence, or background radiation. These are all high entropy systems, and are considered true randomness—so is the bubble of oil in a [lava lamp](. Internet encryption company[Cloudflare famously uses lava lamps]( to seed their RNG. Among other sources of entropy, pictures of the groovy wall in their San Francisco office are fed into a computer and used to kick-start their random number generation. Giphy Quotable “Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.” —Mathematician [John von Neumann]( Amanda Shendruk Light reading A Million Random Digits --------------------------------------------------------------- Random numbers were a necessary resource even before computers could produce them quickly and cheaply. So in 1955 the RAND Corporation published hundreds of pages of random numbers in a book creatively dubbed [A Million Random Digits with 100,000 Normal Deviates](. The digits were produced using an electronic roulette wheel and it was the first time such a large and rigorously tested table of random numbers had been put into print. The tables were also available to order on [punch cards](. (One Amazon reviewer is unimpressed: “A Million Random Digits? HA! They only used 10, and just kept repeating them in different combinations! Don’t be fooled!”) How’s this for random: The New York Public Library originally shelved the book under “psychology.” Reuters/Steve Dipaola Pop quiz Which city was named with a coin flip? Poulsbo, WashingtonBennettsville, South CarolinaSaskatoon, Saskatchewan Portland, Oregon Correct. Its founding fathers, hailing from Portland, Maine and Boston, Massachusetts, went two-out-of-three to pick which would be its namesake. Incorrect. If your inbox doesn’t support this quiz, find the solution at bottom of email. Highly irregular Public randomness --------------------------------------------------------------- Thanks to some altruistic organizations, high entropy numbers are available for use by anyone. They’re called randomness beacons, and they’ve been developed by universities and public corporations. Last year, the [University of Chile]( unveiled a public online random number service to help their comptroller general with its audits. They source their entropy from seismic measurements of the earth, radio waves, and a selection of Twitter posts. In the US, the National Institute of Standards and Technology (NIST) first launched its [Randomness Beacon]( in 2013. And just this year, a beacon from the grandiosely named [League of Entropy]( (“not all heroes wear capes”) was unveiled. The League consists of Cloudflare, Protocol Labs, universities in Chile and Switzerland, Kudelski Security, and others who mix their individual sources of entropy to generate publicly verifiable random values [every minute](. Have a friend who would enjoy our Obsession with Randomness? [ [Forward link to a friend](mailto:?subject=Thought you'd enjoy.&body=Read this Quartz Obsession email – to the email – Reuters/Toru Hanai Brief history [2400 BC:]( The earliest dice in the archaeological record are made in Mesopotamia. [1450:]( The first standardized, balanced dice are created. [1716:]( The first roulette wheel is developed. [1890:]( Francis Galton’s [“dice for statistical experiments”]( are “perhaps the oldest surviving device for simulating normally distributed random numbers.” [1939:]( M.G. Kendall and B. Babington Smith design an electromechanical random number generator and use it to create and publish a table of 100,000 random numbers. [1946:]( The US cracks a vital Soviet intelligence code after the NKVD can’t generate random numbers fast enough. [1949:]( John von Neumann describes the middle-squares method, likely the first pseudo-random number generating algorithm (although [its roots are in the 13th century](. [1957:]( The UK introduces ERNIE, which uses vacuum-tube noise to generate lottery numbers. [1968:]( IBM releases the “infamous,” [“notorious” RANDU]( which computer scientist [Donald Knuth calls]( “a truly horrible random number generator”; nonetheless, it goes into wide use. [1995:]( Two University of California, Berkeley students reverse-engineer how the early Netscape Navigator web browser creates seeds for its random-number generator. [1997:]( The [Mersenne Twister]( “a 623-dimensionally equidistributed uniform pseudo-random number generator,” is introduced; it remains possibly the most common PRNG. [2011:]( Russian scammers begin beating the odds on one company’s slot machines after reverse-engineering the PRNG. Million-dollar question What about quantum randomness? --------------------------------------------------------------- Physics, you’ve done it again! Scientists have already determined how to create a [quantum-based random number generator](. Last year, researchers [publishing in Nature]( explained how they harnessed quantum mechanics to produce truly random numbers. Because the measurements of quantum particles are fundamentally unpredictable, quantum mechanics can be used to generate random numbers. In fact, the current iteration of the National Institute of Standards and Technology’s (NIST) randomness beacon relies in part it. So why aren’t we all using quantum random number generation? There are two fundamental problems: the process is slow, and the apparatus is large. Right now, it takes 10 minutes to produce 1,024 bits of data, which, [according to Wired]( is the equivalent of typing 13 letters per minute. And the set-up itself fills an entire building at a NIST facility in the Rocky Mountains. Fun fact! Not all scientists agree that true randomness even exists. Einstein famously claimed that God doesn’t play dice. He believed there is a yet-unknown underlying order to everything in the universe. Stephen Hawking addressed the theory in[a lecture]( saying “God does play dice with the universe. All the evidence points to him being an inveterate gambler, who throws the dice on every possible occasion.” Watch this! Here’s how a random number generator works—as applied to Super Mario 64. take me down this 🐰 hole! [Game development]( is one of the few industries that values pseudo-randomness. Random number generation is used in gaming, for example, to introduce unpredictability. However, the randomness needs to be reproducible, so the software can be tested and so game mechanics happen the same for every player. Apple’s [GameplayKit Programming Guide]( explains how to achieve the appearance of unpredictability. Giphy Poll Which of these hard-to-price things do you think is most valuable? [Click here to vote]( True randomnessA piña colada; getting caught in the rainThe smell of freshly baked cookiesEinstein’s brain 💬let's talk! In Friday’s poll about [pumpkin spice lattes]( 48% of you said “I’ll pumpkin spice pass.” 📧 Chris writes: “Umm, Got the biggest pumpkin fact wrong…. [2,015 lbs in Alaska]( That’s bigger than the one mentioned in Friday’s email, but even that seems to have been surpassed by [Mathias Willemijns’s 2,246 lb (1,019 kg) beast]( in 2016. 🤔 [What did you think of today’s email?](mailto:obsession%2Bfeedback@qz.com?cc=&subject=Thoughts%20about%20randomness&body=) 💡 [What should we obsess over next?](mailto:obsession%2Bideas@qz.com?cc=&subject=Obsess%20over%20this%20next.&body=) 🎲 [Show me a random Obsession]( Today’s email was written by[Amanda Shendruk]( edited by [Whet Moser]( and produced by [Luiz Romero](. The correct answer to the quiz is Portland, Oregon. Enjoying the Quartz Obsession? [Send this link]( to a friend! Want to advertise in the Quartz Obsession? Send us an email at ads@qz.com. Not enjoying it? No worries. [Click here]( to unsubscribe. Quartz | 675 Avenue of the Americas, 4th Fl | New York, NY 10011 | United States [Share this email](