The one-sentence version
Generating an RSA key means finding two primes hundreds of digits long. The method is "pick a random odd number, test it, and if it fails, pick another". Fermat's little theorem gives the simplest test: if is prime, then . But one class of composites (Carmichael numbers, the smallest being ) fools every base. Miller–Rabin closes this hole using the fact that modulo a prime, has only the two roots .
This article has three layers:
- The intuition: why primality can be decided without factoring, and the idea of "random spot checks"
- Real-world use: RSA key generation, OpenSSL and accidents in prime generation
- The mathematics: the Fermat test, Carmichael numbers, why Miller–Rabin is correct and a worked example with
Layer 1: The Intuition
1.1 Deciding primality is far easier than factoring
"Is prime?" and "which two numbers multiply to ?" sound like the same question, but their difficulty is worlds apart:
- Deciding whether a 1000-digit number is prime: under a second on a laptop;
- Factoring a 600-digit (2048-bit) RSA modulus: beyond all the world's computers put together today.
The remarkable part is that a primality test can declare a number composite without finding any factor. It is like a counterfeit detector: it doesn't need to know who printed the fake, only that some feature is wrong.
1.2 Are there enough primes?
The prime number theorem says that near , about one number in is prime. For a 1024-bit number, ; picking only odd numbers, you hit a prime after about 355 tries on average. Each test is fast, so generating primes isn't slow.
1.3 The "spot check" idea
Primes always satisfy certain properties (such as Fermat's little theorem). We pick some random bases and check:
- If the check fails even once, the number is definitely composite;
- If it passes every time, the number is very likely prime.
The key is designing a check where a composite's chance of slipping through is low enough. Each round of Miller–Rabin is fooled with probability at most . After 40 rounds, the chance of being fooled is at most , lower than the chance of a hardware fault.
Layer 2: Real-World Use
- RSA key generation: OpenSSL, BoringSSL and hardware security modules all use a combination of "trial division by small primes + Miller–Rabin". Trial division by the first few hundred small primes quickly eliminates most candidates.
- Diffie–Hellman parameters: generating a "safe prime" requires both and to be prime.
- Standards: US FIPS 186-5 specifies the number of Miller–Rabin rounds for key generation, sometimes adding a Lucas test (the combination is called Baillie–PSW, and no composite that fools it has been found so far).
What happens when prime generation goes wrong
- Randomness that isn't random: a 2012 scan of RSA public keys across the internet found that many routers and firewalls, short of entropy at boot, generated keys with shared prime factors, which a single factors (see "Why RSA Decrypts Correctly").
- Primes with structure: in the ROCA vulnerability (2017), a chip used primes of a special form for speed, and the results could be factored.
- "Pseudoprimes" built by an adversary: the 2018 study Prime and Prejudice found that some cryptographic libraries, when testing numbers supplied from outside, used too few rounds or fixed bases, so attackers could build composites that fool them and inject insecure DH parameters.
Layer 3: The Mathematics
3.1 The Fermat test
Fermat's little theorem: if is prime and , then . (This is Lagrange's theorem for , whose order is .)
The test: pick a random and compute . If it isn't , is definitely composite.
Most composites fail right here. For example with , , and it is exposed at once.
3.2 Carmichael numbers
There is a class of composites satisfying for every coprime to them, called Carmichael numbers.
Korselt's criterion: a composite is a Carmichael number if and only if is squarefree and for every prime factor .
The smallest example is : is divisible by , and . By the Chinese Remainder Theorem, is modulo each of , and , and so also modulo .
In 1994 Alford, Granville and Pomerance proved there are infinitely many Carmichael numbers. So the Fermat test alone won't do.
3.3 The key lemma: modulo a prime, has only two square roots
If is an odd prime and , then . Because is a field (it has no zero divisors), either or .
Conversely, if you find with , then has zero divisors and must be composite (and is a nontrivial factor).
3.4 Miller–Rabin
Write as with odd. Pick a random and look at this sequence:
Each term is the square of the previous one. If is prime, the last term is by Fermat's little theorem; and by the lemma in 3.3, taking square roots backwards, the first departure from must be . So a prime always satisfies:
Either , or some term of the sequence equals .
If this fails, the number is declared composite. Rabin proved that for a composite , at most of the bases can fool this test.
3.5 A worked example: exposing
, base
Fermat test: . It passes, and the test is fooled.
Miller–Rabin: , so and .
| Term | Value |
|---|---|
The sequence jumps straight from to without passing through . That is, but , which can't happen modulo a prime, so 561 is composite.
We even get factors along the way: and . (Checked with Python.)
3.6 Deterministic algorithms
In 2002 Agrawal, Kayal and Saxena gave the AKS algorithm, the first proof that primality can be tested in deterministic polynomial time (the paper is titled PRIMES is in P). It is much slower than Miller–Rabin, though, so probabilistic algorithms are still used in practice.
Summary
| Layer | Key points |
|---|---|
| Intuition | Deciding primality is far easier than factoring; spot check at random, and one failure means composite |
| Algebra | Fermat's little theorem comes from Lagrange's theorem; Carmichael numbers fool it |
| The key | In a field, has only the roots ; any other square root proves compositeness |
| Engineering | Trial division + many rounds of Miller–Rabin; neither randomness quality nor round count can be skimped |