The one-sentence version
DNA is a "text" consisting of billions of letters written with four characters: A, C, G, and T. Since Large Language Models can learn grammar and semantics from massive amounts of text, can we use the same method to pre-train on massive genomic sequences to learn the "grammar" of the genome: where the genes are, what controls the gene switches, and which mutations are harmful? In recent years, DNA language models (such as Nucleotide Transformer, Evo, Evo 2) and genomic sequence-to-function prediction models (such as Enformer, AlphaGenome) have provided quite positive answers, and have even begun attempting to generate new functional DNA sequences.
Regarding Timeliness
This article is organized based on public information available at the time of writing and has not been cross-checked online item by item.
Layer 1: The Intuition
1.1 A Book Without Punctuation
The human genome contains approximately 3 billion base pairs. Written down, it is equivalent to thousands of thick books, consisting entirely of A, C, G, and T, with no spaces, no punctuation, and no chapter headings.
We know that within it:
- Only about 1–2% directly encode proteins ("genes");
- The rest is heavily involved in regulation: determining which gene is expressed in which cell, at what time, and at what intensity;
- Many disease-related mutations fall within these regulatory regions, but it is difficult for us to decipher their function.
1.2 Learning DNA Like Learning a Language
| Natural Language Model | DNA Language Model |
|---|---|
| Corpus: Internet text | Corpus: Genomes from thousands of species |
| Learns: Grammar, semantics, common sense | Learns: Gene structure, regulatory elements, evolutionarily conserved patterns |
| Task: Predict the next (or masked) word | Task: Predict the next (or masked) base |
| Application: Answering questions, generating text | Application: Determining if mutations are harmful, annotating genomes, designing new sequences |
A key idea is: Evolution itself is a massive "experiment." Sequence patterns preserved through billions of years of evolution are often functional; if a model determines that a certain mutation "does not look like normal DNA," that mutation is very likely to be harmful.
Layer 2: Representative Works
| Model | Institution / Time | Key points |
|---|---|---|
| DNABERT | 2021 | Applied the BERT masking pre-training method to DNA |
| Enformer | DeepMind, 2021 | Not a pure language model, but a model that predicts gene expression and epigenetic signals directly from sequences; can view a context of approximately 200,000 bases, significantly improving the prediction of distal regulatory elements |
| Nucleotide Transformer | InstaDeep et al., 2023 | A large model pre-trained on thousands of human genomes and hundreds of species' genomes |
| HyenaDNA, Caduceus, etc. | 2023–2024 | Uses non-attention long-sequence architectures to push the context window to the million-base scale |
| Evo | Arc Institute et al., published in Science, 2024 | A generative model trained on vast amounts of prokaryotic and bacteriophage genomes; can handle multiple levels including DNA, RNA, and protein; generated functional CRISPR systems and transposons verified by experiments |
| Evo 2 | Arc Institute, NVIDIA, etc., 2025 | Scaled up to tens of billions of parameters, with training data covering a vast number of species on the tree of life (including eukaryotes); context length of approximately one million bases; performs well in predicting the pathogenicity of human gene (e.g., BRCA1) mutations without specialized training; released as open source |
| AlphaGenome | DeepMind, 2025 | Takes a sequence of approximately one million bases as input and simultaneously predicts thousands of functional signals (gene expression, splicing, chromatin accessibility, 3D contacts, etc.) to evaluate the impact of non-coding mutations |
Layer 3: Applications, Limitations, and Risks
3.1 Real-world Use
| Application | Description |
|---|---|
| Variant Effect Prediction | Patient genome sequencing reveals tens of thousands of variants, most of which are of "uncertain significance." Models can help rank which are most likely pathogenic |
| Genome Annotation | Rapidly identifying genes and regulatory elements in newly sequenced species' genomes |
| Non-coding Region Research | Understanding how the non-coding regions, which make up the vast majority of the genome, regulate genes |
| Synthetic Biology | Designing promoters and enhancers to precisely control gene expression in specific cells; generating new functional elements |
| Integration with Protein Models | The complete chain of DNA → RNA → protein (see "AlphaFold and Protein Structure Prediction") |
3.2 Limitations
| Limitation | Description |
|---|---|
| "Looking like normal DNA" ≠ "Normal function" | Model scores primarily reflect evolutionary frequency, which is only correlated with actual biological effects |
| Cell-type specificity | The same DNA sequence acts differently in different cells; pure sequence models struggle to capture this |
| Long-range interactions | Regulatory elements can be hundreds of thousands or even millions of bases away from a gene; context length remains a bottleneck |
| Diversity of human data | Insufficient representation of certain populations in training data may lead to varying prediction accuracy across different populations |
| Experimental validation | Generated sequences must undergo wet-lab validation; current validation scale is much smaller than the generation scale |
3.3 Biosecurity
Models capable of "writing" DNA bring concerns regarding dual-use: they could theoretically be used to design harmful biological sequences. Responsible practices include:
- Excluding specific pathogens from training data (for example, Evo 2's training data excludes viruses that infect eukaryotes);
- Conducting safety assessments before release;
- Screening orders by DNA synthesis companies.
Summary
| Layer | Key points |
|---|---|
| Intuition | Treating the genome as a language for pre-training; evolutionary conservation = functional clues |
| Representatives | Enformer, Nucleotide Transformer, Evo / Evo 2, AlphaGenome |
| Real-world Use | Pathogenicity prediction of variants, understanding non-coding regions, synthetic biology |
| Limitations and Risks | Correlation ≠ causation, cell specificity, long-range regulation, population bias, biosecurity |