BOTANIC-1 is our new family of plant genomic Language Models (gLM). Today, we are releasing three models for research, ranging from 300M to 3B parameters.
The models were trained on the largest plant corpus ever used in the literature: 320 plant genomes, spanning 102 families and 48 orders. Data preparation and selection combined insights from the gLM literature and the rigorous data-centric discipline from the LLM world.
What to take home, an AI Research Scientist Perspective
BOTANIC-1 sets a new state-of-the-art (SOTA) for plant gLM. The models have single base pair resolution, with a context window of approximately 8 kbp at pre-training, subsequently extended to approx. 128 kbp.
We evaluated BOTANIC-1 against 20 published model families, generalists and plant specialists alike, on more than 34 tasks. This new benchmark mixes publicly available tasks with proprietary ones we plan to release publicly in the coming months.
From 22 tasks which do not necessitate any fine-tuning and which cover nine family of well-known applications, we derive a single score (S_bal) that we believe is an accurate representation of the biological strength of a model. On those frozen-model tasks, every BOTANIC-1 size ranks above PlantCAD2-L, the previous SOTA, and the performance increases monotonically with model size.
Scaling shows us where biology gets harder. Splice-site and gene-region detection are essentially resolved by ever recent model but zero-shot variant effect prediction is not. The performance on this task clearly improves with every increase in model size.
Efficiency is also a major strength of our models: BOTANIC-1S, our smallest model, matches or beat PlantCAD2-L with fewer than half of the trainable parameters and roughly 13x fewer pre-training tokens, while generalist models of 7 to 40 billion parameters trained on trillions of tokens underperform.
Zero-shot evaluation only goes so far. Biology is diverse enough that many tasks require specialization, and therefore fine-tuning. On these harder tasks, BOTANIC-1 models outperform all the models we benchmarked, whether other gLMs or strong from-scratch baselines widely used in the filed such as ChromBPNet. Below is an example on ATAC-seq prediction, more results are available in the technical report.
What to take home, a Plant Biologist Perspective
Understanding the genomic mechanisms behind a trait observed in the field can take years of work, thousands of plants and a pinch of luck. Genetics can help narrow the search to a region of the genome. But within that region, large blocks of DNA are inherited together, making it hard to separate the causal variant from all the variants that sit next to it. To separate them and get to the “genomic” resolution biologists need to draw a conclusion, more crosses, more plants and more seasons in the field are usually required.
BOTANIC-1 separates co-inherited variants in a different way. Instead of waiting for a recombination event to fall between them, it reads each in its sequence context and scores how much damage the changes is likely to cause, a judgement grounded in the DNA grammar learned from 320 plant genomes. That score ignores if two variants are inherited together; it can tell them apart even when no plant in a field separates them.
In a published melon sex-determination study, experimental mapping left 3,061 candidate variants on chromosome 2 that segregated together. From that list only, BOTANIC-1 ranked first the experimentally validated causal mutation, a single substitution in the ethylene-signalling gene CmEIN3.
“The real opportunity is not to replace genetics or experimental validation, but to dramatically reduce the search space, telling us which variants to test first, rather than having to separate them one by one through thousands of plants”
Adnane Boualem, Research Director @ INRAE
The genetics still tells you where to look. BOTANIC-1 tells you what to test first, and you no longer have to grow your way to the answer.
Learn More about BOTANIC-1
Hugging Face: Space, Model Weights, Datasets
Commercial use, collaborations or questions? 👉 hello@livingmodels.ai