Skip to content

Phytozome dialects

Dialect schemas for a Phytozome genome's GFF3 and annotation table, and their mappings to the feature model. Conversion profiles says what a dialect schema is.

Two more dialects describe a Phytozome genome's gene_exons.gff3 and its per-transcript annotation_info.txt, items 4 and 5 of https://github.com/turbomam/feature-table-corpus/issues/54. Both were measured on 2026-09-25 against one genome only, Arabidopsis thaliana TAIR10 (Phytozome-167 in the JGI input list): 532,682 GFF3 rows and 35,386 table rows. TAIR restricts redistributing substantial subsets of TAIR10, so no row of either file is in this repository; the tests use a constructed fixture.

  • The GFF3 opens with ##gff-version 3 and ##annot-version TAIR10, then gene blocks: a gene row, then each mRNA row followed by its exon, CDS and UTR rows. There is no score, no percent escape, and no comma in column 9, and each row type writes the same keys in the same order.
  • Each part has its own ID, the mRNA's ID plus its type and a number, as in X.1.TAIR10.CDS.2. Numbers run in transcription order, so on the minus strand coordinates descend. A gene or mRNA ID is its Name plus .TAIR10.
  • Every mRNA row has longest=0 or longest=1, and each gene has exactly one longest=1 (27,416 genes, 7,970 other isoforms). The flag is Phytozome's choice: in 16 of the 5,804 genes with more than one isoform, the flagged one does not have the longest CDS. The feature model carries it as SequenceFeature.is_representative (https://github.com/turbomam/feature-table-corpus/issues/48).
  • Each mRNA's pacid appears once in the table, as PAC:<pacid>, with the mRNA's Name as transcriptName and its gene's Name as locusName. All 35,386 match in both directions.
  • In the table, one space separates values in Pfam, Panther, ec, KOG, KO and GO. ec holds four-part EC numbers, plus the non-EC token EC:PROLINE-MULTI in six rows, kept as written. Partial EC numbers such as EC:1.1.1.- are rejected, since none was read.
  • Both files use LF line endings and end with a newline; a carriage return, a missing final newline, or a file with no data rows is rejected.

just dialect-validate-phytozome-gff3 FILE reads plain or gzip-compressed GFF3 as a stream and validates rows in chunks of 5,000, so the whole genome is never one document. On the TAIR10 file it took 23.5 seconds of wall time on an Apple M5 Max on 2026-09-25. just dialect-validate-phytozome-annotation FILE validates the table, and just dialect-join-phytozome GFF3 FILE checks that the two files name the same transcripts. Beyond the schemas, the GFF3 checks cover ID construction, block order, part numbering, one longest=1 per gene, and spans: an mRNA spans its exons, each CDS and UTR sits inside an exon, and a gene spans its mRNAs. Each writer, run on the TAIR10 files on 2026-09-25, reproduced them byte for byte. There is no conversion profile for these dialects yet.

A second genome, Populus trichocarpa v4.1 (Phytozome-533), was measured on 2026-09-30: 849,978 GFF3 feature rows (849,981 lines) and 52,400 table rows. Its GFF3 differs from TAIR10's in two ways the gene_exons dialect now accepts. It opens with a third directive, ##species Populus trichocarpa. And genes and mRNAs may carry ancestorIdentifier, their v3.1 identifier: on 30,924 of 34,699 genes and 32,185 of 52,400 mRNAs, always after Name on a gene and before Parent on an mRNA. A 200-gene excerpt is in the repository, and the gene_exons mapping round-trips it byte for byte in CI (model/examples/clean-round-trips.yaml).

The Populus table has its own header layout, which the annotation_info dialect now reads and writes back byte for byte, all 52,400 rows (measured 2026-09-30). It has 12 columns, with best_arabi_gene and best_arabi_defline in place of TAIR10's clamy and rice best hits. Its header labels column 7 KOG and column 8 ec, but all 17,431 values under KOG are EC numbers and all 18,574 under ec are KOG IDs. So the columns are in TAIR10's order with the two labels swapped, and the parser puts each value in the slot it is; JGI's files API lists a sibling file named ...annotation_info.txtwrong_header. The mapping reads both layouts. It keys each value by what it is, so Populus's EC numbers are ec Attributes despite their column's label. A Populus peptideName is the mRNA's Name plus .p; in 560 rows it names another locus's transcript instead, and the mapping keeps that one as a peptideName Attribute. Reverse reads the layout from the Attributes, or from --layout. The table rows of the same 200 genes are vendored too, and the two excerpts round-trip together byte for byte in CI, making annotation_info clean as well (https://github.com/turbomam/feature-table-corpus/issues/144).

phytozome-gene-exons-gff3.transform.yaml maps the GFF3 rows to SequenceFeature with linkml-map, the same way as the IMG functional annotation, and scripts/phytozome_gene_exons_map.py adds the same four things linkml-map can't do. Gene, mRNA and part rows each become a SequenceFeature, joined by parent. longest sets SequenceFeature.is_representative (1 true, 0 false) and, like ID and Parent, keeps its attribute copy, which the reverse step requires to agree; Name and pacid have no SequenceFeature slot and travel only as attributes. The two directives are not model data: the reverse step writes ##gff-version 3 and recovers the annot-version from the gene IDs, each of which is the gene's Name, a dot, and the annot-version (.TAIR10 in the Arabidopsis file, .EXv1 in the fixture); all genes must agree. It rebuilds each row's text and parses it with the dialect's own row parser, then maps the result forward again and requires the Dataset back.

just map-phytozome-gff3-roundtrip FILE requires the written file to equal the source byte for byte. On the TAIR10 file on 2026-09-28 it held for all 532,682 rows (1,641,402 attributes) in 211 seconds on an Apple M5 Max, with a peak of 3.2 GB of memory. just map-phytozome-gff3 FILE OUT and just map-phytozome-gff3-back DATASET OUT run each direction and never overwrite a file.

scripts/phytozome_annotation_map.py maps the GFF3 and its annotation_info.txt together into one Dataset. The table has no positions, so each of its values becomes one Attribute on the mRNA its pacId names and placed after the GFF3's own attributes. Each is keyed by what it is: Pfam, Panther, ec, KOG, KO, GO, then the layout's best-hit columns, the four Best-hit columns in TAIR10 or best_arabi_gene and best_arabi_defline in Populus. Populus's header swaps the KOG and ec labels, and its EC numbers are still keyed ec. A list column gives one Attribute per value in table order. The table's URL goes in the mRNA's source_files, since the model has no provenance per attribute. locusName and transcriptName are not stored: they are the gene's Name and the mRNA's Name, and a row where they differ is refused. peptideName is the mRNA's Name in all 35,386 TAIR10 rows, and a TAIR10 row where it differs is refused. In Populus it is the Name plus .p, and in the 560 rows where it names another transcript the mapping keeps it as a peptideName Attribute. The reverse step reads the layout from the best-hit or peptideName Attributes. When there are none, it can't tell the two layouts apart, so it refuses unless --layout is given. The reverse step writes rows sorted by locusName and then transcript number, which is the TAIR10 table's own order (it is not the GFF3's mRNA order), and forward refuses a table in any other order. It also refuses an mRNA whose GFF3 attributes already use one of the table's column names. just map-phytozome-annotation-roundtrip GFF3 TABLE requires both files back byte for byte; on 2026-09-28 it held for the TAIR10 pair (35,386 mRNA carrying 465,471 table attributes) in 266 seconds with a peak of 3.5 GB of memory.