![[Feedback]](/icons/banner/feedbackACT.gif){kind=icon}
![[For Subscribers]](/icons/banner/subscriberACT.gif){kind=icon}
![[Archive]](/icons/banner/archiveACT.gif){kind=icon}
![[Search]](/icons/banner/searchACT.gif){kind=icon}
![[Contents]](/icons/banner/tocACT.gif){kind=icon}
|
|
|
|
|
||
|
LETTER Leveraging the Mouse Genome for Gene Prediction in Human: From Whole-Genome Shotgun Reads to a Global Synteny Map1Department of Computer Science and Engineering and2 Department of Biomedical Engineering, Washington University, St. Louis, Missouri 63130, USA
The availability of draft sequences for both the mouse and human genomes makes it possible, for the first time, to annotate whole mammalian genomes using comparative methods. TWINSCAN is a gene-prediction system that combines the methods of single-genome predictors like GENSCAN with information derived from genome comparison, thereby improving accuracy. Because TWINSCAN uses genomic sequence only, it is less biased toward highly and/or ubiquitously expressed genes than GENEWISE, GENOMESCAN, and other methods based on evidence derived from transcripts. We show that TWINSCAN improves gene prediction in human using intermediate products from various stages of the sequencing and analysis of the mouse genome, from low-redundancy, whole-genome shotgun reads to the draft assembly and the synteny map. TWINSCAN improves on the prior state of the art even when alignments from only 1X coverage of the mouse genome are available. Gene prediction accuracy improves steadily from 1X through 3X, more slowly from 3X to 4X, and relatively little thereafter. The assembly and the synteny map greatly speed the computations, however. Our human annotation using the mouse assembly is conservative, predicting only 25,622 genes, and appears to be one of the best de novo annotations of the human genome to date.
One important motivation for sequencing the mouse genome was to aid in the discovery of human genes. Before the sequencing of the mouse genome, analysis of cDNAs showed that orthologous mouse and human coding exons are typically conserved at 75%–95% nucleotide identity, with an average of 85% (Makalowski et al. 1996  ; Ansari-Lari et al. 1998; Makalowski and Boguski 1998). This
is much higher than the average level of nucleotide
conservation in noncoding regions. Intensive study of specific genomic
regions showed that inspection of percent identity plots (PIPS) in
combination with GENSCAN predictions (Burge and Karlin 1997), cDNA
alignments, and other database searches were useful tools for expert
genome annotation (Ansari-Lari et al. 1998; Schwartz et al. 2000). A
relatively simple algorithm based on finding open reading frames with
>50% identity was able to locate a missing exon in a low-expression
gene (Jang et al. 1999), contributing to a sense of great optimism that
mouse–human conservation would reveal both coding and regulatory
regions (Bouck et al. 2000). Enthusiasm for this approach was
motivated, in part, by the observation that genomic conservation of
coding regions is likely to be evident even for genes that are
expressed at low levels or under very specific conditions and hence
cannot easily be identified by ESTs or cDNAs. (Jang et al. 1999).
These and other regional studies also noted the presence of variable
but non-negligible levels of conservation in noncoding regions,
typically covering about 50%–100% as much genomic sequence as the
coding regions (Koop and Hood 1994
One of the first gene predictors to substantially exceed the
performance of GENSCAN on a genomic scale by using mouse–human
comparison was TWINSCAN (Korf et al. 2001 TWINSCAN begins with local alignments between a target genome and a database of sequences from an informant genome. These alignments are converted into a representation called conservation sequence, which assigns one of several symbols to each nucleotide of the target genome. The version of TWINSCAN described in this paper (TWINSCAN 1.1) uses three different symbols. Each nucleotide of the target genome is paired with one of these symbols if the highest-scoring local alignment overlapping that nucleotide contains a match, another symbol if the highest-scoring alignment contains a gap or mismatch, and a third symbol if there is no overlapping alignment.
TWINSCAN, SGP2, and SLAM (Pachter et al. 2002
Korf et al. (2001)
To evaluate the performance of TWINSCAN, we compared its predictions on the human genome with an alignment of the RefSeq mRNAs (Pruitt and Maglott 2001 ; Kent et al. 2002). RefSeq alignments with obvious errors
were removed leaving 14,060 transcripts at 12,516 nonoverlapping
genomic loci (see Methods). Predicted genes were counted as correct if
they exactly matched the coding region of one RefSeq transcript at a
given locus, and predicted exons were counted as correct if they
exactly matched a RefSeq coding exon. Sensitivity (number predicted
correctly over total number annotated by RefSeq) and specificity
(number predicted correctly over total number predicted) were
calculated for exact prediction of genes, exons, and coding
nucleotides. Sensitivity as compared with RefSeq can be considered a
reasonable estimate of sensitivity for all genes, to the extent that
RefSeq genes are a representative sample of all genes. Specificity as
compared with RefSeq, however, will count all correct predictions for
which there is no RefSeq transcript as errors. Because the RefSeq set
contains transcripts for less than half of all human genes, based on
current estimates, specificity is systematically underestimated. With
this caveat, however, RefSeq specificity can still be used to compare
different prediction sets (this measure is sometimes called
relative enrichment of the prediction set, but we will use
the term specificity).
Effects of Mouse Shotgun Read Redundancy, Assembly, and Synteny Analysis
TWINSCAN's results directly mirror the characteristics of the alignments themselves. Specifically, the fraction of the human genome covered by mouse alignments rose steadily up to 3X, more slowly from 3X to 4X, and very little from 4X to the assembly (Fig. 2), as expected (Lander and Waterman 1988 ).
The coverage of the genome declined slightly when alignments were
restricted to regions of conserved synteny, also as expected. More
telling is the conditional uncertainty of the RefSeq
annotation, given the alignment (see Methods), an information-theoretic
measure reflecting how reliably patterns in the conservation sequence
indicate coding regions. Conditional information of zero would mean
that the coding status of each nucleotide could be computed exactly
from the conservation symbol aligned to it. The marginal information
content of the RefSeq annotation (i.e., when no conservation sequence
is given) is 0.053 bits per nucleotide, reflecting the fact that only
0.62% of the nucleotides are annotated as coding. Therefore, one could
correctly determine the coding status of most nucleotides by simply
guessing that all are noncoding. As shotgun coverage increases, the
conditional uncertainty (given the conservation sequence) declines
rapidly up to 3X and more slowly from 3X to the draft assembly (Fig.
2). Conditional uncertainty rises when alignments are constrained to
regions of conserved synteny. These observations demonstrate that the
effects of mouse genome coverage, assembly, and synteny map on
TWINSCAN's performance do not result from complex interactions
particular to TWINSCAN, but rather from characteristics of the
alignments themselves.
Although moving from 4X shotgun coverage to assembled sequence and then to syntenic sequence has limited effects on accuracy, it dramatically speeds the computation. Aligning each 1X mouse-reads database (with repetitive and low-complexity sequence removed) to the human genome with BLASTN requires  1,700 CPU hours on a cluster of current
high-end commodity processors (2 Ghz x 86 processors with 1GB RAM per
processor; see Methods for BLAST parameters). Moving from 4X
whole-genome shotgun reads to the draft assembly reduces alignment time
from 6,868 CPU hours to 2,068 CPU hours. Aligning only blocks of
conserved synteny reduces alignment time to only 35 CPU hours.
Annotation of the Human Genome Using the Assembled Draft Sequence of the Mouse Genome
Close examination of individual genes provides additional insight into how TWINSCAN improves exact gene prediction. For example, TWINSCAN predicts the transcript shown in Figure 4 perfectly, whereas GENSCAN calls an extra exon, misses two exons, miscalls a number of splice sites, and continues the prediction beyond the end of the coding region. TWINSCAN does not insert the extra exon that GENSCAN does, in part because the mouse alignment (black) stops in the middle of this predicted exon (Fig. 4B). Similarly, two of GENSCAN's miscalled splice sites are not covered by mouse alignments, whereas TWINSCAN correctly predicts splice sites that are covered by mouse alignments (Fig. 4C). Figure 4 also highlights the fact that TWINSCAN does not predict coding exons in every aligned region, nor does every TWINSCAN coding exon overlap an aligned region.
TWINSCAN produces a conservative annotation of the human genome that contains 25,622 genes comprising 198,284 exons and 34,290,737 coding nucleotides (1.15% coding). This gene number is consistent with the low range of recent estimates for total mammalian gene count (Mouse Genome Sequencing Consortium 2002 ) and remarkably close to the latest
ExoFish (Roest Crollius et al. 2000) estimate of 25,925 human genes
(Roest Crollius, pers. comm.). In comparison, the Ensembl gene build
(Hubbard et al. 2002), an annotation system requiring evidence of
transcription in some organism for every exon, predicts 22,980 human
genes comprising 182,922 unique exons (Release 8.30a.1,
http://www.ensembl.org/Homo_sapiens/). NCBI's less conservative
annotation starts with 52,842 genes predicted by GenomeScan (Yeh et al.
2001) and culls these to 34,539 using additional evidence
(http://www.ncbi.nlm.nih.gov/genome/guide/human/HsStats.html).
Except for TWINSCAN, all gene counts described above include
200–300 genes on the Y chromosome. The number of genes and exons in
the TWINSCAN, GENSCAN, and RefSeq annotations (excluding chromosome Y)
and the proportions that match exactly can be visualized as a Venn
diagram (Fig. 5). Note that many of the
genes and exons that do not match exactly are highly similar.
TWINSCAN annotation can be explored further on the UCSC genome browser (http://genome.ucsc.edu/cgi-bin/hgGateway?db=hg12) and TWINSCAN can be run over the Web at http://genes.cs.wustl.edu. Going beyond accuracy measures, the characteristics of the genes TWINSCAN predicts are a reasonable match to those of the aligned RefSeqs. Not only is the mean number of exons per predicted gene (7.74) in the same range as the RefSeqs (9.07), but the distributions are also reasonably well matched (Fig. 6). Perhaps surprisingly, the distribution for the aligned RefSeqs dips markedly at two and three coding exons before rising again at four. Aside from this dip, the two distributions are similar, with the mode at one exon and a steady decline as the number of exons increases. Beyond six exons, the curves tend to converge, except that TWINSCAN predicts fewer transcripts with an extremely large number of exons. This may be attributable to the fact that TWINSCAN was run on 1-Mb segments of the genome, which will have a greater tendency to split genes with very long genomic extent.
Variation in TWINSCAN Performance With GC Percentage TWINSCAN uses different parameter sets depending on the GC percentage of the N-masked input sequence. The four divisions are 0%–43%, 43%–51%, 51%–57%, and 57%–100%. Most 1-Mb segments of the human genome fall into the two lowest divisions (73.4% and 21.4%, respectively). The fraction of TWINSCAN-predicted exons at each GC level is a good match to the fraction of RefSeq exons at that level (ratios vary from 0.94 to 1.05; Fig. 7). TWINSCAN, however, tends to predict genes with too few exons in the lowest division, and genes with too many exons in the highest division. This results in worse performance on exact gene prediction in these two divisions, with the greatest impact on whole-genome performance coming from the lowest division (Fig. 8).
We have presented the first detailed, published description of an annotation of the entire human genome using alignments from the draft sequence of the mouse genome. Our analysis shows that the mouse genome can be used to improve de novo gene-structure prediction on the human genome. The state of the art in exact human gene prediction remains poor in absolute terms, reflecting the inherent difficulty of the problem. The small fraction of RefSeqs that are predicted perfectly may also reflect the limited representation of human transcripts in RefSeq and occasional errors in the RefSeqs themselves and/or their alignment (J. Kent, pers. comm.) that escaped our automated screen (see Methods). Nonetheless, TWINSCAN represents a tremendous relative improvement in exact gene prediction. The progress it represents is real, reproducible under various conditions, and significant in biological applications (Korf et al. 2001 ; Guigó et al. 2002; Toyoda et al. 2002).
Experimental verification by RT-PCR and direct product sequencing has
shown that comparative gene prediction methods, including TWINSCAN, can
identify genes not found by Ensembl (a transcript-based automated
annotation method). Furthermore, these genes tend to have relatively
tissue-restricted expression patterns (Guigó et al. 2002), validating
the concept of genome comparison as a method to identify genes that are
under-represented in cDNA libraries. One of the great strengths of the TWINSCAN approach is its ability to improve gene-structure prediction using alignments of low-redundancy whole-genome shotgun reads. This is possible because TWINSCAN uses local alignments and makes no assumption that aligned regions are orthologous or that they have conserved exonic structure. We found that unassembled 4X mouse shotgun reads are almost as useful for TWINSCAN as the full draft assembly. TWINSCAN's ability to exploit low-redundancy shotgun reads reflects the fact that BLASTN (as we have run it) aligns truly related (though not necessarily orthologous) sequences quite specifically. Although the long, contiguous sequences of the mouse assembly can, in theory, be aligned to their true human orthologs more reliably than individual reads, this provides limited practical benefit for TWINSCAN. Contrary to our expectations, aligning regions of conserved synteny in the same way that we aligned the entire genomes gave slightly less accurate predictions. We believe this is due, in part, to the fact that TWINSCAN can assign negative scores to gene features that have only low-quality alignments from outside the regions of conserved synteny, helping to avoid false positive predictions. Therefore, the elimination of low-quality alignments from outside regions of conserved synteny can, apparently, hurt performance. Additionally, TWINSCAN assigns positive scores to gene features that have high-quality alignments from paralogous genes outside regions of conserved synteny. This sometimes helps identify real genes that would otherwise have been missed. These observations apply, however, only to alignments performed with the same database preparation and parameter set that we found most practical for whole-genome alignment. If we omit masking of low-complexity sequence in the syntenic mouse region, we obtain accuracy similar to that obtained from the entire mouse assembly. Therefore, aligning syntenic regions using more sensitive procedures than we can apply to whole genomes is likely to yield some accuracy improvement. Direct comparison of systems for annotating the whole human genome is not a simple matter. Complications arise from the lack of an independent gold standard for evaluation, regular changes to the sequence, active development of the annotation systems, sensitivity of results to details of repeat masking and segmenting the sequence, and the huge computational resources required for running many systems. To evaluate TWINSCAN, we compared its predictions with genome annotations derived from RefSeq, a manually curated collection of mRNA sequences provided by NCBI. Using RefSeq as a standard, however, requires that the mRNA sequences be computationally unspliced and mapped back to the genome. This process is subject to error because of the substantial portion of the human genome that is currently unfinished, the inherent uncertainty of unsplicing, and occasional errors in RefSeq. Nonetheless, we believe that aligned RefSeqs are the best currently available standard for evaluation of whole-genome annotation systems. We were able to compare the performance of TWINSCAN to that of GENSCAN because (1) neither program uses RefSeq as an input; (2) GENSCAN is run in a well-thought-out way on every new release of the human genome; and (3) GENSCAN is no longer under development.
Other systems we would have liked to compare TWINSCAN with either have
not been run on recent versions of the human genome or use known
transcripts as an input. For example, we are not aware of any
whole-genome annotations using either CEM (Bafna and Huson 2000 We did a three-way comparison of TWINSCAN, Ensembl, and RefSeq to describe our predictions further, with no implications for relative accuracy outside the RefSeq set. We expected that Ensembl would get nearly all RefSeq exons exactly right. In our evaluation, Ensembl predicted 82% of coding exons in our aligned RefSeq set exactly, whereas TWINSCAN (with no prior knowledge of RefSeq) got 71% exactly right. This surprising observation may arise from the uncertainties in RefSeq alignment mentioned above. When the match criterion is relaxed to 80% mutual overlap, Ensembl overlaps 95% of RefSeq exons, whereas TWINSCAN overlaps 77%. Excluding all exons that overlap RefSeq, TWINSCAN overlaps 65% of Ensembl exons, whereas Ensembl overlaps 43% of TWINSCAN exons. Therefore, the two annotations are in good agreement, but both predict exons not predicted by the other. For annotation consumers, we recommend looking at a series of annotation sets. Despite the uncertainties involved, the aligned RefSeqs represent the most conservative and reliable whole-genome annotation available. However, they include only about half of all human genes, based on current estimates. Ensembl provides a more comprehensive, although still conservative annotation, with a gene count equivalent to about four-fifths of the estimated total. TWINSCAN is more comprehensive still, insofar as its gene count is in the range of the estimated total, and its sensitivity (extrapolated from RefSeq) is fairly high. Beyond TWINSCAN, informal analyses suggest that SGP2 may be slightly more sensitive and less specific. Finally, with a gene count in excess of current estimates, the NCBI annotation may be still more sensitive and less specific. Based on the analysis presented here, we do not see any reason for most consumers to consult GENSCAN or other single-genome, de novo methods for mammalian gene prediction.
In this study, we found that significant gains in the accuracy of human
gene prediction can be obtained using low- to moderate-redundancy
whole-genome shotgun sequence from the mouse. Our experience with other
organisms suggests that low-redundancy sequence is even more useful
when the genomes are more closely related than human and mouse. For
example, in preliminary experiments TWINSCAN has shown substantial
performance benefits for gene prediction in Arabidopsis
thaliana using less than 1X shotgun coverage of Brassica
oleracea (estimated divergence 20 Mya) and in Cryptococcus
neoformans strain JEC21 using about 1.5X coverage of strain H99.
These observations may be influenced by several other variables,
including absolute and relative genome size, but we believe that
evolutionary distance is the most significant variable. Therefore, a
possible cost-effective route to further improvement in human
gene-structure prediction would be to obtain
The longer contiguous sequences that can be obtained from deeper
coverage and assembly are likely to be valuable for identification of
conserved intron–exon structure. TWINSCAN does not make explicit use
of conserved intron–exon structure, in part because it uses purely
local alignments. Nonetheless, we have shown in other work that this
structure does have value for discriminating between gene predictions
that are experimentally verifiable and those that are not (Guigó et
al. 2002
Sequences All predictions were made on NCBI Build 30 (June 2002 data freeze) of the human genome sequence (http://genome.ucsc.edu/goldenPath/28jun2002/chromosomes/chr*.fa.gz). Chromosome Y was excluded from TWINSCAN annotation because the draft sequence of the mouse genome does not include chromosome Y. The sequences were divided into nonoverlapping 1-Mb segments for both the BLAST and TWINSCAN portions of the analysis. Mouse genomic sequences were used as the informant database. We downloaded the February 1, 2002, data freeze of the mouse trace database from http://ftp.ncbi.nih.gov/pub/TraceDB/mus_musculus/ClipReads/FINAL/SEQ/Mm.WGS*.fa.dr.mfa.gz and constructed 1X BLAST databases as described below. The downloaded reads had been RepeatMasked, but not quality clipped. We downloaded the MGSC v3 assembly of the mouse genome from http://genome.ucsc.edu/goldenPath/mmFeb2002/chromosomes/chr*.fa.gz.
Fold Coverage Calculations
Synteny Map
BLAST
Sequence Annotation
Conditional Uncertainty Calculation
). We used maximum likelihood estimates of these probabilities
based on the RefSeq annotation of the human genome and conservation
sequence generated from mouse alignments. Conditional uncertainty given
the aligned conservation symbol and the previous five symbols,
corresponding to the 5th-order Markov chain used in TWINSCAN, gives
lower absolute numbers but the same trends as the 0th-order calculation
reported above.
TWINSCAN Nonprofit institutions may obtain TWINSCAN executables and source code from the authors at no cost. Contact twinscan{at}cse.wustl.edu for more information or visit http://genes.cs.wustl.edu. For-profit institutions may obtain TWINSCAN through Washington University's Center for Technology Management.
Comparison of Annotation Sets
http://www.acedb.org/; ACEDB. http://blast.wustl.edu; Gish, W., WU BLAST Archives. http://ftp.genome.Washington.edu/RM/RepeatMasker.html; Smit, A.F.A. and Green, P., RepeatMasker. http://genes.cs.wustl.edu; TWINSCAN. http://genes.cs.wustl.edu/eval/; Eval software package for comparison of annotations. http://genome.ucsc.edu/goldenPath/28jun2002/chromosomes/chr*.fa.gz; NCBI30 Human Sequence (8–1–02). http://genome.ucsc.edu/goldenPath/mmFeb2002/chromosomes/chr*.fa.gz; MGSCv3 Assembled Mouse Sequence (4–24–02). http://genome.ucsc.edu/goldenPath/28jun2002/database/refGene.txt.gz; RefSeqs (9–13–02). http://genome.ucsc.edu/goldenPath/28jun2002/database/genscan.txt.gz; GENSCAN annotation (8-20-02). http://ftp.ncbi.nih.gov/pub/TraceDB/mus_musculus/ClipReads/FINAL/SEQ/Mm.WGS*.fa.dr.mfa.gz; Mouse Trace Files (Feb. 1, 2002 freeze). http://www.ensembl.org/Homo_sapiens/; Ensembl human annotation (Release 8.30a.1). http://www.genome.wi.mit.edu/mouse/synteny/index.html; Mouse–human synteny map. http://www.ncbi.nlm.nih.gov/genome/guide/human/HsStats.html; Humane Genome: Current Statistics—Build 30. http://genome.ucsc.edu/cgi-bin/hgGateway?db=hg12; UCSC Genome Browser, Human Build 30.
We are grateful to the Mouse Genome Sequencing Consortium for sequencing the mouse genome and making it publicly available; and to the Mouse Genome Analysis group, especially Ewan Birney, Roderic Guigó, Manolis Dermitzakis, Jim Kent, and Webb Miller, for helpful discussions. We thank Mark Bober for maintaining our compute cluster; Deanna Church for making the mouse reads available on short notice; Michael Kamal and the Whitehead Institute for providing the global synteny map; and Min Wang for the conditional uncertainty calculations. P.F. is supported by a Whitaker Foundation Graduate Fellowship and I.K. is supported by a grant from the National Human Genome Research Institute (K22 HG-0064-01). The remainder of this work was supported by grants HG02278-01A2 from the National Human Genome Research Institute and DBI-0091270 from the National Science Foundation to M.R.B. The publication costs of this article were defrayed in part by payment of page charges. This article must therefore be hereby marked "advertisement" in accordance with 18 USC section 1734 solely to indicate this fact.
3 Present address: The Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, UK.  E-MAIL brent{at}cse.wustl.edu; FAX (314) 935-7302. Article and publication are at http://www.genome.org/cgi/doi/10.1101/gr.830003.
 locus. Mol. Phylogenet. Evol. 5: 33-49.[CrossRef][Medline]
Received September 24, 2002; accepted in revised format October 30, 2002. 13:46-54 © by 2003 Cold Spring Harbor Laboratory Press ISSN 1088-9051/03 $5.00  CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati What's this?
This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
ABSTRACT
RESULTS
