|
|
|
|
Genome Res. 14:2469-2477, 2004 ©2004 by Cold Spring Harbor Laboratory Press; ISSN 1088-9051/04 $5.00
Letter Computing prokaryotic gene ubiquity: Rescuing the core from extinctionGenome Atlantic, and Department of Biochemistry and Molecular Biology, Dalhousie University, Halifax, Nova Scotia, Canada B3H 1X5
The genomic core concept has found several uses in comparative and evolutionary genomics. Defined as the set of all genes common to (ubiquitous among) all genomes in a phylogenetically coherent group, core size decreases as the number and phylogenetic diversity of the relevant group increases. Here, we focus on methods for defining the size and composition of the core of all genes shared by sequenced genomes of prokaryotes (Bacteria and Archaea). There are few (almost certainly less than 50) genes shared by all of the 147 genomes compared, surely insufficient to conduct all essential functions. Sequencing and annotation errors are responsible for the apparent absence of some genes, while very limited but genuine disappearances (from just one or a few genomes) can account for several others. Core size will continue to decrease as more genome sequences appear, unless the requirement for ubiquity is relaxed. Such relaxation seems consistent with any reasonable biological purpose for seeking a core, but it renders the problem of definition more problematic. We propose an alternative approach (the phylogenetically balanced core), which preserves some of the biological utility of the core concept. Cores, however delimited, preferentially contain informational rather than operational genes; we present a new hypothesis for why this might be so.
The concept of a genomic core plays a key role in the literature of evolutionary and comparative prokaryotic genomics (Makarova et al. 1999
In prokaryotic species for which genomes of several different strains have been completely sequenced, orthologous genes of the species core can usually be identified easily; they are conserved in chromosomal position as well as in sequence. For deeper and more inclusive taxa, cores become progressively smaller and more elusive because of weak phylogenetic signals, genomic rearrangements, and problems in recognizing paralogy. Nevertheless, there is much interest in such deep cores, especially the universal (Bacteria + Archaea + Eukarya) core. Several investigators argue that this core's composition might reflect that of the genome of the Last Universal Common Ancestor (LUCA), while phylogenies of its genes, if congruent, could delineate the earliest branchings of the Tree of Life (Brown et al. 2001
Recent attempts to define the universal core have concluded that it contains very few genes. Harris et al. (2003 Here, we describe a similarly motivated study, addressing a more extensive set of prokaryotic genomes (130 Bacteria and 17 Archaea) with a greater variety of methods of analysis. We, too, find a diminutive set of truly ubiquitous genes among prokaryotes. Our interest, however, is not so much in this number as in whether or not it might be a statistical illusion, what more useful algorithms for defining cores might be possible, and the difficult question of biological significance of core size and composition.
Ubiquitous cores defined by reciprocal best matches Table 1 summarizes our enumeration of genes shared by all members of selected prokaryotic taxa. Reported values are the means of all queries in which a search was launched with any one genome from a member of the taxon against all others from that taxon, requiring a reciprocal best match (RBM, see Methods) in each. Variation around these means is small, and reflects cases in which genes are variously disconnected from one another in the BLAST method. (By disconnection, we mean a situation in which an occasional gene X will be an above-threshold best-reciprocal match between genomes A and B and A and C, but not between B and C). The average prokaryotic genomic core defined in this way (genes shared by a launching genome and all other Bacteria and Archaea) has only 14.82 such ubiquitous genes. A more generous result, and one which actually produces a unique list of gene names, is obtained by taking the union of all such sets shared by all 147 genomes. The size of this union of RBM sets is 30 genes.
It seems unlikely that this number is falsely large. Artifacts that could make it so would require extreme degrees of sequence convergence, and in any case, all or most of the genes identified in such an analysis are expected to be highly conserved on biological grounds (see below; Table 3, below). It could easily be falsely small, however, and the remaining analyses presented here variously address possible artifactual explanations for the diminutive size of the prokaryotic core.
Effects of BLAST parameters and genome size on apparent core size One such possibility is that we have set cut-off values for BLASTP too stringently, so that legitimate orthologs have been overlooked. If this were so, the average number of genes with RBMs in all genomes should be exquisitely sensitive to that cut-off value. It is not, as Table 2 illustrates. There is remarkably little variation in average numbers of genes obtained for expectation values between 1.0e-3 and 1.0e-7. Nevertheless, a few more genes were added by combining the results of the union of shared RBMs at 1.0e-5 with the results of the consensus gene name, or CGN approach (see Methods), which might recover some cases of poorly or nonreciprocally matching orthologs. Both sets are listed in Table 3, and are largely overlapping (30 genes from the union of shared RBMs, 34 genes from CGNs, 26 in their intersection and 38 in their union).
Another possible artifact (or misleading bias) could result from inclusion of the highly reduced genomes of endosymbionts or parasites, which can lack many genes required by free-living cells. To examine the effect of excluding small genomes, we performed the analysis shown in Figure 1A, using the CGN method. To limit errors due to inconsistent annotations of rare genes, we excluded any gene missing from more than 50 of the 147 genomes; this left 474 genes for analysis. The lower curve in Figure 1A plots the mean number of such genes found shared in 10,000 randomized comparisons using the indicated number of different genomes, selected randomly (without replacement) from the total set of 147. This method of representation mimics the general historical course of research in the definition of the universal coreeach new survey involving more genomes has reduced the apparent number of genes, and some asymptotic low value less than 100 appears to be on the horizon. With our methods, restricting the comparisons to genomes of more than 1000 ORFs, 1500 ORFs, and so on (upper curves) raises the apparent asymptote, but the effect is not dramatic. Even with genomes of more than 2000 ORFs, we expect fewer than 75 ubiquitous genes.
Figure 1B, which restricts comparisons to genomes in a single clade, shows additional, presumably biological, effects. At a given sample size, there are more genes shared between genomes when these are taken from the Proteobacteria than from the Bacteria generally, or prokaryotes more generally still. The proteobacterial core defined in this way is indeed larger than (and includes) the bacterial core. Archaea appear to comprise a generally less-coherent group. In part, this could be a genome size effect; the average sequenced archaeal genome is smaller than the average sequenced bacterial genome. The diminutive Nanoarchaeum equitans genome (only 563 ORFs and a seriously impoverished metabolic repertoire) may in particular exert an effect (Waters et al. 2003
Core composition and the problem of missing genes
As it happens, the distribution and conservation of ribosomal proteins has recently been subjected to a careful analysis by Lecompte et al. (2002
Some further losses clearly have occurred among several ribosomal protein genes described by Lecompte et al. (2002 Our case-by-case searches described above revealed that the true ubiquitous core consists of 29 prokaryotic ribosomal protein genes, but our strictest measurement of the core using an automated analysis identified only 11 ribosomal protein genes (Table 3). Many of the false negatives in the automated search, which is necessarily based on annotated genes, were due to missed annotations (especially of small proteins not designated as ORFs in genome databases) and misannotations (usually where a larger overlapping ORF was selected at the expense of a smaller gene). Some of the false negatives were, however, attributable to deficiencies in our methods, where BLASTP thresholds were too strict or where alternate annotations confounded assignment of the CGN. Ribosomal proteins might, however, be worst-case examples both for annotation artifacts and algorithmic shortcomings; many ribosomal proteins are very small, and can easily be overlooked by annotators and by BLASTP alike. Furthermore, some of the apparently genuine absences affect very few genomes; four more genes could be added to the core if the ubiquity requirement were relaxed, even very slightly. It will not be simple to extract from our analysis of ribosomal proteins any reliable estimate of how many genes in other functional categories have been excluded from Table 3 because of error, and how many are genuinely missing from at least one or a few genomes. Our focus, in any case, is not so much on the precise number or identity of core genes as on the methods best used to define them. In that regard, we infer that the requirement for ubiquity in defining genomic coresas well as being completely unforgiving with respect to errorsmight be standing in the way of our recognition of some biologically more significant collection of almost ubiquitous genes.
Relaxing the requirement for ubiquity
A comparison of Figures 2 and 1 also illustrates a potential misunderstanding in the graphical representation of such analyses. Both achieve a terminal value of 34 for the number of consensus gene names found in all 147 of the genomes examined in this study. But at, say, 125 genomes, the number of shared genes has increased by only a few in Figure 1A and to more than 200 in Figure 2. This is because Figure 1A asks, for all possible samplings of 125 of the 147 genomes, what is the average number of genes commonly ubiquitous in these 125, while Figure 2 asks how many genes are independently ubiquitous in any sampling of 125 genomes. Thus, the second analysis differs from the first in that genes found in at least 125 genomes do not have to be found in the same 125 genomes (and manifestly are not). Figure 1A, we think, mimics the historical course of studies of the core. These have usually asked how many genes are present in all genomes sequenced at the time of analysis. Such studies quite quickly settled down to reporting fewer than 100 genes, an estimate that seems to be holding up well (Brown et al. 2001
It is appealing to propose a model in which each gene has a different and independent characteristic probability of going missing from a genome (Krylov et al. 2003
Toward a biologically more significant phylogenetically balanced core
This approach has at least three distinct advantages over ubiquity-requiring global analyses. First, for the mean values of multiphylum comparisons, genes that are missing (either truly or through error) from only a few genomes, will usually have less effect. Second, and for related reasons, highly reduced genomes have less impact on the size of the coreunless they are the only representatives of their phyla. The maximum and minimum values show the extent to which large and small genomes (endosymbiotic or parasitic) within well-sampled phyla influence our comparisons. Third, as more genomes are added within existing phyla, this estimate will become more precise (its standard error will decrease). Its value is not expected to diminish and will likely even increase, as larger genomes from phyla, so far poorly sampled, appear. The addition of new phyla will diminish the PBC core, but the list of such completely unsampled phyla is not, like that of unsequenced genomes, limitless. Our ultimate interest here is in such a core for all prokaryotes, defined as genes present in some reasonable fraction of genomes in each and every one of 12 bacterial phyla and two archaeal phyla (Crenarchaeota and Euryarchaeota, including Nanoarchaeum equitans). In Table 5, we list all genes that are (1) present by consensus name in at least one genome in each of these 14 groups, and (2) within these groups, present in 100%, at least 90%, or at least 80% of genomes. Of course, the 100% gene set is identical to that shown in Table 3, except for minor differences arising from Table 5's more comprehensive data set; there is no phylogenetic balancing when ubiquity is required. (For constructing this table, we have used, in addition to the 147 prokaryotic genomes available in January 2004, 23 that have since appeared.) Interestingly, rpoB and rpoC, which are necessary and ubiquitous components of RNA polymerase, are only retrieved using our relaxed definition of the core. In some genomes, these genes are fused, and thus thwart the retrieval of rpoB and/or rpoC by RBM or CGN. This example helps to underline the need for flexibility in defining core genes, not only in compensation for sequencing and annotation artifacts, and the odd rarely lost gene, but also for some genes' tendency to form multidomain proteins.
Gene loss is a known factor in genome evolution, but since ancestral genomes cannot have been much larger than present-day genomes, gene genesis (largely by duplication and divergence) is necessary in order to compensate (Snel et al. 2002 The most inclusive core defined in this way contains 71 genes (Table 5). We suspect that its size may increase as more or larger genomes for some of the sparsely sampled phyla especially Aquificales, Chlorobi, Planctomycetes, and Thermotogalesbecome available. Although the potentially distorting effect of individually aberrant genomes that are among the few representatives of their respective phyla cannot be ignored, its importance will diminish as more genome sequences appear (Two examples are as follows: ffh appears to be absent from Nanoarchaeum equitans [acceptable to PBC], but also from both strains of Leptospira interrogans and thus, from 40% of our Spirochaetes; ftsY appears to be absent from N. equitans [acceptable], but also from Sulfolobus tokodaii and thus, from 25% of our Crenarchaeota). The decision to choose genes present in 90%, 80%, or some other fraction of genomes in each phylum remains arbitrary. But, the requirement imposed by the PBC approach for presence at such a level in all phyla guarantees a more representative biological sampling, whereas a core defined as including genes present in 80% of all genomes regardless of phyletic distribution would be a bacterial core.
Although universal prokaryotic cores as described here and in other recent literature are often suggestively similar in size, this apparent convergence lacks biological significance. When each core gene is required to be in every genome, cores will inevitably be artifactually small. Some genes will be missing because of sequencing, assembly, and annotation errors, and some genuine orthologs will have diverged beyond detectability. Because of such errors alone, the size of ubiquity-requiring genomic cores should continue to decline slowly, as more genome sequences appear. The impact of errors like this might be reduced by relaxing the requirement for ubiquity (to <100% of all genomes). However, the set of almost ubiquitous genes rises almost continuously in number as percent representation is relaxed, and includes more and more genes whose nonubiquity is not artifactual. There is no obvious place to draw the line, other than that at which we can discount all Archaea (about 80%). To do this would be to abandon the claim for (prokaryotic) universality. Defining cores as genes present in some fraction of genomes less than 100% is thus not only arbitrary, but gives disproportionate weight to taxa favored, for whatever reason, by sequencers. The PBC approach we describe will address the problem of errors and the problem of disproportionate weighting of popular phyla when ubiquity is not required. It also favors universality (within a phylogenetic context), but it still does not tell us where to draw the line. We do find it encouraging that the core defined in this way increases less than 20% in size, when prevalence required within taxa is dropped from 90% to 80%. If, indeed, there is a conserved set of genes that might be considered the basic heritage for all prokaryotes, but which defies precise definition because of occasional loss or orthologous replacement in scattered lineages, the PBC might provide a good approximation.
Although a nonarbitrary delimitation of core size may be impossible, core composition is not random. Cores are generally dominated by genes of the translational apparatus. Why should this be? The widely accepted explanation is that genes of this important informational process are intrinsically less exchangeable than genes of operational processes, like metabolism (Woese 1987
But there is an alternative explanation, which we will sketch out here, and have represented by a cartoon in Figure 4. All cells need genes of each of several functional categories, such as (put very crudely) translation, metabolism, and cell envelope formation. Lake et al. (1999
Exchanges of this nature need not be cryptic in phylogenetic analyses. Informational genes of the universal prokaryotic core should not produce congruent phylogenetic trees if cells have exchanged them for foreign (but still orthologous) versions as often as they seem to have traded nonorthologous (indeed, nonhomologous) operational genes. Whether or not genes of the universal core do show congruent phylogenies is, however, still a matter of legitimate debate; at this (phylum) depth few individual genes have reliable phylogenetic signal. Several years ago, Teichmann and Mitchison (1999 Thus, objections to our alternative explanation (Fig. 4) must rest on one or both of two lines of argument as follows: (1) that genes for such components are intrinsically less transferable for some reason related to their function (the complexity hypothesis) or (2) that the general support for the three-domain Tree of Life among informational genes is best explained by their relative nonexchangeability. The first is as yet unproven, and the presence among ubiquitous genes of some presumably noncomplexing proteins outside of the informational class argues against it, and the second isgiven that domains have themselves been defined primarily with reference to their translational componentsdangerously circular.
Determining the size and composition of a core of genes ubiquitous among a set of genomes requires a method with which to compute orthologous relationships in bulk. For practical reasons, we can assume that a pair of reciprocally best (or near-best) matching genes are likely to be orthologs, and can thus automate the process of comparative genomic analysis (Charlebois et al. 2003
Using BLASTP scores, we can distinguish between ordinary matches (including many which may be paralogous) and reciprocal best matches (RBMs), which are more often orthologous. Additionally, we can make use of information present in genomic annotations, permitting genes with standardized gene names to find their orthologs despite BLASTP matches that may fall slightly below threshold. Where annotations are reasonably consistent, this permits the union of overlapping sets of genes where BLASTP matches alone might miss members outside of the sets' intersection. Matches are still based on BLASTP, but some-what disconnected outliers can then link through the bridge of a common consensus gene name (CGN). These are computed as follows: For each ORF in a genome, its RBM, if any, is found in each of the other genomes, and the RBM's annotated name is appended to a list. The dominant name in this list (e.g., ftsA) becomes the query ORF's CGN. Lists of prevalent names, found in most or all members of a set of genomes, are generated by the "List named ubiquitous genes" query within NGIBWS (http://www.neurogadgets.com/bws.php
A strict definition of a core of genes has those genes present in every member of the set of genomes under consideration. We performed such an analysis on clades of genomes (defined according to a genomic phylogeny [Gophna et al. 2004
Both RBM and CGN approaches present some limitations. False negatives can arise in the RBM approach when BLASTP matches fall below threshold; false positives can arise when match thresholds are set so low as to permit spurious matches, and when paralogs match for want of lost orthologs. The use of CGNs as bridges for weak matches overcomes some of the problems inherent in the pure RBM approach, but introduces new problems relating to inconsistent annotation. Where alternative names for a gene are popular, an orthologous cluster may break up into name cliques, where a gene is truly ubiquitous, but fails to appear as a ubiquitous CGN. False positives are theoretically possible with the CGN approach, but only if homonymous names are used in annotation, which should be rare among core or otherwise prevalent genes, lysyl-tRNA synthetase (lysS) not-withstanding (Ibba et al. 1999
Both RBM and CGN methods of finding ubiquitous genes are exquisitely sensitive to missed annotation (where a gene is present in the sequence, but is not annotated as an ORF), and to sequencing artifacts (during cloning or sequence assembly). We assessed the extent of the former problem by repeating the work of Lecompte et al. (2002
This work was supported by Genome Atlantic (Genome Canada) and the Canadian Institutes for Health Research, and the Canada Research Chairs Program.
Article and publication are at http://www.genome.org/cgi/doi/10.1101/gr.3024704.
1 Corresponding author.
Altschul, S.F., Madden, T.L., Schaffer, A.A., Zhang, J., Zhang, Z., Miller, W., and Lipman, D.J. 1997. Gapped BLAST and PSI-BLAST: A new generation of protein database search programs. Nucleic Acids Res. 25: 3389-3402. Boucher, Y., Douady, C.J., Papke, R.T., Walsh, D.A., Boudreau, M.E., Nesbø, C.L., Case, R.J., and Doolittle, W.F. 2003. Lateral gene transfer and the origins of prokaryotic groups. Annu. Rev. Genet. 37: 283-328.[CrossRef][Medline] Brochier, C., Bapteste, E., Moreira, D., and Philippe, H. 2002. Eubacterial phylogeny based on translational apparatus proteins. Trends Genet. 18: 1-5.[CrossRef][Medline] Brown, J.R., Douady, C.J., Italia, M.J., Marshall, W.E., and Stanhope, M.J. 2001. Universal trees based on large combined protein sequence data sets. Nat. Genet. 28: 281-285.[CrossRef][Medline] Charlebois, R.L., Clarke, G.D.P., Beiko, R.G., and St. Jean, A. 2003. Characterization of species-specific genes using a flexible, web-based querying system. FEMS Microbiol. Lett. 225: 213-220.[CrossRef][Medline] Charlebois, R.L., Beiko, R.G., and Ragan, M.A. 2004. Genome phylogenies. In Organelles, genomes and eukaryote phylogeny: An evolutionary synthesis in the age of genomics (eds. R.P. Hirt and D.S. Horne), pp. 189-206. CRC Press, Boca Raton, FL.
Clarke, G.D.P., Beiko, R.G., Ragan, M.A., and Charlebois, R.L. 2002. Inferring genome trees using a filter to eliminate phylogenetically discordant sequences and a distance matrix based on normalized BLASTP scores. J. Bacteriol. 184: 2072-2080.
Daubin, V., Gouy, M., and Perrière, G. 2002. A phylogenomic approach to bacterial phylogeny: Evidence of a core of genes sharing a common history. Genome Res. 12: 1080-1090.
Daubin, V., Moran, N.A., and Ochman, H. 2003. Phylogenetics and the cohesion of bacterial genomes. Science 301: 829-832. Gophna, U., Doolittle, W.F., and Charlebois, R.L. 2004. Weighted genome trees: Refinements and applications. J. Bacteriol. (in press).
Harris, J.K., Kelley, S.T., Spiegelman, G.B., and Pace, N.R. 2003. The genetic core of the universal ancestor. Genome Res. 13: 407-412.
Ibba, M., Losey, H.C., Kawarabayasi, Y., Kikuchi, H., Bunjun, S., and Söll, D. 1999. Substrate recognition by class I lysyl-tRNA synthetases: A molecular basis for gene displacement. Proc. Natl. Acad. Sci. 96: 418-423. Jain, R., Rivera, M.C., Moore, J.E., and Lake, J.A. 2002. Horizontal gene transfer in microbial genome evolution. Theor. Popul. Biol. 61: 489-495.[CrossRef][Medline] Koonin, E.V. 2003. Comparative genomics, minimal gene-sets and the last universal common ancestor. Nat. Rev. Microbiol. 1: 127-136.[CrossRef][Medline] Koonin, E.V., Mushegian, A.R., and Bork, P. 1996. Non-orthologous gene displacement. Trends Genet. 12: 334-336.[Medline]
Kunin, V. and Ouzounis, C.A. 2003. The balance of driving forces during genome evolution in prokaryotes. Genome Res. 13: 1589-1594.
Krylov, D.M., Wolf, Y.I., Rogozin, I.B., and Koonin, E.V. 2003. Gene loss, protein sequence divergence, gene dispensability, expression level, and interactivity are correlated in eukaryotic evolution. Genome Res. 13: 2229-2235.
Lake, J.A., Jain, R., and Rivera, M.C. 1999. Mix and match in the tree of life. Science 283: 2027-2028.
Lecompte, O., Ripp, R., Thierry, J.C., Moras, D., and Poch, O. 2002. Comparative analysis of ribosomal proteins in complete genomes: An example of reductive evolution at the domain scale. Nucleic Acids Res. 30: 5382-5390.
Lerat, E., Daubin, V., and Moran, N.A. 2003. From gene trees to organismal phylogeny in prokaryotes: The case of the Makarova, K.S. and Koonin, E.V. 2003. Comparative genomics of archaea: How much have we learned in six years, and what's next? Genome Biol. 4: 115.[CrossRef][Medline]
Makarova, K.S., Aravind, L., Galperin, M.Y., Grishin, N.V., Tatusov, R.L., Wolf, Y.I., and Koonin, E.V. 1999. Comparative genomics of the Archaea (Euryarchaeota): Evolution of conserved protein families, the stable core, and the variable shell. Genome Res. 9: 608-628.
Matte-Tailliez, O., Brochier, C., Forterre, P., and Phillippe, H. 2002. Archaeal phylogeny based on ribosomal proteins. Mol. Biol. Evol. 19: 631-639. Mirkin, B.G., Fenner, T.I., Galperin, M.Y., and Koonin, E.V. 2003. Algorithms for computing parsimonious evolutionary scenarios for genome evolution, the last universal common ancestor and dominance of horizontal gene transfer in the evolution of prokaryotes. BMC Evol. Biol. 3: 2.[CrossRef][Medline]
Mushegian, A.R. and Koonin, E.V. 1996. A minimal gene set for cellular life derived by comparison of complete bacterial genomes. Proc. Natl. Acad. Sci. 93: 10268-10273. Nesbø, C.L., Boucher, Y., and Doolittle, W.F. 2001. Defining the core of nontransferable prokaryotic genes: The euryarchaeal core. J. Mol. Evol. 53: 340-350.[CrossRef][Medline] Ragan, M.A. and Charlebois, R.L. 2002. Distributional profiles of homologous open reading frames among bacterial phyla: Implications for vertical and lateral transmission. Int. J. Syst. Evol. Microbiol. 52: 777-787.[Abstract]
Snel, B., Bork, P., and Huynen, M.A. 2002. Genomes in flux: The evolution of archaeal and proteobacterial gene content. Genome Res. 12: 17-25.
Tatusov, R.L., Koonin, E.V., and Lipman, D.J. 1997. A genomic perspective on protein families. Science 278: 631-637. Teichmann, S.A. and Mitchison, G. 1999. Is there a phylogenetic signal in prokaryote proteins? J. Mol. Evol. 49: 98-107.[CrossRef][Medline]
Waters, E., Hohn, M.J., Ahel, I., Graham, D.E., Adams, M.D., Barnstead, M., Beeson, K.Y., Bibbs, L., Bolanos, R., Keller, M., et al. 2003. The genome of Nanoarchaeum equitans: Insights into early archaeal evolution and derived parasitism. Proc. Natl. Acad. Sci. 100: 12984-12988.
Woese, C. 1998. The universal ancestor. Proc. Natl. Acad. Sci. 95: 6854-6859.
Woese, C.R. 1987. Bacterial evolution. Microbiol. Rev. 51: 221-271.
. 2000. Interpreting the universal phylogenetic tree. Proc. Natl. Acad. Sci. 97: 8392-8396.
. 2002. On the evolution of cells. Proc. Natl. Acad. Sci. 99: 8742-8747.
Zimmer, C. 2003. Genomics. Tinker, tailor: Can Venter stitch together a genome from scratch? Science 299: 1006-1007.
http://www.neurogadgets.com/bws.php; The NeuroGadgets Inc. Bioinformatics Web Service (NGIBWS).
Received July 20, 2004; accepted in revised format October 7, 2004. This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||