|
|
|
|
Published online before print
May 15, 2006, 10.1101/gr.5011206 Genome Res. 16:787-795, 2006 ©2006 by Cold Spring Harbor Laboratory Press; ISSN 1088-9051/06 $5.00 OPEN ACCESS ARTICLE
Methods High-density haplotyping with microarray-based expression and single feature polymorphism markers in Arabidopsis1 Department of Plant Sciences 2 The Genome Center, University of CaliforniaDavis, Davis, California 95616-8780, USA; 3 Department of Statistics, Purdue University, West Lafayette, Indiana 47907-2067, USA
Expression microarrays hybridized with RNA can simultaneously provide both phenotypic (gene expression) and genotypic (marker) data. We developed two types of genetic markers from Affymetrix GeneChip expression data to generate detailed haplotypes for 148 recombinant inbred lines (RILs) derived from Arabidopsis thaliana accessions Bayreuth and Shahdara. Gene expression markers (GEMs) are based on differences in transcript levels that exhibit bimodal distributions in segregating progeny, while single feature polymorphism (SFP) markers rely on differences in hybridization to individual oligonucleotide probes. Unlike SFPs, GEMs can be derived from any type of DNA-based expression microarray. Our method identifies SFPs independent of a genes expression level. Alleles for each GEM and SFP marker were ascertained with GeneChip data from parental accessions as well as RILs; a novel algorithm for allele determination using RIL distributions capitalized on the high level of genetic replication per locus. GEMs and SFP markers provided robust markers in 187 and 968 genes, respectively, which allowed estimation of gene order consistent with that predicted from the Col-0 genomic sequence. Using microarrays on a population to simultaneously measure gene expression variation and obtain genotypic data for a linkage map will facilitate expression QTL analyses without the need for separate genotyping. We have demonstrated that gene expression measurements from microarrays can be leveraged to identify polymorphisms across the genome and can be efficiently developed into genetic markers that are verifiable in a large segregating RIL population. Both marker types also offer opportunities for massively parallel mapping in unsequenced and less studied species.
Comprehensive gene expression microarrays have enabled transcriptome profiling in a wide variety of organisms. When the expression levels of individual genes are considered as quantitative traits, their variation can be described as expression level polymorphisms (ELPs) and their determinants can be mapped with molecular markers as expression QTLs (eQTLs) (Jansen and Nap 2001
Several different types of molecular markers have been developed over the past three decades (Kumar 1999 Gene expression markers (GEMs) can be obtained from expression microarrays based on long oligonucleotide probes, spotted cDNAs, or short oligonucleotides, because they are based on gene expression differences, not on individual probe hybridization. GEMs, characterized by large differences in transcript levels between the parents of a segregating population, provide expression data that are distinctly bimodal in distribution when surveyed in a recombinant inbred line (RIL) population.
We demonstrate that data derived from gene expression microarrays can also be used at little or no extra expense to develop large numbers of robust molecular markers. We used two different approaches (GEMs and SFPs) and two methods to define markers and their alleles. The first method was based on parental polymorphisms detected with replicated microarray data. The second method capitalized on the high level of replication available for each locus in the RIL population to identify robust polymorphisms in the progeny microarray data. We validated these novel markers by using a large segregating A. thaliana RIL population. Sequencing studies have demonstrated the existence of hundreds of SNPs between accessions Bayreuth (Bay-0) and Shahdara (Sha), but these polymorphisms had not been developed into tractable genetic markers (Nordborg et al. 2005
Gene expression markers We conducted two factorial experiments (i.e., biological replication of different treatments and time points) using Bay-0 and Sha plants in order to assess differences in gene expression between these two genotypes (unpublished data). Labeled RNA from the control plants was hybridized to Affymetrix ATH1 GeneChip microarrays, resulting in 16 Bay-0 and 16 Sha parental GeneChips. To identify potential GEMs, we compared gene expression values averaged over the 16 GeneChips per accession and identified 1431 genes that exhibited an empirically chosen twofold or greater expression difference between the two genotypes. Subsequently, to identify genes with non-overlapping expression value distributions, a "gap" value was determined for each gene: The minimum expression value of the higher expressing allele was divided by the maximum expression value of the lower expressing allele. A gap of 1.0 indicates non-overlapping distributions; the larger the gap, the greater the distance between the distributions. Of the 1431 genes, 324 had a gap 1.0 and were used as our candidate GEMs. Genotypes were assigned to 148 Bay-0 x Sha F9 RILs for each of the 324 candidate GEMs by comparing RIL gene expression values from replicated GeneChips with the parental expression level distributions. For each GEM, RILs with expression values that fell within the range of the 16 Bay-0 parental values were assigned a Bay-0 genotype; Sha parental values were likewise used to assign Sha genotypes to the RILs. RILs with gene expression levels that fell between the two parental distributions could not initially be assigned a genotype. To minimize the number of ambiguous "missing data" genotype scores, we incrementally adjusted the parental distributions used for allele assignments by utilizing a slicing procedure (see Methods). Because bimodal distributions can sometimes be caused by complex genetic inheritance, we filtered the GEMs to remove those markers that showed severe segregation distortion (allele segregation ratios > 1:3) or had >10% missing data scores, as these would suggest nonqualitative inheritance of the GEM polymorphism. A total of 188 markers (corresponding to 187 genes) remained after filtering.
These 188 GEMs, plus 38 previously genotyped microsatellite markers (Loudet et al. 2002 Six GEMs did not map to the expected chromosomal locations (Supplemental Fig. 1a). For example, At2g26630 and At2g38520 exhibit anomalous red segments off the diagonal; their expected physical location is in linkage group 2, but they are genetically linked to linkage group 5. Markers At1g29410 and At4g01350 are represented by probe sets that recognize alternate gene family members whose chromosomal locations are consistent with their genetic linkages (Supplemental Fig. 1). Markers At2g26630, At2g38520, and At5g34920 correspond to genes with transposon-related functions. Only marker At3g60980 showed a linkage that cannot be readily explained and may reflect a small translocation relative to Col-0 in either the Bay-0 or Sha accession. At3g60980 also contains a SFP which shows genetic linkage consistent with the GEM marker position for the same gene (see SFP section below). Graphical genotypes (haplotypes) were generated for each of the 148 RILs by using the GEMs and microsatellite markers (Supplemental Fig. 2). The 188 GEMs define haplotypes that are consistent with, and more detailed than, the haplotypes generated by the 38 microsatellite markers alone (http://www.inra.fr/qtlat). Whereas 836 crossovers are apparent when the 38 microsatellite markers are used to genotype these 148 RILs, the haplotypes defined by the 188 GEMs identify 1264 recombination breakpoints.
SFP markers
SFP detection with parental data
The SFPdev distributions were used to assign genotypes to the 148 RILs, using the same RIL GeneChips described previously and a "parental min-max" algorithm (see Methods). After genotyping the RILs, SFP markers were discarded if there was >10% missing data or if a deviation of allele frequency >1:4 was observed. After this filtering process, 1157 SFP markers remained, with an average frequency of 1.23 SFP markers per probe set (Table 1). One SFP marker per probe set with the least missing data was selected, resulting in 943 SFP markers with an average of 4.7% missing data. An analysis of the 148 RIL genotypes clustered these 943 SFP markers plus the 38 microsatellite markers into 479 bins, with an average of 2.05 SFP markers per bin (Table 1).
To investigate how the parental min-max algorithm impacted missing genotype scores, we randomly selected 10 SFP markers with
SFP detection with RIL data To reduce the frequency of missing genotype data, we capitalized on the high level of replication available for each allele in the RIL population. Our "RIL distribution" algorithm (see Methods) searched for bimodal distributions of SFPdev values in the 296 RIL GeneChips and then assigned alleles based on these distributions in the progeny. For the SFP marker illustrated in Figure 1, the RIL distribution algorithm resulted in 0% missing scores, compared with the 24.3% missing scores obtained using the parental min-max algorithm. The RIL distribution algorithm identified 1259 SFPs between Bay-0 and Sha (Table 1). Because of limited segregation distortion and minimal missing scores, the post-genotyping filtering process eliminated only two of these 1259 SFP markers. The 1257 markers represented 968 probe sets (1.30 SFP markers per probe set) (Table 1). One SFP marker was selected per probe set, resulting in a data set with only 0.43% genotype scores missing. Cluster analysis grouped the 968 SFP markers plus the 38 microsatellite markers into 563 bins, with an average of 1.79 markers per bin. If a bin with multiple markers did not contain any markers with 0% missing data, two SFP markers were selected to represent that bin in order to avoid loss of information due to missing genotype scores, resulting in a set of 599 SFP markers. A heat map was generated by using pair-wise recombination values for the 599 SFP markers plus the 38 microsatellite markers (Fig. 2, Supplemental Fig. 5a). The genetic map spanned 409 cM with an average resolution of one marker every 0.64 cM (187 kb) and a maximum gap size of 7.2 cM. Each linkage group was also analyzed separately with JoinMap (Supplemental Fig. 5); the average estimated intervals between markers for each linkage group are as follows: AT1, 0.62 cM; AT2, 0.73 cM; AT3, 0.56 cM; AT4, 0.67 cM; and AT5, 0.66 cM, providing a near-saturated genetic map for this size of population.
The linkage of 595 of the 599 SFP markers was consistent with the positions expected from the genomic sequence of Col-0. In agreement with the GEM results, there are no major rearrangements of the genes containing these SFP markers between Col-0 and Bay-0 and Sha. However, when the order of the SFP markers was determined with JoinMap, small local deviations from the Col-0 gene order were observed (Supplemental Fig. 5), which could be due to small inversions or translocations relative to Col-0, multipoint recombination estimates, or sampling error. The segregation data for four out of 599 SFP markers were inconsistent with the Col-0 sequence (indicated by red segments off the diagonal in Fig. 2, Supplemental Fig. 5a). The genes corresponding to these four markers (At1g10380, At3g60980, At5g49250, and At5g55530) do not belong to multi-gene families, are not associated with transposable elements, and have no known function. Their map locations in this population may be indicative of small translocations relative to the Col-0 sequence. At3g60980 is also represented by a GEM, which likewise shows linkage to the same inconsistent position evidenced by the SFP marker (see Supplemental Fig. 1).
Using of the 599 SFP and 38 microsatellite markers, graphical genotypes were generated for each of the 148 RILs and the parental controls (Fig. 3; for individual linkage groups, see Supplemental Fig. 6). These 637 markers identified a total of 1533 recombination breakpoints, in comparison with the 1264 breakpoints detected with the 188 GEMs plus 38 microsatellite markers. Only 33% of the genes identified as GEMs overlapped with the set of genes for which we identified SFP markers. This small degree of overlap was not surprising as calculations of gene expression levels average the hybridization intensities for all probes of a probe set and specifically eliminate the outlier probe values (http://www.affymetrix.com). Individual SFPs have only a small effect on quantifying gene expression levels (Kliebenstein et al. 2006
GEM detection with RIL data We tested the utility of the RIL distributions of gene expression levels to identify potential GEMs. When the same RIL distribution algorithm was applied to the scaled gene expression data for all genes from the 296 RIL GeneChips, only 42 or 70 GEMs were identified with a gap value of 2.0 or 1.5, respectively. Filtering markers after genotyping did not eliminate any markers; 93% of the markers were previously identified with the parental distribution method. Therefore, identifying GEMs based on the RIL rather than parental distributions resulted in fewer markers; however, there were few missing scores with GEMs identified from RIL distributions (0.5% missing scores for the 42 GEMs with gap >2.0 and 0.6% missing scores for the 70 GEMs with gap >1.5) (Supplemental Fig. 7).
We demonstrated that hybridization of gene expression microarrays with RNA allows the simultaneous phenotyping and genotyping of a segregating RIL population without requiring an independent analysis of the genomic DNA for each line. This dual use of the data saves both time and expense and allows confirmation of the source for each analyzed RNA. The ATH1 GeneChip, containing multiple 25-mer probes per gene, allowed the determination of marker genotypes at the level of gene expression with GEMs, as well as at the level of hybridization to individual probes with SFP markers. These dominant markers facilitated efficient high-density haplotyping of homozygous inbred RILs and identification of the majority of recombination breakpoints in this sample of 148 lines. These new markers enabled the construction of a high-density genetic map of the Bay-0 x Sha RIL population.
Both types of dominant markers are applicable to species other than Arabidopsis, including those for which genomics resources may be limited. However, their accessibility depends on the microarray technology available. Since GEMs can be developed by using either short or long oligonucleotide microarrays, they provide an alternative for identifying microarray-based markers, which previously required short oligonucleotide probes (Winzeler et al. 1998
Gene Expression Markers
In contrast to SFPs, the use of RIL distributions for GEM marker identification was not an improvement over the parental min-max method (Table 1). This is likely due to transgressive segregation, epistasis, and genotype x environment interactions that influence gene expression patterns in segregating populations (Mackay 2001 The genetic map order of the vast majority of GEMs was consistent with the expected physical order of the genes represented on the microarray (see Supplemental Fig. 1). Therefore, for this subset of genes showing distinct bimodal parental distributions, most of the eQTLs are cis to the gene being analyzed and thus represent an extreme class of ELPs (Supplemental Fig. 1). If this is a common phenomenon, a GEM-derived genetic map provides the opportunity to approximate the physical order of such genes in less intensively studied species. A disadvantage of markers that rely on quantifying transcript levels is that gene expression can be influenced by the environment. A GEM that is robust under one set of experimental conditions (e.g., tissue, developmental stage) may not perform similarly under a different set of conditions. This is not a major issue for genotypically stable populations such as advanced generation RILs, where genotypes of the homozygous lines need to be determined only once, and the haplotypes and a linkage map can be subsequently used to explore additional traits of interest. For less genetically stable experimental population structures, if expression microarrays have already been used to obtain expression profile data, GEM-derived genotypes can be leveraged from the existing data. Any candidate markers can be tested for phenotypic instability so that the most stable markers can be selected.
SFP markers
Because the boundaries of the allele-specific SFPdev ratio distributions could not be fully delineated by using parental microarray data, we developed a second approach to identify SFP markers. This approach queried the progeny data for polymorphisms, taking advantage of the higher level of replication available per locus in a RIL population: Microarray data from two biological replicates for each of 148 RILs provided
Marker number and population size
High-density dominant markers and residual heterozygosity
Future directions
Plant material and experimental conditions Seeds for A. thaliana accessions Bayreuth (Bay-0), Shahdara (Sha), and a Bay-0 x Sha F8 RIL population were obtained from TAIR (http://www.arabidopsis.org; this population was originally developed by Loudet et al. (2002) Preliminary replicated factorial experiments were conducted with Bay-0 and Sha plants grown on three separate dates in a growth chamber under short day conditions (8 h light at 100120 µEi, 20°C day/ 20°C night) to maintain plants in the vegetative phase. As part of a larger experiment investigating differential gene expression, at 6 wk these plants were sprayed to run-off with 0.02% Silwet L77, a surfactant (Lehle Seeds; http://www.arabidopsis.com) as a control treatment. All rosette leaves of three plants per genotype-time point combination were bulk harvested 4, 28, or 52 h post-treatment and quick-frozen in liquid nitrogen. The same tissue and developmental stage was used for all of the experiments described here. Subsequently, the Bay-0 x Sha RIL experiment was conducted. Five plants for each replicate of 148 RILs, plus parental controls, were grown in growth chambers under short day conditions (8 h light at 100120 µEi, 20°C day/ 20°C night) for 6 wk. At 6 wk, the plants were sprayed to run-off with 0.02% Silwet L77 and harvested 28 h later. All rosette leaves of three plants per genotype were bulk harvested and quick-frozen in liquid nitrogen. Because of limitations in growth chamber space, the two biological replications of 148 RILs plus controls were grown sequentially, one complete replication at a time.
RNA isolation and microarray hybridization
Microarray quality control
Microarray data analysis for genotyping with GEMs
In addition to examining differences in gene expression levels, we also considered the distributions of parental gene expression values. For each gene, the minimum and maximum gene expression values within the 16 GeneChips from a given parental genotype were used to define the end-points of the parental expression distribution for that gene (Bay-0min/Bay-0max and Shamin/Shamax) and to compare the expression value distributions from the two genotypes. We estimated the distance between the parental gene expression value distributions by calculating the "gap" for each differentially expressed gene, by dividing the minimum value from the higher expressing parent by the maximum value for the lower expressing parent. For example, in genes where the average Sha expression values were greater than the average Bay-0 expression values, gap = Shamin/Bay-0max. If the parental expression value distributions did not overlap, the gap calculation yielded a value >1. Inclusion of the gap criterion enforces selection of genes whose expression levels exhibit bimodal distributions in the parents, since our goal was to use these genes as markers. Of the 1431 differentially expressed genes, 324 had a gap value
Defining allele scoring rules for genotyping the RILs If RIL gene expression values fell between the factorial experiment parental distributions, this created ambiguous genotypes in the RILs, resulting in missing scores. To minimize the number of missing genotype scores in the RILs that resulted from using only 16 replicates of each parental genotype, we developed a slicing approach, in which each parental distribution was sliced into 1000 segments (see Supplemental Fig. 9). If the Bay-0 and Sha gene expression value distributions showed a gap >1.1, one segment was added sequentially to the appropriate end of each parental distribution until the adjusted gap was equal to 1.1. For example, for genes where Sha is the higher expressing parent, slices were iteratively added to the lower end of the Sha distribution to decrease the Shamin value, resulting in an adjusted Shamin value, Shamin-adj. Slices were likewise added to the higher end of the Bay-0 distribution. If the Bay-0 and Sha parental distributions were close together, with a gap <1.1, slices were subtracted iteratively from each parental distribution until the gap was equal to 1.1. Again, this resulted in adjusted distribution endpoints. By dividing each parental distribution into 1000 segments, the slicing approach conservatively and incrementally refined the parental distribution boundaries to better fit the boundaries of the RIL data and minimize missing marker data (allele assignments) in the RILs. The slicing approach is in effect a model optimization algorithm that attempts to maximize the use of the full data set for allele assignments.
GEM allele scoring A custom Python script was written to use the slicing approach to adjust the parental distributions and then to assign genotype scores based on RIL GeneChip expression data (http://elp.ucdavis.edu/data/analysis/elp_map/). A "master locus" file was generated with raw expression marker genotypes, in which Bay-0 (B) or Sha (A) genotypes were assigned only if both replicate RIL GeneChips gave identical genotypes for a GEM; otherwise the marker was scored as a missing genotype for that RIL. The master locus file was processed by the Python MadMapper RECBIT program version V248 (http://cgpdb.ucdavis.edu/XLinkage/MadMapper/) to filter the data set. GEMs with >10% missing data were excluded, as were markers displaying pronounced distortion (1:3 or greater) from the expected 1:1 allele distribution in the RILs.
Microarray data analysis for SFP marker genotyping Two different methods and algorithms were developed to use SFPdev values to detect and use SFPs as genetic markers. The first algorithm identified SFPs by using the 32 parental GeneChips from the factorial experiments and then assigned marker genotypes to the 148 RILs based on the parental SFPdev distributions. The second algorithm identified SFPs by using the RIL GeneChips and used the RIL SFPdev distributions, in conjunction with the parental SFPdev distributions, to assign marker genotypes.
We thank Rebecca Walker and Tanya Tang for technical assistance and Steve Edberg for assistance with data management and dissemination. This research was supported by the National Science Foundation 2010 Project, grant MCB-0115109 to D.A.S., R.W.M., and R.W.D.
4 Corresponding author.
E-mail mlwest{at}ucdavis.edu; fax (530) 752-9659. [Supplemental material is available online at www.genome.org. The microarray data from this study have been submitted to ArrayExpress under accession nos. E-TABM-60, E-TABM-61, and E-TABM-62. Algorithms and scripts are available at http://elp.ucdavis.edu/data/analysis/elp_map/ and at http://elp.ucdavis.edu/data/analysis/sfp_map/SFPscan.html.] Article published online before print. Article and publication date are at http://www.genome.org/cgi/doi/10.1101/gr.5011206
Altschul S.F., Madden T.L., Schaffer A.A., Zhang J., Zhang Z., Miller W., Lipman D.J. 1997. Gapped BLAST and PSI-BLAST: A new generation of protein database search programs. Nucleic Acids Res. 25: 33893402. . The Arabidopsis Genome Initiative. 2000. Analysis of the genome sequence of the flowering plant Arabidopsis thaliana. Nature 408: 796815.[CrossRef][Medline] Borevitz J.O., Liang D., Plouffe D., Chang H.-S., Zhu T., Weigel D., Berry C.C., Winzeler E., Chory J. 2003. Large-scale identification of single-feature polymorphisms in complex genomes. Genome Res. 13: 513523. Brem R.B. and Kruglyak L. 2005. The landscape of genetic complexity across 5700 gene expression traits in yeast. Proc. Natl. Acad. Sci. 102: 15721577. Cui X., Xu J., Asghar R., Condamine P., Svensson J.T., Wanamaker S., Stein N., Roose M., Close T.J. 2005. Detecting single-feature polymorphisms using oligonucleotide arrays and robustified projection pursuit. Bioinformatics 21: 38523859. de Koning D.-J. and Haley C.S. 2005. Genetical genomics in humans and model organisms. Trends Genet. 21: 377381.[CrossRef][Medline] Doerge R.W. 2002. Mapping and analysis of quantitative trait loci in experimental populations. Nat. Rev. Genet. 3: 4352.[CrossRef][Medline] Gautier L., Cope L., Bolstad B.M., Irizarry R.A. 2004. Affy: Analysis of Affymetrix GeneChip data at the probe level. Bioinformatics 20: 307315. Gentleman R.C., Carey V.J., Bates D.M., Bolstad B., Dettling M., Dudoit S., Ellis B., Gautier L., Ge Y., Gentry J.et al. 2004. Bioconductor: Open software development for computational biology and bioinformatics. Genome Biol. 5: R80.[CrossRef][Medline] Gupta P.K. and Rustgi S. 2004. Molecular markers from the transcribed/expressed region of the genome in higher plants. Funct. Integr. Genomics 4: 139162.[Medline] Hazen S.P. and Kay S.A. 2003. Gene arrays are not just for measuring gene expression. Trends Plant Sci. 8: 413416.[CrossRef][Medline] Irizarry R.A., Bolstad B.M., Collin F., Cope L.M., Hobbs B., Speed T.P. 2003. Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res. 31: e15. Jansen R.C. and Nap J.-P. 2001. Genetical genomics: The added value from segregation. Trends Genet. 17: 388391.[CrossRef][Medline] Kliebenstein D.J., West M.A.L., van Leeuwen H., Kim K., Doerge R.W., Michelmore R.W., St. Clair D.A. 2006. Genomic survey of gene expression diversity in Arabidopsis thaliana. Genetics 172: 11791189. Kumar L.S. 1999. DNA markers in plant improvement: An overview. Biotechnol. Adv. 17: 143182.[CrossRef][Medline] Loudet O., Chaillou S., Camilleri C., Bouchez D., Daniel-Vedele F. 2002. Bay-0 x Shahdara recombinant inbred line population: A powerful tool for the genetic dissection of complex traits in Arabidopsis. Theor. Appl. Genet. 104: 11731184.[CrossRef][Medline] Mackay T.F.C. 2001. The genetic architecture of quantitative traits. Annu. Rev. Genet. 35: 303339.[CrossRef][Medline] Nordborg M., Hu T.T., Ishino Y., Jhaveri J., Toomajian C., Zheng H., Bakker E., Calabrese P., Gladstone J., Goyal R.et al. 2005. The pattern of polymorphism in Arabidopsis thaliana. PLoS Biol. 3: e196.[CrossRef][Medline] Ronald J., Akey J.M., Whittle J., Smith E.N., Yvert G., Kruglyak L. 2005. Simultaneous genotyping, gene-expression measurement, and detection of allele-specific expression with oligonucleotide arrays. Genome Res. 15: 284291. Rostoks N., Borevitz J.O., Hedley P.E., Russell J., Mudie S., Morris J., Cardle L., Marshall D.F., Waugh R. 2005. Single-feature polymorphism discovery in the barley transcriptome. Genome Biol. 6: R54.[CrossRef][Medline] Schadt E.E., Monks S.A., Drake T.A., Lusis A.J., Che N., Colinayo V., Ruff T.G., Milligan S.B., Lamb J.R., Cavet G.et al. 2003. Genetics of gene expression surveyed in maize, mouse and man. Nature 422: 297302.[CrossRef][Medline] Winzeler E.A., Richards D.R., Conway A.R., Goldstein A.L., Kalman S., McCullough M.J., McCusker J.H., Stevens D.A., Wodicka L., Lockhart D.J.et al. 1998. Direct allelic variation scanning of the yeast genome. Science 281: 11941197.
Received December 12, 2005; accepted in revised format March 31, 2006. This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||