|
|
|
|
Published online before print
February 15, 2006, 10.1101/gr.4337206 Genome Res. 16:527-535, 2006 ©2006 by Cold Spring Harbor Laboratory Press; ISSN 1088-9051/06 $5.00
Methods Broad-spectrum respiratory tract pathogen identification using resequencing DNA microarrays1 Center for Bio/Molecular Science & Engineering, Code 6900, Naval Research Laboratory, Washington, District of Columbia 20375, USA; 2 NOVA Research Incorporated, Alexandria, Virginia 22308, USA; 3 Epidemic Outbreak Surveillance Advanced Diagnostics Laboratory, Lackland Air Force Base, San Antonio, Texas 78236, USA; 4 Texas A&M University System, San Antonio, Texas 78223, USA; 5 Department of Infectious Disease, Wilford Hall USAF Medical Center, Lackland Air Force Base, San Antonio, Texas 78236, USA; 6 Department of Defense Center for Deployment Health Research, Naval Health Research Center, San Diego, California 92186, USA; 7 School of Computational Sciences, George Mason University, Manassas, Virginia 20110, USA; 8 Air Force Institute for Operational Health, Brooks Air Force Base, San Antonio, Texas 78235, USA; 9 HQ USAF/SGR, Falls Church, Virginia 22041, USA
The exponential growth of pathogen nucleic acid sequences available in public domain databases has invited their direct use in pathogen detection, identification, and surveillance strategies. DNA microarray technology has offered the potential for the direct DNA sequence analysis of a broad spectrum of pathogens of interest. However, to achieve the practical attainment of this potential, numerous technical issues, especially nucleic acid amplification, probe specificity, and interpretation strategies of sequence detection, need to be addressed. In this report, we demonstrate an approach that combines the use of a custom-designed Affymetrix resequencing Respiratory Pathogen Microarray (RPM v.1) with methods for microbial nucleic acid enrichment, random nucleic acid amplification, and automated sequence similarity searching for broad-spectrum respiratory pathogen surveillance. Successful proof-of-concept experiments, utilizing clinical samples obtained from patients presenting adenovirus or influenza virus-induced febrile respiratory illness (FRI), demonstrate the ability of this approach for correct species- and strain-level identification with unambiguous statistical interpretation at clinically relevant sensitivity levels. Our results underscore the feasibility of using this approach to expedite the early surveillance of diseases, and provide new information on the incidence of multiple pathogens.
The critical need for advanced infectious diagnostic and surveillance systems has taken on a new urgency with increased concerns over bioterrorism agents as well as natural pathogens (e.g., Bacillus anthracis, coronavirus, avian influenza virus). A DNA microarray platform that can simultaneously detect and characterize many different types of human pathogens that cause similar symptoms provides considerable potential for both medical use and national defense purposes (Bodrossy and Sessitsch 2004
The effective use of microarrays for pathogen detection requires the optimization of several factors, such as sample amplification, probe specificity, and interpretation strategy in order to obtain unambiguous and reproducible results (Striebel et al. 2003
Another hurdle to using spotted microarrays is that the design of specific oligonucleotide probes for pathogen identification is dependent on assumptions regarding target sequence composition. Long (5070mer) oligonucleotide probes used in most prior studies have the disadvantage of decreased specificity (threshold for differentiation at 75%87% sequence similarity), making it necessary to target multiple markers and rely on hybridization patterns for pathogen identification, which can lead to unquantifiable errors (Bodrossy and Sessitsch 2004
The exponentially increasing availability of microbial sequences makes it possible to envision the use of direct sequence for routine pathogen diagnostics and surveillance; however, this requires that pathogen sequence information be rapidly obtained. "Resequencing" microarrays use "tiled" sets of 105 to 106 probes of either 25mers or 29mers, containing one perfectly matched and three mismatched probes per base for both strands of target genes (Hacia 1999
In this study, our overall objective was to demonstrate the utility of a resequencing microarray approach for simultaneous detection of respiratory pathogens in a format that can be used in a clinical environment without requiring the design of pathogen-specific PCR primers (W. Wilson et al. 2002
Specificity of the RPM v.1 chip The accuracy of RPM v.1 chips for sequence-specific pathogen detection was validated using clinical and/or controlled laboratory samples. Samples were amplified with either pathogen-specific PCR or random amplification strategies and then hybridized to the arrays. To test whether prototype tile regions could be used for the identification of a broad number of variants without relying on predetermined hybridization patterns, we used febrile respiratory illness (FRI)-causing adenoviruses (HAdV) as our model system. The capability of the RPM v.1 to discriminate HAdV serotypes was tested by interrogating degenerate PCR amplicons (Lin et al. 2004 Our results demonstrate that the tile regions of HAdV-4, HAdV-5, and HAdV-7 of RPM v.1 can differentiate various FRI-associated HAdV strains (Table 1), and prove that prototype tile regions can be used for identifying a broad range of variants. Strain-level identification was obtained in all cases except for subgroup B2 strains that were identified only as belonging to that subgroup. In a similar fashion, the remaining tiled regions on the RPM v.1 were successfully validated with the exception of those for West Nile virus. For these validation tests, control laboratory strains (Table 2; Supplemental data) were used, except in the case of influenza A virus H5N1 tiled regions. Instead, total RNA obtained from a patient infected with influenza A-H5N1 in Southeast Asia was used for validating the H5N1 tile regions (Table 1). Each pathogen was validated with at least three independent amplifications. Our results reproducibly revealed that prototype reference regions exhibited little or no discernible cross-hybridization, and interference from one pathogen with one of the others never caused an erroneous identification. Similar results were obtained with either specific or random amplification (data not shown). No false positives were obtained due to microarray base call or analysis errors.
RPM v.1 process development For resequencing (i.e., genotyping) applications, the Affymetrix GeneChip system and attendant protocols were optimized for highly accurate detection of SNPs by using specific PCR amplification. In order to achieve unbiased pathogen detection with RPM v.1, a random amplification strategy was developed. Rather than the random amplification protocols previously developed that require multiple amplification steps (Wang et al. 2002
To enhance detection sensitivity, separate human DNA and RNA subtraction pathways coupled with random NA amplification were developed. For DNA targets (e.g., HAdV-4), the isolated total NA from nasal wash or throat swab specimens was first subjected to McrBC enzymatic digestion at methylated CpG sites (Panne et al. 1999 10 kb), followed by the subtraction of repetitive sequences using Cot-1 human DNA (Fig. 1). The remaining DNA was subjected to whole-genome amplification (Lovmar et al. 2003
Capability of RPM v.1 for multiple pathogen detection
In addition to accurately identifying single pathogenic species, Figure 2D provides an example of another benefit of using this protocol for pathogen detectionthe ability to detect co-infections without using reference control hybridizations for comparison. A throat swab specimen collected from a symptomatic patient previously vaccinated against HAdV-4 and HAdV-7 was shown to harbor adenovirus-specific DNA that hybridized specifically to the HAdV-5 and HAdV-7 prototype regions. The result of our analysis for subsequences from the HAdV-5 E1A gene (Fig. 3A) indicated the presence of HAdV-5 as expected. However, our analysis of the subsequences from the HAdV-7 E1A prototype region revealed that the subsequences were a match to HAdV-21 and not HAdV-7 (Fig. 3B; annotated genome sequence of HAdV-21, GenBank accession number AY601633). Similar results were also obtained from the HAdV-7 hexon and fiber gene prototype regions and strongly suggested the presence of two adenoviral species, HAdV-5 and HAdV-21. This finding was verified by several independent conventional and molecular adenovirus identification methods (G.J. Vora, B. Lin, K. Gratwick, C.E. Meador, C. Hansen, C. Tibbetts, D.A. Stenger, M. Irvine, D. Seto, A. Purkayastha, et al., in prep.).
For influenza virus strains, it is important not only to distinguish subtypes but also to identify the differences associated with significant shifts of the subtype from year to year. The accuracy of our microarray to identify these variations was demonstrated for an influenza A virus (Fig. 3C). Visual examination of the hybridization of amplicons from a clinical sample on the influenza A virus hemagglutinin (H1) gene prototype sequence correctly identified the presence of an influenza A virus. Sequence-based REPI analysis revealed the identity of the virus to be most nearly identical to subtype A/Madrid/1082/2001, another H1N1 strain that had been circulating during the same flu season as the A/New Caledonia/20/99 vaccine strain. This identification corresponded to identification made based on the sequence obtained using the conventional DNA sequencing. For every other clinical sample identified as influenza A virus H3 or H1 whose sequence was obtained using conventional DNA sequencing methods, the two methods identified strains that corresponded with each other (data not shown). The accuracy of the sequence information produced by the RPM v.1 for typing-level identification has been established and was not affected by either the reduced stringency settings of the Affymetrix base-calling algorithm or by the methods used to randomly amplify the pathogen targets from clinical specimens. A more detailed analysis of the accuracy of this microarray for specific strain identification compared with conventional sequencing for several influenza A and B strains is covered in a separate paper (Wang et al. 2006
Preliminary clinical study A preliminary clinical study was performed with samples collected from the NHRC and EOS team at Lackland Air Force Base. For comparison, two amplification strategies, random amplification and multiplex PCR, were employed with the microarray for the same set of clinical samples. As shown in Table 2, 21 influenza A virus culture-positive (15 nasal wash and six throat swab) samples were tested using both random and multiplex PCR amplification methods. Using multiplex PCR, 13 out of 13 samples were correctly diagnosed in comparison to the culture method. While using random amplification, 15 out of 18 samples were identified. RPM v.1 not only identified the samples as H3N2 and H1N1 subtype but also differentiated these samples, demonstrating the potential of the resequencing microarray. Using the hemagglutinin gene sequence as an example, sample NW2003111403.7 collected in November 2003 was identified as A/New York/61A/2003, a relative of the dominant strain (A/Fujian/411/2002) for the 20032004 flu season, while sample NHRC30481, collected in February 2000, was identified as A/France/11/00 (H3N2) strain, a relative of the dominant strain (A/Panama/2007/99) for the 19992000 flu season. It is not surprising to see that samples collected from the same geographic region in the same season usually contained similar strains. Similar BLAST search results were observed from sequences generated for samples in which both random amplification and multiplex PCR were done, further suggesting that random amplification methods correspond well with multiplex PCR (Table 2). In addition, our assay correctly identified and typed 11 clinical samples (9-HAdV-4, 1-HAdV-3/HAdV-4 coinfection, and 1-Streptococcus pyogenes) when compared with traditional culture detection methods (Table 2). Two clinical samples that tested negative using conventional methods also did not have any pathogens detected when tested with our microarray (data not shown). These results demonstrate the ability of our microarray-based diagnostic to correctly identify and type clinically relevant HAdV and influenza A strains in a manner consistent with conventional culture detection. An additional study was carried out with clinical samples collected from the NHRC to further assess the utility of the microarray-based diagnostic for respiratory pathogen detection. The samples (n = 41) consisted of negative and positive throat swabs in viral transport medium from subjects with clinically documented respiratory illness. At the sites of collection, the samples were tested using conventional methods and sent to us in a coded fashion for testing. The experiments were conducted by two independent investigators, and the sample identities were revealed only after the resulting assessments had been finalized. The comparison demonstrated a complete concordance between our method and conventional methods for 19 of 21 samples (Table 3) for HAdV-4positive clinical samples, and 20 of 20 negative samples. The failure to detect pathogens present in two HAdV-4 samples in this test and three influenza A samples in the first study, and the five false negatives, was probably due to the insufficient sensitivity of the current method. However, no falsepositive results were obtained due to microarray base call or analysis errors. Future efforts will focus on improving assay sensitivity.
We have demonstrated a straightforward approach that capitalizes on the ever-increasing availability of pathogen nucleic acid sequences that will provide strain-level information without relying on fixed hybridization patterns and does not rely on specific oligonucleotide sequences to amplify for specific targets. This approach not only allows simultaneous detection and differentiation of common circulating respiratory pathogens at clinically relevant sensitivity levels, but also enables us to identify coinfections and rarely encountered or typically unexpected pathogens. This technology does not require a priori knowledge of a differential hybridization pattern for pathogen identification. Thus, it is not necessary to build up a database of reference hybridization patterns through control experiments for differentiating subtypes of pathogens. Also, the falsepositive rate caused by cross-hybridization when two sequences share a high degree of similarity was greatly reduced through this approach. In >300 experiments for all validated tiled sequences on the RPM v.1, no falsepositive results were obtained when using clinical samples, and no misidentifications were made when using controlled laboratory samples (Table 1; Supplemental Fig. 1). Another salient benefit of this technique is demonstrated in the case of complex mixture samples, i.e., HAdV coinfection and Flu Vaccine (data not shown); the microarray not only distinguished the presence of two or more coinfectants, but also identified them correctly at the strain level. In all cases, the interpretation of a pathogens identity is much more straightforward.
While this system demonstrates remarkably low falsepositive rates (a high specificity), the system remains somewhat limited in sensitivity as indicated by the false negatives. The sensitivity of the current system can detect adenovirus at target concentrations of 103 copies/µL of the starting clinical sample and influenza A virus at 2.5 x 103 plaque-forming units/µL using a combination of human NA background subtraction and random amplification of the remaining NA. Due to the use of specific primers and the exponential amplification of PCR, it is not surprising to see that the multiplex PCR is more sensitive than random amplification. Limitations in sensitivity can result in false negatives, especially if a patient is tested early or late in the infection or the pathogen of interest does not typically shed in high titer, such as HAdV or influenza A virus. Our future work will focus on improving assay sensitivity using more complete subtraction of background human DNA and RNA. Nevertheless, this assay may still be useful for some applications. This assay is not yet optimized for detailed strain-level identification, speed (
In comparison to the current state of the art, which would require multiple diagnostic tests to discern the offending agent, our assay is able not only to look for the most commonly occurring infectious agents, but also to survey for less common pathogens in a single test. The ability to order a single assay for effective differential diagnosis among the majority of pathogens causing FRI syndrome will increase the number of diagnoses made with far fewer assays. In a public health or an epidemic outbreak scenario, the ability to rapidly identify less common pathogens among a background of seasonal FRI will enable more effective identifications, leading to a better response to a naturally occurring epidemic outbreak or even a bioterrorism event. This information, combined with clinical symptom data and confirmatory laboratory tests, will result in more accurate disease reporting, decreased disease exposure, and improved outcomes for individuals and public health. Because viral agents cause most infections of the respiratory tract, it is satisfying that we could detect and type both DNA and RNA viruses from clinical samples using random amplification methods at clinically relevant sensitivity levels (Couch et al. 1966
RPM v.1 design The RPM v.1 (Fig. 2A) was designed primarily to test the hypothesis that a single tiled region could act as a prototype for the identification of a broad number of variants without relying on predetermined hybridization patterns. Prototype regions were selected to allow for both efficient hybridization and unique identification of most or all of a subtype of pathogenic species. (For probe tiling information of RPM v.1, see Supplemental Table 1). Two pathogens, HAdV and influenza A virus (H1N1, H3N2 and H5N1), were treated in much more detail. They were selected based upon recent outbreak information (Erdman et al. 2002
Prototype strains
Clinical samples
Subtractive random amplification strategy
Random amplification for DNA samples was carried out with either bacteriophage For analysis of clinical specimens, samples were subjected to both DNA and RNA subtraction and amplification, then the amplified products were combined and subjected to purification and processing prior to hybridizing to the RPM v.1.
Multiplex RTPCR
Microarray hybridization and processing
Resequencing Pathogen Identifier (REPI)
Quantification of HAdV-4 and influenza A viruses
Similar assays were carried out to determine the plaque-forming units/µL of influenza A virus in each sample by using primers AMP-For and AMP-Rev (Stone et al. 2004
Sequencing confirmation
Support for this research was provided by the Defense Threat Reduction Agency, the United States Army Medical Material Research Command, the Air Force Medical Service (Office of HQ USAF Surgeon General), and the Office of Naval Research. The help and constructive suggestions from members of the Epidemic Outbreak Surveillance Consortium were gratefully acknowledged. Dr. Klaus Schafers constructive advice is gratefully appreciated. We thank Margaret Ryan, Kevin Russell, and Christopher Barrozo at NHRC, Linda Canas at AFIOH, and Ted Hadfield at AFIP for kindly providing samples used in this study. This research has been conducted in compliance with all applicable federal and international regulations governing the protection of human subject in research, as documented in DoD protocol NHRC 1999.0002. The opinions and assertions contained herein are those of the authors and are not to be construed as official or reflecting the views of the Department of Defense or the U.S. Government.
10 These authors contributed equally to this study.
E-mail dstenger{at}cbmse.nrl.navy.mil; fax (202) 767-9598. [Supplemental material is available online at www.genome.org. REPI software is freely available at http://nrlbio.nrl. navy.mil/downloads/repi.zip.] Article published online ahead of print. Article and publication date are at http://www.genome.org/cgi/doi/10.1101/gr.4337206
12 Patent pending. This software embodies subject matter that is or may be claimed in one or more patent applications and/or issued patents. Please contact the Technology Transfer Office at the U.S. Naval Research Laboratory if you are interested in obtaining a license.
Altschul S.F., Gish W., Miller W., Myers E.W., Lipman D.J. 1990. Basic local alignment search tool. J. Mol. Biol. 215: 403410.[CrossRef][Medline] Bodrossy L. and Sessitsch A. 2004. Oligonucleotide microarrays in microbial diagnostics. Curr. Opin. Microbiol. 7: 110. Boivin G., Coulombe Z., Wat C. 2003. Quantification of the influenza virus load by real-time polymerase chain reaction in nasopharyngeal swabs of patients treated with oseltamivir. J. Infect. Dis. 188: 578580.[CrossRef][Medline] Bryant P.A., Venter D., Robins-Browne R., Curtis N. 2004. Chips with everything: DNA microarrays in infectious diseases. Lancet Infect. Dis. 4: 100111.[CrossRef][Medline] Cleland C.A., White P.S., Deshpande A., Wolinsky M., Song J., Nolan J.P. 2004. Development of rationally designed nucleic acid signatures for microbial pathogens. Expert Rev. Mol. Diagn. 4: 303315.[CrossRef][Medline] Couch R.B., Cate T.R., Fleet W.F., Gerone P.J., Knight V. 1966. Aerosol-induced adenoviral illness resembling the naturally occurring illness in military recruits. Am. Rev. Respir. Dis. 93: 529535.[Medline] Cutler D.J., Zwick M.E., Carrasquillo M.M., Yohn C.T., Tobin K.P., Kashuk C., Mathews D.J., Shah N.A., Eichler E.E., Warrington J.A. 2001. High-throughput variation detection and genotyping using microarrays. Genome Res. 11: 19131925. Erdman D.D., Xu W., Gerber S.I., Gray G.C., Schnurr D., Kajon A.E., Anderson L.J. 2002. Molecular epidemiology of adenovirus type 7 in the United States, 19662000. Emerg. Infect. Dis. 8: 269277.[Medline] Gingeras T.R., Ghandour G., Wang E., Berno A., Small P.M., Drobniewski F., Alland D., Desmond E., Holodniy M., Drenkow J. 1998. Simultaneous genotyping and species identification using hybridization pattern recognition analysis of generic Mycobacterium DNA arrays. Genome Res. 8: 435448. Hacia J.G. 1999. Resequencing and mutational analysis using oligonucleotide microarrays. Nat. Genet. 22: 164167.[CrossRef][Medline] Hoffmann E., Stech J., Guan Y., Webster R.G., Perez D.R. 2001. Universal primer set for the full-length amplification of all influenza A viruses. Arch. Virol. 146: 22752289.[CrossRef][Medline] Kessler N., Ferraris O., Palmer K., Marsh W., Steel A. 2004. Use of the DNA flow-thru chip, a three-dimensional biochip, for typing and subtyping of influenza viruses. J. Clin. Microbiol. 42: 21732185. Kolavic-Gray S.A., Binn L.N., Sanchez J.L., Cersovsky S.B., Polyak C.S., Mitchell-Raymundo F., Asher L.V., Vaughn D.W., Feighner B.H., Innis B.L. 2002. Large epidemic of adenovirus type 4 infection among military trainees: Epidemiological, clinical, and laboratory studies. Clin. Infect. Dis. 35: 808818.[CrossRef][Medline] Kothapalli R., Yoder S.J., Mane S., Loughran Jr. , T.P. . 2002. Microarray results: How accurate are they? BMC Bioinformatics 3: 22.[CrossRef][Medline] Kozal M.J., Shah N., Shen N., Yang R., Fucini R., Merigan T.C., Richman D.D., Morris D., Hubbell E., Chee M. 1996. Extensive polymorphisms observed in HIV-1 clade B protease gene using high-density oligonucleotide arrays. Nat. Med. 2: 753759.[CrossRef][Medline] Lin B., Vora G.J., Thach D., Walter E., Metzgar D., Tibbetts C., Stenger D.A. 2004. Use of oligonucleotide microarrays for rapid detection and serotyping of acute respiratory disease-associated adenoviruses. J. Clin. Microbiol. 42: 32323239. López M.M., Bertolini E., Olmos A., Caruso P., Gorris M.T., Llop P., Penyalver R., Cambra M. 2003. Innovative tools for detection of plant pathogenic viruses and bacteria. Int. Microbiol. 6: 233243.[CrossRef][Medline] Lovmar L., Fredriksson M., Liljedahl U., Sigurdsson S., Syvanen A.C. 2003. Quantitative evaluation by minisequencing and microarrays reveals accurate multiplexed SNP genotyping of whole genome amplified DNA. Nucleic Acids Res. 31: e129. Panne D., Raleigh E.A., Bickle T.A. 1999. The McrBC endonuclease translocates DNA in a reaction dependent on GTP hydrolysis. J. Mol. Biol. 290: 4960.[CrossRef][Medline] Saitoh-Inagawa W., Oshima A., Aoki K., Itoh N., Isobe K., Uchio E., Ohno S., Nakajima H., Hata K., Ishiko H. 1996. Rapid diagnosis of adenoviral conjunctivitis by PCR and restriction fragment length polymorphism analysis. J. Clin. Microbiol. 34: 21132116.[Abstract] Stone B., Burrows J., Schepetiuk S., Higgins G., Hampson A., Shaw R., Kok T. 2004. Rapid detection and simultaneous subtype differentiation of influenza A viruses by real time PCR. J. Virol. Methods 117: 103112.[CrossRef][Medline] Striebel H.M., Birch-Hirschfeld E., Egerer R., Foldes-Papp Z. 2003. Virus diagnostics on microarrays. Curr. Pharm. Biotechnol. 4: 401415.[CrossRef][Medline] Thompson W.W., Shay D.K., Weintraub E., Brammer L., Cox N., Anderson L.J., Fukuda K. 2003. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 289: 179186. Vora G.J., Meador C.E., Stenger D.A., Andreadis J.D. 2004. Nucleic acid amplification strategies for DNA microarray-based pathogen detection. Appl. Environ. Microbiol. 70: 30473054. Wang D., Coscoy L., Zylberberg M., Avila P.C., Boushey H.A., Ganem D., DeRisi J.L. 2002. Microarray-based detection and genotyping of viral pathogens. Proc. Natl. Acad. Sci. 99: 1568715692. Wang D., Urisman A., Liu Y.T., Springer M., Ksiazek T.G., Erdman D.D., Mardis E.R., Hickenbotham M., Magrini V., Eldred J. 2003. Viral discovery and sequence recovery using DNA microarrays. PLoS Biol. 1: 257260.[CrossRef] (in press).Wang Z., Daum L.T., Vora G.J., Metzgar D., Walter E.A., Canas L.C., Malanoski A.P., Lin B., Stenger D.A. on behalf of the Epidemic Outbreak Surveillance Consortium. 2006. Rapid, broad-spectrum identification of influenza viruses by resequencing microarrays. Emerg. Infect. Dis. Wilson W.J., Strout C.L., DeSantis T.Z., Stilwell J.L., Carrano A.V., Andersen G.L. 2002. Sequence-specific identification of 18 pathogenic microorganisms using microarray technology. Mol. Cell. Probes 16: 119127.[CrossRef][Medline] Wilson K.H., Wilson W.J., Radosevich J.L., DeSantis T.Z., Viswanathan V.S., Kuczmarski T.A., Andersen G.L. 2002. High-density microarray of small-subunit ribosomal DNA probes. Appl. Environ. Microbiol. 68: 25352541. Wong C.W., Albert T.J., Vega V.B., Norton J.E., Cutler D.J., Richmond T.A., Stanton L.W., Liu E.T., Miller L.D. 2004. Tracking the evolution of the SARS coronavirus using high-throughput, high-density resequencing arrays. Genome Res. 14: 398405.
Received June 24, 2005; accepted in revised format December 22, 2005. This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||