Let’s say you want to download some sequences from the sequence read archive (SRA) at NCBI. How do you do it? First, we need to make sure the SRA toolkit binaries are in your path.
Agency's central repository for sequence data. Using fully indexed columnar database design, the SRA toolkit has reduced lossless compression of sequence, qualities, and alignment properties from 32 bits per base in 2008 to under 5 bits per base for 1000 genomes phase 1 data. The cSRA technology provides a single toolkit interface that. The updated version of the popular Security Risk Assessment (SRA) Tool was released in October 2018 to make it easier to use and apply more broadly to the risks of the confidentiality, integrity, and availability of health information. The tool diagrams HIPAA Security Rule safeguards and provides enhanced functionality to document how your.
Should produce the full path to your fastq-dump binary. If it’s not present,then we’ll need to add the directory where it is stored to your path. Copythe following commands (for tcsh and bash, respectively) and run them to addthe SRA toolkit bin directories into your path.
Now you can run exec tcsh or exec bash (or exit and re-login) to get yourupdated $PATH. Now, when you type which fastq-dump, you should be presentedwith the full path to the fastq-dump binary.
The next step is to set the path where the temporary (large) sequence readarchive files will be stored. By default, these go into your home directory,and they can quickly use up all of your home directory space. Therefore, wewant to specify a location to store them that is on a /nfs drive.
Generally, this location will be something like:
So, identify the full path to this location, and then copy it, and run thefollowing command:
Type 4, and then paste the path that you previously copied. Type Y and<ENTER>.
The SRA Toolkit Trace
You can confirm that these changes were made by running this command:
You should see that /repository/user/main/public/root has the value that youprovided to vdb-config.
Next, we’ll need to download the SRA accession(s) that you are interested in.
Navigate to the SRA website and copy the run accession(s) that you want. These have the SRR prefix. Generally I like toput these into a text file. I name the file accs.txt but the name isn’timportant for the operation as long as you remember the purpose of the file inthe future.
For the prefetch stage, you’ll want to sign into files.cgrb.oregonstate.eduinstead of shell.cgrb.oregonstate.edu. This is to help reduce web trafficcongestion on the shell. Once you are on files, you should navigate to thefolder that contains the accs.txt file, and type this command:
This will feed the accessions to the prefetch command, which will result in theraw sequences being downloaded from NCBI’s servers. You should see someprogress messages as the files download.
Next, head back over to shell.cgrb.oregonstate.edu so we can extract the FASTQ files from the raw prefetched data. At this point it’s important to note what type of reads you are expecting. You’ll have to ensure that you get pairedend reads from the SRA accessions where they are expected. The newer programcalled fasterq-dump appears to be aware of paired-end datasets, and splitsthem accordingly even if the option is not specified. Therefore, you shouldbe able to run:
And get your reads extracted into .fastq files. By default, fasterq-dump uses6 threads, but you can specify a different amount using the -e flag. I wouldNOT recommend submitting this type of operation as an array job becauseyou will hammer the filesystem and potentially bring everyone’s jobs to acrawl. You need to specify the temporary directory with -t so that theprogram uses the local hard drive for the node to store the intermediate filesand only copy the final .fastq files to the networked file system.
Using the -I'{}' flag of xargs, your accessions will be extrated serially,which is what we want in this case, in order to reduce the load on the file servers.
Now, you should be able to ls and see your brand new .fastq files named withthe SRRXXXXX.fastq for single end data, and SRRXXXXX_[12].fastq for paired enddata. You can explicitly specify the -S flag to split the paired end filesas well if you aren’t getting the expected outputs.
Use Sra Toolkit
Happy downloading!