blt fastx-derep(1)

SYNOPSIS

blt fastx-derep < file.fastq > file-uniq.fastq

DESCRIPTION
blt fastx-derep remove replicates from a FASTA or FASTQ file by hashing each sequence and printing only records for which the hash has not previously been encountered. If memory is a limiting factor for your dereplication jobs, see fastq-derep.sh(3), a shell script utilizing the Unix sort command, which automatically breaks large files into chunks so they can be sorted within available memory. fastq-derep.sh is much slower, but may be able to complete tasks where other methods run out of memory.

EXAMPLES
blt fastx-derep < file.fasta > file-uniq.fasta
xzcat file.fasta.xz | blt fastx-derep > file-uniq.fasta
xzcat file.fastq.xz | blt fastx-derep > file-uniq.fastq

SEE ALSO
blt-fastx2tsv(1), blt-fastq-derep.sh(1)

AUTHOR
J. Bacon