Sorry, forgot to CC this to the list...
---------- Forwarded message ----------
Date: Tue, 26 Dec 2017 13:18:06 +0000
From: rmd <r...@sanger.ac.uk>
To: Hu Jing <hj_hanna2...@hotmail.com>
Subject: Re: [Samtools-help] samtools view error: convert SAM to BAM fail
silently
On 2017-12-26 00:56, Hu Jing wrote:
Recently we ran into an issue with converting SAM to BAM using samtools view
command called in python, please see the commands below,
cmd = 'samtools view -b -o {} {}'.format(output_BAM, input_SAM)
os.system(cmd)
The SAM was generated from the Bowtie2, and appended with a few more tags
defined by our own, such as sequence of UMI (unique molecular index). The SAM
file is ~4Gb in size. The samtools we're using is version 1.3.1.
Samtools 1.3.1 is very out of date. Do you see the same problem if you
use the latest release (1.6)? If nothing else, it may be better at
telling you that something went wrong.
When we tried to run the above command in a VM with 32Gb RAM, it failed
silently, and we couldn't see the BAM file generated. After we increased to
64Gb RAM, the BAM was generated.
Wonder why increasing the RAM could help the conversion of SAM to BAM (64Gb is
really big compared to the SAM size 4Gb)? Is there any option that we may use
to print the logs for easier troubleshooting?
samtools view shouldn't need anywhere near this much memory. I suspect
something in your file is triggering a bug that caused it to allocate
much more memory than it should have, but it's difficult to tell without
having the actual file. A number of these bugs have been fixed since
version 1.3.1, so it would be very useful to know what happens with the
latest release. You could also try using `head` to limit the number of
SAM file lines passed to `samtools view`. If you get no output at all
then it's very likely that the failure happens when reading the header,
so you may be able to reproduce the problem with a much smaller file.
Please let us know how you get on,
Rob Davies r...@sanger.ac.uk
The Sanger Institute http://www.sanger.ac.uk/
Hinxton, Cambs., Tel. +44 (1223) 834244
CB10 1SA, U.K. Fax. +44 (1223) 494919
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Samtools-help mailing list
Samtools-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/samtools-help
