Hi Vladimir,

That's good. Increasing the -m setting to 2 to 4 G may reduce maximum
memory. There's a section in the manual that discusses memory usage and the
merge phase memory increases as -m is reduced and it's possible to set -m
too low.

Anyway, I'm glad it's done. If you try biobambam could you let me know how
it compares to novosort?

Thanks, Colin

On 28 April 2015 at 19:21, Vladimir Morozov <vmmoro...@gmail.com> wrote:

> I was able to finish the job with -@ 2 -m 1G  parameters. Though  it
> occupied about 4.5G memory by the end of the job
>
> On Mon, Apr 27, 2015 at 9:33 PM, Colin Hercus <co...@novocraft.com> wrote:
>
>> Hi Vladimir,
>>
>> You could try using novosort, it's included in download at
>> http://www.novocraft.com/support/download/
>>
>>       novosort -m 4G -t . my_sample.bam >sorted.bam
>>
>> To get multi-threading you need a license and I've attached a 1 month
>> trial. Just extract the file novoalign.lic from the attached tar and save
>> into the folder with novosort program. You can freely use without a license
>> but only 1 thread.
>>
>> Kind Regards, Colin
>>
>> Colin Hercus,
>> *Novocraft Technologies Sdn Bhd* <http://www.novocraft.com/>
>> C-23A-05, 3 Two Square, Jalan 19/1, Section 19,
>> 46300 Petaling Jaya, Selangor Darul Ehsan, Malaysia
>> Tel: +6016 2482 668   Fax: +603 7960 0540
>> www.novocraft.com
>>
>>
>>
>> On 28 April 2015 at 05:19, Vladimir Morozov <vmmoro...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I struggle to sort whole-genome 110G BAM file. I run 8G RAM and 2CPU
>>> google instance.
>>>
>>> vmorozov@gstat:/disk/genomes$ samtools
>>>
>>> Program: samtools (Tools for alignments in the SAM format)
>>> Version: 1.2 (using htslib 1.2.1)
>>>
>>>
>>> I tried:
>>> nohup samtools sort -n -@ 2 -m 2G my_sample.bam sort > e&
>>> the command crashed after 2 hours creating ~40G of sort.*.bam files.
>>> Empty error/stdoutput.  Now I have launched:
>>>  nohup samtools sort -n -@ 2 -m 1G my_sample.bam sort > e&
>>>
>>> and I observe gradual increase of samtools occupied memory. After 30 min
>>> run and creating 20Gb of temporary files, samtools occupied ~4G of memory:
>>>
>>> top - 21:16:10 up  4:38,  2 users,  load average: 2.15, 2.01, 1.70
>>> Tasks:  94 total,   1 running,  93 sleeping,   0 stopped,   0 zombie
>>> %Cpu(s): 89.2 us, 10.7 sy,  0.0 ni,  0.2 id,  0.0 wa,  0.0 hi,  0.0 si,
>>>  0.0 st
>>> KiB Mem:   7660696 total,  7526644 used,   134052 free,     1244 buffers
>>> KiB Swap:        0 total,        0 used,        0 free.  3234156 cached
>>> Mem
>>>
>>>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+
>>> COMMAND
>>>  6045 vmorozov  20   0 4116052 3.769g   1796 S 196.5 51.6  49:14.03
>>> samtools
>>>    26 root      39  19       0      0      0 S   2.3  0.0   0:10.80
>>> khugepaged
>>>    39 root      20   0       0      0      0 S   1.0  0.0   3:13.44
>>> kswapd0
>>>
>>>
>>> Is it memory leakage? I guess it will crash eventually. Any suggestion
>>> how to get the sort job done?
>>>
>>> Thanks
>>> Vlad
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> One dashboard for servers and applications across Physical-Virtual-Cloud
>>> Widest out-of-the-box monitoring support with 50+ applications
>>> Performance metrics, stats and reports that give you Actionable Insights
>>> Deep dive visibility with transaction tracing using APM Insight.
>>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>>> _______________________________________________
>>> Samtools-help mailing list
>>> Samtools-help@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/samtools-help
>>>
>>>
>>
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Samtools-help mailing list
> Samtools-help@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/samtools-help
>
>
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Samtools-help mailing list
Samtools-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/samtools-help

Reply via email to