Hi Vladimir,
That's good. Increasing the -m setting to 2 to 4 G may reduce maximum
memory. There's a section in the manual that discusses memory usage and the
merge phase memory increases as -m is reduced and it's possible to set -m
too low.
Anyway, I'm glad it's done. If you try biobambam could you let me know how
it compares to novosort?
Thanks, Colin
On 28 April 2015 at 19:21, Vladimir Morozov <vmmoro...@gmail.com> wrote:
> I was able to finish the job with -@ 2 -m 1G parameters. Though it
> occupied about 4.5G memory by the end of the job
>
> On Mon, Apr 27, 2015 at 9:33 PM, Colin Hercus <co...@novocraft.com> wrote:
>
>> Hi Vladimir,
>>
>> You could try using novosort, it's included in download at
>> http://www.novocraft.com/support/download/
>>
>> novosort -m 4G -t . my_sample.bam >sorted.bam
>>
>> To get multi-threading you need a license and I've attached a 1 month
>> trial. Just extract the file novoalign.lic from the attached tar and save
>> into the folder with novosort program. You can freely use without a license
>> but only 1 thread.
>>
>> Kind Regards, Colin
>>
>> Colin Hercus,
>> *Novocraft Technologies Sdn Bhd* <http://www.novocraft.com/>
>> C-23A-05, 3 Two Square, Jalan 19/1, Section 19,
>> 46300 Petaling Jaya, Selangor Darul Ehsan, Malaysia
>> Tel: +6016 2482 668 Fax: +603 7960 0540
>> www.novocraft.com
>>
>>
>>
>> On 28 April 2015 at 05:19, Vladimir Morozov <vmmoro...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I struggle to sort whole-genome 110G BAM file. I run 8G RAM and 2CPU
>>> google instance.
>>>
>>> vmorozov@gstat:/disk/genomes$ samtools
>>>
>>> Program: samtools (Tools for alignments in the SAM format)
>>> Version: 1.2 (using htslib 1.2.1)
>>>
>>>
>>> I tried:
>>> nohup samtools sort -n -@ 2 -m 2G my_sample.bam sort > e&
>>> the command crashed after 2 hours creating ~40G of sort.*.bam files.
>>> Empty error/stdoutput. Now I have launched:
>>> nohup samtools sort -n -@ 2 -m 1G my_sample.bam sort > e&
>>>
>>> and I observe gradual increase of samtools occupied memory. After 30 min
>>> run and creating 20Gb of temporary files, samtools occupied ~4G of memory:
>>>
>>> top - 21:16:10 up 4:38, 2 users, load average: 2.15, 2.01, 1.70
>>> Tasks: 94 total, 1 running, 93 sleeping, 0 stopped, 0 zombie
>>> %Cpu(s): 89.2 us, 10.7 sy, 0.0 ni, 0.2 id, 0.0 wa, 0.0 hi, 0.0 si,
>>> 0.0 st
>>> KiB Mem: 7660696 total, 7526644 used, 134052 free, 1244 buffers
>>> KiB Swap: 0 total, 0 used, 0 free. 3234156 cached
>>> Mem
>>>
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
>>> COMMAND
>>> 6045 vmorozov 20 0 4116052 3.769g 1796 S 196.5 51.6 49:14.03
>>> samtools
>>> 26 root 39 19 0 0 0 S 2.3 0.0 0:10.80
>>> khugepaged
>>> 39 root 20 0 0 0 0 S 1.0 0.0 3:13.44
>>> kswapd0
>>>
>>>
>>> Is it memory leakage? I guess it will crash eventually. Any suggestion
>>> how to get the sort job done?
>>>
>>> Thanks
>>> Vlad
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> One dashboard for servers and applications across Physical-Virtual-Cloud
>>> Widest out-of-the-box monitoring support with 50+ applications
>>> Performance metrics, stats and reports that give you Actionable Insights
>>> Deep dive visibility with transaction tracing using APM Insight.
>>> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
>>> _______________________________________________
>>> Samtools-help mailing list
>>> Samtools-help@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/samtools-help
>>>
>>>
>>
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Samtools-help mailing list
> Samtools-help@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/samtools-help
>
>
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Samtools-help mailing list
Samtools-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/samtools-help
