Hey Brad,
it looks like you do not have enough space on your disk. Could you make
sure that your system's temporary directory has enough disk space?
Alternatively, you can use "TMP_DIR" to point it to a place with more space.
N
On Fri, Nov 21, 2014 at 8:39 PM, Langhorst, Brad <langho...@neb.com> wrote:
>
> I was finally able to parse these nextseq runs.
>
> The key was to allow more reads per tile in RAM.
>
> I used:
>
> MAX_READS_IN_RAM_PER_TILE=12000000 \
>
> I think it’s supposed to work with default settings, using disk if
> necessary, but i get these kinds of back traces If i use the default
> setting (i’m certainly not out of disk space).
>
>
> INFO 2014-11-12 16:05:11 IlluminaBasecallsConverter Read
> 139,000,000 records. Elapsed time: 00:36:56s. Time for last 1,000,000:
> 15s. Last read position: */*
> INFO 2014-11-12 16:05:15 IlluminaBasecallsConverter Write
> 93,000,000 records. Elapsed time: 00:37:00s. Time for last 1,000,000:
> 14s. Last read position: */*
> Exception in thread "pool-1-thread-26" ERROR 2014-11-12 16:05:16
> IlluminaBasecallsConverter Failure encountered in worker thread;
> attempting to shut down remaining worker threads and terminate ...
> net.sf.samtools.util.RuntimeIOException: java.io.IOException: No space
> left on device
> at
> net.sf.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:232)
> at
> net.sf.samtools.util.SortingCollection.add(SortingCollection.java:152)
> at
> net.sf.picard.illumina.IlluminaBasecallsConverter$TileProcessingRecord.addRecord(IlluminaBasecallsConverter.java:372)
> at
> net.sf.picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:482)
> at
> net.sf.picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:577)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:345)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> at
> org.xerial.snappy.SnappyOutputStream.flush(SnappyOutputStream.java:101)
> at
> org.xerial.snappy.SnappyOutputStream.close(SnappyOutputStream.java:137)
> at
> net.sf.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:223)
> ... 7 more
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:503)
> at
> net.sf.picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.awaitWorkComplete(IlluminaBasecallsConverter.java:628)
> at
> net.sf.picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:248)
> at
> net.sf.picard.illumina.IlluminaBasecallsToFastq.doWork(IlluminaBasecallsToFastq.java:157)
> at
> net.sf.picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:179)
> at
> net.sf.picard.cmdline.CommandLineProgram.instanceMainWithExit(CommandLineProgram.java:120)
> at
> net.sf.picard.illumina.IlluminaBasecallsToFastq.main(IlluminaBasecallsToFastq.java:304)
> INFO 2014-11-12 16:06:03 IlluminaBasecallsConverter Read
> 140,000,000 records. Elapsed time: 00:37:48s. Time for last 1,000,000:
> 52s. Last read position: */*
>
>
> On Nov 20, 2014, at 5:19 PM, Nils Homer <nho...@broadinstitute.org>
> wrote:
>
> Hey Brad,
>
> it looks like someone or something stopped the execution midstream. Is
> there a time limit on your job, or some other resource manager that would
> shut this down? This tool supports NextSeq data so the machine type should
> not be an issue.
>
> N
>
> On Mon, Nov 10, 2014 at 3:56 PM, Langhorst, Brad <langho...@neb.com>
> wrote:
>
>> [Mon Nov 10 11:07:48 EST 2014]
>> picard.illumina.IlluminaBasecallsToFastq
>> BASECALLS_DIR=/mnt/ngswork/langhorst/141029_NS500355_0002_AH15U8BGXX/Data/Intensities/BaseCalls
>> LANE=2 RUN_BARCODE=141029_NS500355_0002_AH15U8BGXX MACHINE_NAME=NS500355
>> FLOWCELL_BARCODE=H15U8BGXX READ_STRUCTURE=151T7B151T
>> MULTIPLEX_PARAMS=/mnt/ngswork/langhorst/141029_NS500355_0002_AH15U8BGXX/lane2_multiplex_params.txt
>> NUM_PROCESSORS=32 ADAPTERS_TO_CHECK=[INDEXED, DUAL_INDEXED, NEXTERA_V2,
>> FLUIDIGM] APPLY_EAMSS_FILTER=true FORCE_GC=true
>> MAX_READS_IN_RAM_PER_TILE=1200000 MINIMUM_QUALITY=2
>> INCLUDE_NON_PF_READS=true READ_NAME_FORMAT=CASAVA_1_8
>> COMPRESS_OUTPUTS=false VERBOSITY=INFO QUIET=false
>> VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000
>> CREATE_INDEX=false CREATE_MD5_FILE=false
>> [Mon Nov 10 11:07:48 EST 2014] Executing as langhorst@seq-himem01-i on
>> Linux 3.13.0-32-generic amd64; OpenJDK 64-Bit Server VM 1.7.0_65-b32;
>> Picard version: 1.124(69ecf101f612fdc0f3d555aa2d3cc0b1ea193c68_1415030499)
>> IntelDeflater
>>
>> ...
>>
>> INFO 2014-11-10 11:25:11 IlluminaBasecallsConverter Write
>> 62,000,000 records. Elapsed time: 00:17:15s. Time for last 1,000,000:
>> 6s. Last read position: */*
>> ERROR 2014-11-10 11:25:11 IlluminaBasecallsConverter Failure
>> encountered in worker thread; attempting to shut down remaining worker
>> threads and terminate ...
>> java.lang.InterruptedException
>> at java.lang.Object.wait(Native Method)
>> at java.lang.Object.wait(Object.java:503)
>> at
>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.awaitWorkComplete(IlluminaBasecallsConverter.java:695)
>> at
>> picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:311)
>> at
>> picard.illumina.IlluminaBasecallsToFastq.doWork(IlluminaBasecallsToFastq.java:186)
>> at
>> picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:187)
>> at
>> picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:89)
>> at
>> picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:99)
>>
>> Anybody have experience parsing nextseq data will these tools?
>> The code does refer to nextseq, but I'm not sure how well tested it is.
>>
>> Brad
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Samtools-help mailing list
>> Samtools-help@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/samtools-help
>>
>>
>
>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Samtools-help mailing list
Samtools-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/samtools-help
