Hi Stephen, I have a hard time thinking this was an error on your part if all you did was run the program. The space character was in the column that contains the actual bases from the read, and I can't figure out why that would contain a space character.
If you can provide me with your ExtractIlluminaBarcodes command line I'll try to figure out what is going on. -Alec On May 29, 2014, at 2:08 PM, Gire, Stephen <sg...@oeb.harvard.edu> wrote: > Oh thanks. I did use ExtractIlluminaBarcodes. Could this be something wrong > with my barcodeData file used for the extraction? I can try recreating the > file and running ExtractIlluminaBarcodes again. > > > On May 29, 2014, at 11:42 AM, Alec Wysoker <al...@broadinstitute.org> wrote: > >> Hi Stephen, >> >> There is a problem with your *_barcode.txt files. In some of the lines, >> there is a space character after the second barcode (the one in column 3). >> Were these created with ExtractIlluminaBarcodes, or did you use some other >> process, or did they get modified after creating with >> ExtractIlluminaBarcodes? >> >> The immediate solution is to remove all those space characters, but if there >> is a bug in ExtractIlluminaBarcodes I would like to know it. >> >> -Alec >> >> On May 28, 2014, at 4:11 PM, Gire, Stephen <sg...@oeb.harvard.edu> wrote: >> >>> I am demultiplexing a TruSeq small RNA run using 6 cycle barcodes. I am >>> able to Extract the Illumina barcodes into metrics.txt and they look fine. >>> However, when I try to convert the raw files to .bams using the >>> IlluminaBasecallsToSam.jar I keep getting several errors. I’ve used this >>> script with 8 cycle barcodes with no problem. >>> >>> The first error states that there was an unexpected number of elements when >>> parsing file s_2_1102_barcode.txt. It says the expected maximum of elements >>> per line is 5, which there are no more than 5, so I am a bit confused. >>> >>> Any help would be appreciated. I’ve never seen this error before and I’m >>> wondering if it’s a specific issue with smallRNA pipelines? >>> >>> Thanks. >>> >>> Log Report: >>> >>> Your job looked like: >>> >>> ------------------------------------------------------------ >>> # LSBATCH: User input >>> java -Xmx10g -jar >>> /seq/software/picard/current/bin/IlluminaBasecallsToSam.jar >>> BASECALLS_DIR=/broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls/ >>> LANE=2 READ_STRUCTURE=125T6B125T >>> LIBRARY_PARAMS=/idi/sabeti-data/SGire/EBOV_miRNA/library_params.txt >>> SEQUENCING_CENTER=Broad RUN_BARCODE=140114_SL-HDC_0407_AFCH7576ADXX >>> NUM_PROCESSORS=4 ADAPTERS_TO_CHECK=TRUSEQ_SMALLRNA >>> MAX_READS_IN_RAM_PER_TILE=100000 MAX_RECORDS_IN_RAM=100000 FORCE_GC=false >>> ------------------------------------------------------------ >>> >>> Exited with exit code 1. >>> >>> Resource usage summary: >>> >>> CPU time : 9.87 sec. >>> Max Memory : 191 MB >>> Max Swap : 10591 MB >>> >>> Max Processes : 3 >>> Max Threads : 25 >>> >>> The output (if any) follows: >>> >>> [Wed May 28 15:28:39 EDT 2014] picard.illumina.IlluminaBasecallsToSam >>> BASECALLS_DIR=/broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls >>> LANE=2 RUN_BARCODE=140114_SL-HDC_0407_AFCH7576ADXX SEQUENCING_CENTER=Broad >>> READ_STRUCTURE=125T6B125T >>> LIBRARY_PARAMS=/idi/sabeti-data/SGire/EBOV_miRNA/library_params.txt >>> ADAPTERS_TO_CHECK=[INDEXED, DUAL_INDEXED, NEXTERA_V2, FLUIDIGM, >>> TRUSEQ_SMALLRNA] NUM_PROCESSORS=4 FORCE_GC=false >>> MAX_READS_IN_RAM_PER_TILE=100000 MAX_RECORDS_IN_RAM=100000 >>> PLATFORM=illumina APPLY_EAMSS_FILTER=true MINIMUM_QUALITY=2 >>> INCLUDE_NON_PF_READS=true VERBOSITY=INFO QUIET=false >>> VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 CREATE_INDEX=false >>> CREATE_MD5_FILE=false >>> [Wed May 28 15:28:39 EDT 2014] Executing as sgire@node1005 on Linux >>> 2.6.18-194.8.1.el5 amd64; Java HotSpot(TM) 64-Bit Server VM 1.6.0_35-b10; >>> Picard version: 1.727(06603b8dd6204c5d29e2c4e28a55bef15a7324d1_1400679360) >>> JdkDeflater >>> INFO 2014-05-28 15:30:24 IlluminaBasecallsToSam DONE_READING >>> STRUCTURE IS 125T6B125T >>> ERROR Exception in thread "pool-1-thread-2" picard.PicardException: >>> Unexpected number of elements found when parsing file >>> /broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls/s_2_1102_barcode.txt: >>> 5. Expected a maximum of 5 elements per line:TTAGGC Y TTAGGC 0 >>> 3 >>> at >>> picard.util.AbstractInputParser.parseLine(AbstractInputParser.java:129) >>> at picard.util.AbstractInputParser.advance(AbstractInputParser.java:88) >>> at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44) >>> at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57) >>> at >>> picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29) >>> at >>> picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75) >>> at >>> picard.illumina.parser.IlluminaDataProvider.addData(IlluminaDataProvider.java:178) >>> at >>> picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:145) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:499) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:601) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> 2014-05-28 15:30:45 IlluminaBasecallsConverter Failure encountered in >>> worker thread; attempting to shut down remaining worker threads and >>> terminate ... >>> java.lang.InterruptedException >>> at java.lang.Object.wait(Native Method) >>> at java.lang.Object.wait(Object.java:485) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.awaitWorkComplete(IlluminaBasecallsConverter.java:653) >>> at >>> picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:269) >>> at >>> picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:205) >>> at >>> picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183) >>> at >>> picard.illumina.IlluminaBasecallsToSam.main(IlluminaBasecallsToSam.java:411) >>> WARNING 2014-05-28 15:30:45 IlluminaBasecallsConverter >>> Ignoring exception stopping background GC thread. >>> java.lang.NullPointerException >>> at >>> picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:284) >>> at >>> picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:205) >>> at >>> picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183) >>> at >>> picard.illumina.IlluminaBasecallsToSam.main(IlluminaBasecallsToSam.java:411) >>> Exception in thread "pool-1-thread-1" picard.PicardException: IOException >>> opening cluster binary file >>> /broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls/L002/s_2_1101.filter >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:119) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getByteIterator(MMapBackedIteratorFactory.java:66) >>> at >>> picard.illumina.parser.readers.FilterFileReader.<init>(FilterFileReader.java:68) >>> at picard.illumina.parser.FilterParser$1.<init>(FilterParser.java:55) >>> at >>> picard.illumina.parser.FilterParser.makeTileIterator(FilterParser.java:54) >>> at >>> picard.illumina.parser.PerTileParser.advanceTile(PerTileParser.java:80) >>> at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:121) >>> at >>> picard.illumina.parser.PerTileParser.maybeAdvance(PerTileParser.java:99) >>> at picard.illumina.parser.PerTileParser.next(PerTileParser.java:109) >>> at >>> picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:133) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:499) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:601) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.nio.channels.ClosedByInterruptException >>> at >>> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) >>> at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:304) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:113) >>> ... 14 more >>> Exception in thread "pool-1-thread-3" picard.PicardException: IOException >>> opening cluster binary file >>> /broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls/L002/s_2_1103.filter >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:119) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getByteIterator(MMapBackedIteratorFactory.java:66) >>> at >>> picard.illumina.parser.readers.FilterFileReader.<init>(FilterFileReader.java:68) >>> at picard.illumina.parser.FilterParser$1.<init>(FilterParser.java:55) >>> at >>> picard.illumina.parser.FilterParser.makeTileIterator(FilterParser.java:54) >>> at >>> picard.illumina.parser.PerTileParser.advanceTile(PerTileParser.java:80) >>> at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:121) >>> at >>> picard.illumina.parser.IlluminaDataProvider.hasNext(IlluminaDataProvider.java:104) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:498) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:601) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.nio.channels.ClosedByInterruptException >>> at >>> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) >>> at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:304) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:113) >>> ... 12 more >>> [Wed May 28 15:30:45 EDT 2014] picard.illumina.IlluminaBasecallsToSam done. >>> Elapsed time: 2.11 minutes. >>> Runtime.totalMemory()=367525888 >>> To get help, see http://picard.sourceforge.net/index.shtml#GettingHelp >>> Exception in thread "main" picard.PicardException: Failure encountered in >>> worker thread; see log for details. >>> at >>> picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:272) >>> at >>> picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:205) >>> at >>> picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183) >>> at >>> picard.illumina.IlluminaBasecallsToSam.main(IlluminaBasecallsToSam.java:411) >>> Exception in thread "pool-1-thread-4" picard.PicardException: IOException >>> opening cluster binary file >>> /broad/hptmp/SGire/20140522.140114_SL-HDC_0407_AFCH7576ADXX/Data/Intensities/BaseCalls/L002/s_2_1104.filter >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:119) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getByteIterator(MMapBackedIteratorFactory.java:66) >>> at >>> picard.illumina.parser.readers.FilterFileReader.<init>(FilterFileReader.java:68) >>> at picard.illumina.parser.FilterParser$1.<init>(FilterParser.java:55) >>> at >>> picard.illumina.parser.FilterParser.makeTileIterator(FilterParser.java:54) >>> at >>> picard.illumina.parser.PerTileParser.advanceTile(PerTileParser.java:80) >>> at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:121) >>> at >>> picard.illumina.parser.PerTileParser.maybeAdvance(PerTileParser.java:99) >>> at picard.illumina.parser.PerTileParser.next(PerTileParser.java:109) >>> at >>> picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:133) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:499) >>> at >>> picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:601) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: java.nio.channels.ClosedByInterruptException >>> at >>> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) >>> at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:304) >>> at >>> picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:113) >>> ... 14 more >>> ------------------------------------------------------------------------------ >>> Time is money. Stop wasting it! Get your web API in 5 minutes. >>> www.restlet.com/download >>> http://p.sf.net/sfu/restlet >>> _______________________________________________ >>> Samtools-help mailing list >>> Samtools-help@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/samtools-help >> > ------------------------------------------------------------------------------ Time is money. Stop wasting it! Get your web API in 5 minutes. www.restlet.com/download http://p.sf.net/sfu/restlet _______________________________________________ Samtools-help mailing list Samtools-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/samtools-help
