Re: [Samtools-help] fail to read header in v1.10, v1.11

2021-03-11 Thread John Marshall
On 10 Mar 2021, at 21:04, Remik Ziemlinski wrote: > The attached file worked with version 1.9, but is failing with 1.10 and 1.11. > Is the file valid? Is there a regression after v1.9? > [...] > # output from version 1.9 > @HD VN:1.6 SO::unsorted > @SQSN:referenceLN:5495 Your @HD line

Re: [Samtools-help] Samtools collate - location of temporary files

2020-06-08 Thread John Marshall
On 8 Jun 2020, at 13:16, Keiran Raine wrote: > > Just a quick query. Where does samtools write the temporary files for the > collate step? Can I assume /tmp or the location defined by $TMPDIR? At the moment if you want them to appear somewhere in particular other than /tmp (which is

Re: [Samtools-help] samtools sort command results in symbols calculation

2020-04-23 Thread John Marshall
On 22 Apr 2020, at 17:10, Pedro Hollanda Carvalho mailto:hollandacarva...@gmail.com>> wrote: I'm unable to perform samtools sort command. Result is a weird buch of symbols followed by letters and numbers as follows (just a part of it): yOR�G(��< {`�m�wF�D/�q|lOS�}�H This weird bunch of

Re: [Samtools-help] samtools not working with BWA -C flag

2019-03-14 Thread John Marshall
On 14 Mar 2019, at 15:14, Aengus Stewart wrote: > I completely understand that and I can fix it. However I am just pointing > out the current default we are getting from the > illumina sequencers. > > So either > Illumina needs to conform to the current SAM format in bcl2fastq > The SAM format

Re: [Samtools-help] Bug in tabix: ignores 0-based (-0) option.

2018-04-17 Thread John Marshall
On 17 Apr 2018, at 14:40, Heredia Genestar, Txema wrote: > After further reading the documentation I've finally found the part where it > explains the -0 option. > > I was mistaken and the -0 option is not there if you fancy one or another > indexing. It is there for the

Re: [Samtools-help] Bug in tabix: ignores 0-based (-0) option.

2018-04-17 Thread John Marshall
On 16 Apr 2018, at 15:55, Heredia Genestar, Txema wrote: > HTSLIB 1.5 and 1.8 both have a bug in tabix. It ignores the option for > 0-based input files. You are right that there is awkwardness here, but the full story is rather more involved. The -0 option is

Re: [Samtools-help] Extremely long reference sequences

2018-03-10 Thread John Marshall
On 9 Mar 2018, at 23:56, Nowoshilow,Sergej > wrote: Apparently, the BAM file is not quite correctly sorted after all. A simple test samtools view test.sorted.bam | cut -f3 | uniq proofs that it is indeed the case, since the scaffold

Re: [Samtools-help] mpileup output (base column)

2018-02-22 Thread John Marshall
On 21 Feb 2018, at 20:05, Nicholas Hill wrote: > Say my pileup line is this: > chr373912 A 21 g,,..G,.gGGGgGGg,.Ggg JJ So at chr3:73912 the reference was an A. On the forward strand (read1), there > are 7 guanine base pairs that aligned to the

Re: [Samtools-help] CSI stats bins

2017-12-14 Thread John Marshall
On 15 Dec 2017, Brent Pedersen wrote: > With bam/tabix, we can recognize the stats bin in index 37450 > > It is not documented how to find this bin for CSI which can have real > data in 37450. There exists a draft of a fleshed-out CSI document [1], but alas it still needs

Re: [Samtools-help] additional header lines "@GA"

2017-07-07 Thread John Marshall
On 7 Jul 2017, at 15:39, Gerben Menschaert wrote: > I’m testing interoperability with existing software tools using our proBAM > format. > > Within the format we added an novel header line, holding information on the > annotation used: > @GA AS:ENSEMBL

Re: [Samtools-help] please remove my email from the list

2017-01-05 Thread John Marshall
On 5 Jan 2017, at 06:52, Hadas Ner Gaon wrote: > [unsubscribe] I have manually unsubscribed from the mailing list. For anyone else wanting to unsubscribe in the future: the URL at the bottom of every mail you get from the list has a form for unsubscribing

Re: [Samtools-help] samtools sort for paired-end .bam cannot find chromosome name in text header

2017-01-04 Thread John Marshall
On 3 Jan 2017, at 16:21, Holbrook J. wrote: > I am trying to manipulate .bam files created by ernebs5 > (http://erne.sourceforge.net) aligning against hg19. > I am running samtools 1.3.1 [...] > > samtools sort -T /dev/shm/jostemp -@ 8 -m 4G -o sample1b_paired_sorted.bam

Re: [Samtools-help] Samtools mpileup reference is different from input reference fasta

2016-12-14 Thread John Marshall
On 14 Dec 2016, at 11:54, John Marshall <j...@sanger.ac.uk> wrote: > > On 13 Dec 2016, at 23:56, Jacob Ulirsch <julir...@broadinstitute.org> wrote: >> Here is output from faidx, showing the correct fasta sequence for these >> positions: >> >> samtools

Re: [Samtools-help] Samtools mpileup reference is different from input reference fasta

2016-12-14 Thread John Marshall
On 13 Dec 2016, at 23:56, Jacob Ulirsch wrote: > When I run samtools (v1.3.1) mpileup I am running into some very concerning > issues. The reference bases for the output pileup file are not in the same > position as the references in the input fasta. In fact, they

Re: [Samtools-help] Information about the algorithm

2016-12-02 Thread John Marshall
On 28 Nov 2016, at 17:51, Sebastian Gregoricchio wrote: > I am a student of molecular biology (master degree) and I am studying genome > assembly against a reference genome. > I have a, maybe stupid, question: if i have a quality raw in a fastq file > that

Re: [Samtools-help] samtools-1.2 install problem

2016-11-28 Thread John Marshall
On 25 Nov 2016, at 18:09, yincl2013 wrote: > Recently, when i installed the samtools-1.2 on redhat 6.6 system, i got the > following problem: > gcc -pthread -o samtools bam_index.o bam_plcmd.o sam_view.o bam_cat.o > bam_md.o bam_reheader.o bam_sort.o bedidx.o kprobaln.o

Re: [Samtools-help] faidx error_could not build fai index

2016-11-21 Thread John Marshall
On 21 Nov 2016, at 05:20, 陈然 wrote: > Thank you very much for your reply. I tried the command you advised me, it > shows like: > > $ cd /biotool/samtools-1.3.1 > $ strace -e open ./samtools faidx /data/hg19.fasta > [...] > open("/data/hg19.fasta", O_RDONLY) = -1

Re: [Samtools-help] Instruction to access bam files on aws s3 storage

2016-09-27 Thread John Marshall
On 26 Sep 2016, at 19:49, Xing Xu wrote: > > I am trying to access bam files that are stored on aws S3 storage of my own > bucket. > > I read the release note of samtools and htslib 1.3 support the access to aws > s3. > > Could you provide a detailed instruction that

Re: [Samtools-help] bcftools index Segmentation fault

2016-08-23 Thread John Marshall
On 22 Aug 2016, at 21:22, Thomas W. Blackwell wrote: > > I would try downloading the file and indexing your local copy. That is indeed what the OP did: >> $ bcftools index geno.TRA.bcf >> Segmentation fault > Unless I'm mistaken, nothing in the bcftools documentation >

Re: [Samtools-help] incorrect reference bases

2016-07-26 Thread John Marshall
On 22 Jul 2016, at 21:49, Annie Cowell wrote: > samtools mpileup -C50 -Bug -t AD -Q10 -f my.fasta sample.bam | bcftools call > -mv -Ov > my.vcf > > However, I am getting unusual bases for a couple of indels in the reference > field, such as W (as below) and R. > >

Re: [Samtools-help] tabix giving error

2016-05-27 Thread John Marshall
On 18 May 2016, at 18:23, Arti Tandon wrote: > I am using bcftools-1.3.1/htslib-1.3.1/tabix* to index the dbNSFP file to be > used by the program SnpSift, using the following commands and am getting an > error: > > $ (head -n 1 dbNSFP3.2c_variant.chr1 ; cat

Re: [Samtools-help] segfault: bcftools stats

2016-05-27 Thread John Marshall
On 27 May 2016, at 06:14, Peter Johansson wrote: > Seems the problematic entry had a negative QUAL (-2147480064), which looks > very suspicious. Need to go upstream and look for the source of that. QUAL is Phred-scaled, so by definition valid values are non-negative. So the

Re: [Samtools-help] samtools view with -U option

2016-05-09 Thread John Marshall
On 8 May 2016, at 17:27, Peter Chovanec wrote: > Why doesn't this work: > > samtools view sample.bam 2:33050509-33154206 -U without-region.sam > > or > > samtools view -b sample.bam 2:33050509-33154206 -o other_stuff.bam -U > without-region.bam You don't say

Re: [Samtools-help] when installing 1.31. on Centos (redhat) linux I get bam_tview_curses.c:46:5: error: unknown type name ‘WINDOW', no obvious reason

2016-05-03 Thread John Marshall
On 3 May 2016, at 19:11, Dan Hyatt wrote: > > I am installing samtools 1-3-1, the newest version released last week > and trying to compile it on Centos (redhat) 6.7 [...] > > I checked and verified that I installed ncurses and devel-ncurses as > seen at the bottom.

Re: [Samtools-help] number of mapped reads with flagstat

2016-04-28 Thread John Marshall
On 28 Apr 2016, at 15:03, Tagliamonte,Massimiliano S wrote: > ... I'm ashamed to ask... what is the dash ( "-" )before redirecting the > output? > i.e.: flagstat - > myfile.stats. If I don't use the dash, it gives me a > broken pipe error. I did some googling, still not

[Samtools-help] Samtools/BCFtools/HTSlib 1.3.1 released

2016-04-22 Thread John Marshall
New 1.3.1 versions of Samtools, BCFtools, and HTSlib have been released. This is mainly a bug fix release, fixing a number of bugs and issues in December's 1.3 release. In particular: * Improved management of samtools sort temporary files. Running several separate "... | samtools sort |

Re: [Samtools-help] When would 1.3.1 be released?

2016-04-05 Thread John Marshall
On 4 Apr 2016, at 21:40, Shinn Kondo wrote: > I was using version 1.1 of samtools mpileup, but it exited abnormally > after the following warning: "[bam_plp_destroy] memory leak: 45. > Continue anyway." > > Reading a post about the same warning in >

Re: [Samtools-help] mpileup - [E..hts_open_format] error

2016-03-19 Thread John Marshall
On 16 Mar 2016, at 11:20, Wybouw, Nicky wrote: > I get this output > > [E..hts_open_format] fail to open file '.bam' > [mpileup] failed to open '.bam': Value too large for defined data type > > The input BAM files worked fine using the old version and are only 8.2GB big. You

Re: [Samtools-help] -N setting

2016-03-04 Thread John Marshall
On 4 Mar 2016, at 12:11, Ryan Kevin wrote: > In version 0.1.19 of bcftools view there is the setting: > > -N which is defined as Skip sites where the REF field is not A/C/G/T > > I am currently running version 1.2 of bcftools and I realised that I still > have -N set when

Re: [Samtools-help] gzopen64 error

2016-02-22 Thread John Marshall
On 22 Feb 2016, at 08:34, William Hsiao wrote: > Thanks for your help. Below is the output you requested. Btw, I was able to > get Samtools to compile by issuing "sudo make LDFLAGS="-L/usr/local/lib”, > where a newer version of zlib (libz.so.1.2.8) is available. Could it

Re: [Samtools-help] rmdup Segmentation Fault

2016-02-17 Thread John Marshall
On 16 Feb 2016, at 23:22, Friedman, Ryan wrote: > The entire pipeline works fine up until rmdup where it immediately results in > a segmentation fault. > > What's particularly interesting is that this seg fault only occurs in some > files. I originally thought this had

Re: [Samtools-help] error building htslib 1.3 on mac OS X (macports)

2016-02-16 Thread John Marshall
On 16 Feb 2016, at 15:41, Stephane Plaisance wrote: > gcc: error: -compatibility_version only allowed with -dynamiclib [...] > libhts.dylib: $(LIBHTS_OBJS) > $(CC) -Wl,-export_dynamic -install_name > $(libdir)/libhts.$(LIBHTS_SOVERSION).dylib

Re: [Samtools-help] error building samtools 1.3

2016-02-16 Thread John Marshall
On 16 Feb 2016, at 11:44, Stephane Plaisance wrote: > I get the following on OSX 10.10.5 [...] > > gcc: error: unrecognized command line option '-rdynamic' See https://sourceforge.net/p/samtools/mailman/message/34699333/ ; most GCC compiler drivers understand

Re: [Samtools-help] Windows10 cygwin64 and samtools problem installing

2015-12-31 Thread John Marshall
On 31 Dec 2015, at 10:15, Robert May wrote: > > I have just installed cygwin64 on a Windows10 machine and have run into > problems with installing as below. Any assiance appreciated. [...] > checking whether the C compiler works... no > configure: error: in

Re: [Samtools-help] Seek and tell a BAM file

2015-12-08 Thread John Marshall
On 7 Dec 2015, at 21:47, Claudio Alberti wrote: > I am implementing a parser that is able to read the BAM file in pairs so > whenever I read a record where pos < mpos I search for the mate and I > create a pair structure. > Once I find the mate I have to roll back to

Re: [Samtools-help] Seek and tell a BAM file

2015-12-08 Thread John Marshall
On 8 Dec 2015, at 13:56, Claudio Alberti wrote: > > It seems that bgfz_seek and bgzf_tell can work as well, do you see any issue > with them? It would be a similar situation to that noted in that September BCF / VCF.GZ thread, though you are in a better position due

Re: [Samtools-help] I cam cross a problem in compiling the C script using the samtools api

2015-12-01 Thread John Marshall
On 1 Dec 2015, at 12:20, hgong wrote: > I am new to the samtools. I install samtools 1.2 as it's directed and it > succeeds. when I try compile your example of samtools C : > > http://samtools.sourceforge.net/sam-exam.shtml This is an old web page containing a program

Re: [Samtools-help] bcftools annotate could not parse header line

2015-11-05 Thread John Marshall
On 4 Nov 2015, at 21:25, Tagliamonte,Massimiliano S wrote: > I am trying to add an annotation column to my vcf file, after calling > variants with the Samtools 1.x pipeline. I am using bcftools annotate, but I > keep getting the same error regarding one of the headers: >

Re: [Samtools-help] samtools index commands pretends it has succeeded, but it has not

2015-11-05 Thread John Marshall
Hello Michael, On 5 Nov 2015, at 16:56, Michael Nuhn wrote: > I am using samtools 1.2. When I run the index command like this: > > /software/ensembl/funcgen/samtools index > /lustre/scratch109/ensembl/funcgen/mn1/problem_bam/HEL9217:hist:BR2_WCE_3526_bwa_samse_1.unfiltered.bam

Re: [Samtools-help] bgzip removal of EOF blocks?

2015-10-05 Thread John Marshall
On 3 Oct 2015, at 23:52, Pär Larsson <par.lars...@umu.se> wrote: > Sorry to bother you with a question relating to an older discussion > (http://sourceforge.net/p/samtools/mailman/message/34109200/). John Marshall > expressed concerns that removal of the 28 byte EOF block from

Re: [Samtools-help] Samtools faidx "Segmentation fault (core dumped)"

2015-09-28 Thread John Marshall
On 24 Sep 2015, at 09:06, Maria Sutton wrote: > I am preparing my fasta file so I can apply it to HaplotypeCaller. At the > stage where I use samtools faidx tmp.fasta I'm getting segmentation fault > (core dumped). At first it specified the errors so I got rid of

Re: [Samtools-help] Samtools mpileup: some positions missing fields

2015-09-21 Thread John Marshall
On 16 Sep 2015, at 22:24, Brendan Kohrn wrote: > I found the following issue in the output from samtools mpileup (-f), with a > single bam file input: > > gi|255961284|ref|NC_011713.2| 140 G 1 ,$ C > gi|255961284|ref|NC_011713.2| 149 A 0 >

Re: [Samtools-help] BG Zip

2015-09-11 Thread John Marshall
On 10 Sep 2015, at 21:54, Terry Casstevens wrote: > If you use bgzip to zip an already gzipped file, does that result in a > correctly bgzipped file? > > Or should you first ungzip the file and then bgzip? Your tools are expecting a file with a single layer of bgzipping, so

Re: [Samtools-help] Pileup format

2015-09-07 Thread John Marshall
On 7 Sep 2015, at 13:49, Tom Adlerteg wrote: > We are currently working on a tool for statistical analysis of substitutions > and indels. As we see it is no longer supported to be able to print mapping > qualities as a separate column when using samtools mpileup. This >

Re: [Samtools-help] what is a reference skip?

2015-08-27 Thread John Marshall
On 26 Aug 2015, at 21:02, Samantha Klasfeld sj...@cornell.edu wrote: I am using samtools mpileup and I was wondering what a reference skip is. I got output that showed there were nucleotides that skipped the reference. Then, when I compared the reads to the reference I noticed that they were

Re: [Samtools-help] Header

2015-07-06 Thread John Marshall
On 6 Jul 2015, at 12:04, ma.meyerh...@freenet.de wrote: I read something about the -C parameter and retried with bwa mem -aHMpP instead of bwa mem -aCHMpP, but the header looks the same: samtools view -H ./output/KMT/9557-FGS-R.sam @SQSN:B.FR.1983.HXB2-LAI-IIIB-BRU.K03455LN:9719

Re: [Samtools-help] tabix bug on cat'ed vcf.gz

2015-05-14 Thread John Marshall
On 13 May 2015, at 12:17, Kanterakis, Efstathios ekantera...@illumina.com wrote: bgzip chr1_h.vcf bgzip chr2.vcf cat chr1_h.vcf.gz chr2.vcf.gz test.vcf.gz ...i.e., constructs test.vcf.gz with many BGZF blocks, including an EOF trailer block from each of chr1_h.vcf.gz (in the middle of

Re: [Samtools-help] samtools sort creates millions of files

2015-03-11 Thread John Marshall
On 11 Mar 2015, at 14:24, Bob Harris rshar...@bx.psu.edu wrote: That's one seriously broken grammar that considers 1750M to mean 175 but thinks 1.75G means 1. You are not wrong. Coincidentally we've been working on improving ways to specify numbers in htslib for a while, and can probably

Re: [Samtools-help] picard compatibility with samtools -u bams?

2015-02-12 Thread John Marshall
On 11 Feb 2015, at 13:12, Sendu Bala s...@sanger.ac.uk wrote: I'm using the latest samtools (1.1) and picard (1.128). If I make a bam using the -u option of eg. calmd, it now creates a raw uncompressed bam (as opposed to a bgzip bam with compression level 0, which is what samtools v0 did).

[Samtools-help] Samtools/BCFtools/HTSlib 1.2 released

2015-02-04 Thread John Marshall
New 1.2 versions of Samtools and BCFtools and a 1.2.1 version of HTSlib have been released. These provide improvements in CRAM support and variant calling, as well as a number of bug fixes and speed improvements. (The 1.2 HTSlib release was immediately superseded by 1.2.1 when a compatibilty

Re: [Samtools-help] unsubscribing from samtools-help

2014-12-08 Thread John Marshall
On 4 Dec 2014, at 17:52, Kevin kevin.geo...@mho.com wrote: remove this email from the list….I have another one I am having this sent to …so this is a duplicate Presuming you are asking for your @mho.com email address to be unsubscribed from the mailing list, I have done so. If you meant

Re: [Samtools-help] samtools depth question: minQLen

2014-11-20 Thread John Marshall
On 19 Nov 2014, at 17:47, Peter Cock p.j.a.c...@googlemail.com wrote: I have no idea what 'minQLen' was short for, unless the Q was a typo? It would be Minimum Query Length, where query is a synonym for read (imagine you were an aligner being asked to map a read...). This is also what

Re: [Samtools-help] samtools mpileup -- error status 141

2014-11-13 Thread John Marshall
On 13 Nov 2014, at 12:23, Karel Břinda karel.bri...@gmail.com wrote: I observe a strange problem and I am not able to find out what is the exact reason of it. When I use samtools mpileup | another_program, samtools return error code 141. When a Unix shell reports that a command has an exit

Re: [Samtools-help] Tag lost during samtools sort

2014-11-06 Thread John Marshall
On 6 Nov 2014, at 01:28, Mark Ravinet mravi...@nig.ac.jp wrote: Grepped lines from a typical header are as follows: @PG ID:GSNAPPN:gsnapVN:2013-11-27 CL:gsnap -d stick_ref --maxsearch=10 -M 0 -m 5 -t 6 -n 1 -A sam --quiet-if-excessive --terminal-threshold=10 -i 2

[Samtools-help] samtools/bcftools/htslib 1.1 released

2014-09-23 Thread John Marshall
New 1.1 versions of htslib, samtools, and bcftools have been released. These are minor bug-fix releases, and include the following notable fixes and new functionality compared to 1.0: * Samtools fixmate and flagstat now consider supplementary reads * Sorting BAM files with thousands of

Re: [Samtools-help] Samtools sort (merging) is slow

2014-09-05 Thread John Marshall
On 5 Sep 2014, at 06:52, Adam Skarshewski a.skarshew...@uq.edu.au wrote: Running into a problem with samtools 1.0 sort, specifically the merging phase. It's very slow compared to the previous version. I have a 965MB bam file (which is already sorted) Does your BAM file have rather a lot of