Re: [galaxy-dev] samtools version detection issue

[Nate Coraor]

On Jan 20, 2012, at 12:12 AM, ambarish biswas wrote:

> Hi,
>   I'm not sure if this is related or not but I'm also noticed a
> similar problem where the program is not found in the path, but it
> exists. I have it submitted as an issue , but it could be related here
> so mentioned here. The problem started when I recently upgraded my
> Galaxy installation(It was running in cluster with SGE, in Red hat
> linux). Currently, I'm running Galaxy with drmaa in a cluster in Red
> Hat linux, where the web server  is running in one Server, and the
> execution is being performed in another. I noticed that programs like
> ClustalW, dialign-tx etc, which uses a wrapper, is giving error as
> follows:
> sh: dialign-tx: command not found
> The original command line in the wrapper is as follows:
> command = "dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" %
> (GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length,
> length_low_score_pos, input_file_name, output_file)
> 
> Even though, the programs are present in the Execution node's path,
> and if I add, explicit path, like "/usr/local/bin" in the command as
> bellow:
> command = "/usr/local/bin/dialign-tx %s/dialign-tx %s %s -g%s -n%s -D
> %s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity,
> max_frag_length, length_low_score_pos, input_file_name, output_file)
> 
> Then the programs works fine without any error.
> 
> I compared, the wrapper files with my old installation and the current
> installation, and both are 100% same. So, I guess the path to the
> executable is missing from the environment. I searched a lot, but
> couldn't find any solution to rectify this issue.
> Can someone please point out where I need to change to avoid adding
> explicit path to each programs.
> Thanks in advance.

I replied to your bitbucket ticket first, just in case you don't see it, here 
it is:

https://bitbucket.org/galaxy/galaxy-central/issue/702/program-command-not-found-in-path

--nate

> 
> 
> With Regards,
> 
> Ambarish Biswas,
> University of Otago
> Department of Biochemistry,
> Dunedin, New Zealand,
> 
> 
> On Fri, Jan 20, 2012 at 11:07 AM, Nate Coraor  wrote:
>> On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:
>> 
>>> Hi all. I'm sorry if this has been answered before, but I've searched and 
>>> cannot find a solution other than "add SAMtools to your PATH", which I 
>>> already have done. I can invoke samtools from BASH while logged in as the 
>>> "galaxy" user I set up according to the production installation guide. When 
>>> I try to add .bam files to a shared data library, I get the following 
>>> message:
>>> 
>>> Traceback (most recent call last):
>>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in
>>> __main__()
>>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in 
>>> __main__
>>> add_file( dataset, registry, json_file, output_path )
>>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in 
>>> add_file
>>> if link_data_only == 'copy_files' and 
>>> datatype.dataset_content_needs_grooming( output_path ):
>>> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in 
>>> dataset_content_needs_grooming
>>> version = self._get_samtools_version()
>>> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in 
>>> _get_samtools_version
>>> output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, 
>>> stdout=subprocess.PIPE ).communicate()[1]
>>> File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
>>> errread, errwrite)
>>> File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
>>> raise child_exception
>>> OSError: [Errno 2] No such file or directory
>>> 
>>> I've taken a peek at the "dataset_content_needs_grooming" function and it 
>>> looks like there is an OS call that executes "$ samtools" and splits the 
>>> resulting string to access the version number of samtools (why couldn't the 
>>> developers add a --version flag?). It seems to me like samtools cannot be 
>>> executed, but I can't think of any reason why. Invoking samtools as user 
>>> "galaxy" (with the user's associated PATH) gives:
>>> 
>>> Program: samtools (Tools for alignments in the SAM format)
>>> Version: 0.1.12a (r862)
>>> Usage: samtools  [options]
>>> Command: view SAM<->BAM conversion
>>> sort sort alignment file
>>> pileup generate pileup output
>>> mpileup multi-way pileup
>>> faidx index/extract FASTA
>>> tview text alignment viewer
>>> index index alignment
>>> idxstats BAM index stats (r595 or later)
>>> fixmate fix mate information
>>> glfview print GLFv3 file
>>> flagstat simple stats
>>> calmd recalculate MD/NM tags and '=' bases
>>> merge merge sorted alignments
>>> rmdup remove PCR duplicates
>>> reheader replace BAM header
>>> 
>>> Does anyone know where I've gone wrong?
>> 
>> Hi Matt,
>> 
>> Are you running tools on a cluster?  If running locally, are you starting 
>> Galaxy by hand (i.e. run.sh) from that same she

Re: [galaxy-dev] samtools version detection issue

[ambarish biswas]

Hi,
   I'm not sure if this is related or not but I'm also noticed a
similar problem where the program is not found in the path, but it
exists. I have it submitted as an issue , but it could be related here
so mentioned here. The problem started when I recently upgraded my
Galaxy installation(It was running in cluster with SGE, in Red hat
linux). Currently, I'm running Galaxy with drmaa in a cluster in Red
Hat linux, where the web server  is running in one Server, and the
execution is being performed in another. I noticed that programs like
ClustalW, dialign-tx etc, which uses a wrapper, is giving error as
follows:
sh: dialign-tx: command not found
The original command line in the wrapper is as follows:
command = "dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" %
(GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length,
length_low_score_pos, input_file_name, output_file)

Even though, the programs are present in the Execution node's path,
and if I add, explicit path, like "/usr/local/bin" in the command as
bellow:
command = "/usr/local/bin/dialign-tx %s/dialign-tx %s %s -g%s -n%s -D
%s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity,
max_frag_length, length_low_score_pos, input_file_name, output_file)

Then the programs works fine without any error.

I compared, the wrapper files with my old installation and the current
installation, and both are 100% same. So, I guess the path to the
executable is missing from the environment. I searched a lot, but
couldn't find any solution to rectify this issue.
Can someone please point out where I need to change to avoid adding
explicit path to each programs.
Thanks in advance.


With Regards,

Ambarish Biswas,
University of Otago
Department of Biochemistry,
Dunedin, New Zealand,


On Fri, Jan 20, 2012 at 11:07 AM, Nate Coraor  wrote:
> On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:
>
>> Hi all. I'm sorry if this has been answered before, but I've searched and 
>> cannot find a solution other than "add SAMtools to your PATH", which I 
>> already have done. I can invoke samtools from BASH while logged in as the 
>> "galaxy" user I set up according to the production installation guide. When 
>> I try to add .bam files to a shared data library, I get the following 
>> message:
>>
>> Traceback (most recent call last):
>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in
>> __main__()
>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in 
>> __main__
>> add_file( dataset, registry, json_file, output_path )
>> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in 
>> add_file
>> if link_data_only == 'copy_files' and 
>> datatype.dataset_content_needs_grooming( output_path ):
>> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in 
>> dataset_content_needs_grooming
>> version = self._get_samtools_version()
>> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in 
>> _get_samtools_version
>> output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, 
>> stdout=subprocess.PIPE ).communicate()[1]
>> File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
>> errread, errwrite)
>> File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
>> raise child_exception
>> OSError: [Errno 2] No such file or directory
>>
>> I've taken a peek at the "dataset_content_needs_grooming" function and it 
>> looks like there is an OS call that executes "$ samtools" and splits the 
>> resulting string to access the version number of samtools (why couldn't the 
>> developers add a --version flag?). It seems to me like samtools cannot be 
>> executed, but I can't think of any reason why. Invoking samtools as user 
>> "galaxy" (with the user's associated PATH) gives:
>>
>> Program: samtools (Tools for alignments in the SAM format)
>> Version: 0.1.12a (r862)
>> Usage: samtools  [options]
>> Command: view SAM<->BAM conversion
>> sort sort alignment file
>> pileup generate pileup output
>> mpileup multi-way pileup
>> faidx index/extract FASTA
>> tview text alignment viewer
>> index index alignment
>> idxstats BAM index stats (r595 or later)
>> fixmate fix mate information
>> glfview print GLFv3 file
>> flagstat simple stats
>> calmd recalculate MD/NM tags and '=' bases
>> merge merge sorted alignments
>> rmdup remove PCR duplicates
>> reheader replace BAM header
>>
>> Does anyone know where I've gone wrong?
>
> Hi Matt,
>
> Are you running tools on a cluster?  If running locally, are you starting 
> Galaxy by hand (i.e. run.sh) from that same shell in which you can run 
> samtools?
>
> --nate
>
>>
>> --
>> Matt Shirley
>> Ph.D Candidate - BCMB
>> Pevsner Lab
>> Johns Hopkins Medicine
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>>  http:

Re: [galaxy-dev] samtools version detection issue

[Nate Coraor]

On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:

> Hi all. I'm sorry if this has been answered before, but I've searched and 
> cannot find a solution other than "add SAMtools to your PATH", which I 
> already have done. I can invoke samtools from BASH while logged in as the 
> "galaxy" user I set up according to the production installation guide. When I 
> try to add .bam files to a shared data library, I get the following message:
> 
> Traceback (most recent call last):
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in
> __main__()
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in 
> __main__
> add_file( dataset, registry, json_file, output_path )
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in 
> add_file
> if link_data_only == 'copy_files' and 
> datatype.dataset_content_needs_grooming( output_path ):
> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in 
> dataset_content_needs_grooming
> version = self._get_samtools_version()
> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in 
> _get_samtools_version
> output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, 
> stdout=subprocess.PIPE ).communicate()[1]
> File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
> errread, errwrite)
> File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
> raise child_exception
> OSError: [Errno 2] No such file or directory
> 
> I've taken a peek at the "dataset_content_needs_grooming" function and it 
> looks like there is an OS call that executes "$ samtools" and splits the 
> resulting string to access the version number of samtools (why couldn't the 
> developers add a --version flag?). It seems to me like samtools cannot be 
> executed, but I can't think of any reason why. Invoking samtools as user 
> "galaxy" (with the user's associated PATH) gives:
> 
> Program: samtools (Tools for alignments in the SAM format)
> Version: 0.1.12a (r862)
> Usage: samtools  [options]
> Command: view SAM<->BAM conversion
> sort sort alignment file
> pileup generate pileup output
> mpileup multi-way pileup
> faidx index/extract FASTA
> tview text alignment viewer
> index index alignment
> idxstats BAM index stats (r595 or later)
> fixmate fix mate information
> glfview print GLFv3 file
> flagstat simple stats
> calmd recalculate MD/NM tags and '=' bases
> merge merge sorted alignments
> rmdup remove PCR duplicates
> reheader replace BAM header
> 
> Does anyone know where I've gone wrong?

Hi Matt,

Are you running tools on a cluster?  If running locally, are you starting 
Galaxy by hand (i.e. run.sh) from that same shell in which you can run samtools?

--nate

>  
> -- 
> Matt Shirley
> Ph.D Candidate - BCMB
> Pevsner Lab
> Johns Hopkins Medicine
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] samtools version detection issue

[Matt Shirley]

Hi all. I'm sorry if this has been answered before, but I've searched and
cannot find a solution other than "add SAMtools to your PATH", which I
already have done. I can invoke samtools from BASH while logged in as the
"galaxy" user I set up according to the production installation guide. When
I try to add .bam files to a shared data library, I get the following
message:

Traceback (most recent call last):
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in
> __main__()
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in
> __main__
> add_file( dataset, registry, json_file, output_path )
> File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in
> add_file
> if link_data_only == 'copy_files' and
> datatype.dataset_content_needs_grooming( output_path ):
> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79,
> in dataset_content_needs_grooming
> version = self._get_samtools_version()
> File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63,
> in _get_samtools_version
> output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
> stdout=subprocess.PIPE ).communicate()[1]
> File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
> errread, errwrite)
> File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
> raise child_exception
> OSError: [Errno 2] No such file or directory


I've taken a peek at the "dataset_content_needs_grooming" function and it
looks like there is an OS call that executes "$ samtools" and splits the
resulting string to access the version number of samtools (why couldn't the
developers add a --version flag?). It seems to me like samtools cannot be
executed, but I can't think of any reason why. Invoking samtools as user
"galaxy" (with the user's associated PATH) gives:

Program: samtools (Tools for alignments in the SAM format)
> Version: 0.1.12a (r862)
> Usage: samtools  [options]
> Command: view SAM<->BAM conversion
> sort sort alignment file
> pileup generate pileup output
> mpileup multi-way pileup
> faidx index/extract FASTA
> tview text alignment viewer
> index index alignment
> idxstats BAM index stats (r595 or later)
> fixmate fix mate information
> glfview print GLFv3 file
> flagstat simple stats
> calmd recalculate MD/NM tags and '=' bases
> merge merge sorted alignments
> rmdup remove PCR duplicates
> reheader replace BAM header


Does anyone know where I've gone wrong?

-- 
Matt Shirley
Ph.D Candidate - BCMB
Pevsner Lab 
Johns Hopkins Medicine
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/