[galaxy-dev] Is there a bowtie2 wrapper

[Oleksandr Moskalenko]

I wonder if someone has a functional bowtie2 wrapper that they can share.

Thanks,

Alex
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Remove libraries using Galaxy code or API

[Nate Coraor]

On Apr 23, 2012, at 4:55 AM,  
 wrote:

> Hello,
>  
> I am using Galaxy API for some actions and I must say that this is indeed a 
> really great feature with a great power.
> Anyway, I am trying to write a python script that one of its goals is to 
> remove some data libraries,
> But until now, I was unable to find a way to remove data library or some of 
> its datasets using the API or by direct call to Galaxy’s code.
> I found a old post that claim that this feature is not yet implemented.
> My questions:
> 1)  Is this has changed since? I mean, is there a way now to clean or 
> remove completely a data library?
> 2)  Is there a way to use Galaxy code to remove a library?  Such as a 
> function that can be used in my script to remove this library?

Hi Liram,

I've just implemented library deletion in changeset 1640cbaafd09.

--nate

>  
> Thanks in advance!
> Liram
>  
>  
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] installing tool from ToolShed - Python Error 'too many values to unpack'

[Shantanu Pavgi]

That's right Greg, I was using the latest revision from the galaxy-dist and not 
the galaxy-central. It worked fine after using the latest code on from the 
galaxy-central. Thanks for the help. 

--
Shantanu

On Apr 30, 2012, at 1:54 PM, Greg Von Kuster wrote:

> Hi Shantanu,
> 
> Your Galaxy instance is not current with the Galaxy central repository - are 
> you tracking Galaxy dist?  In order to use the features that provide 
> communication between Galaxy and a tool shed, you'll have use the latest code 
> from Galaxy central - see https://bitbucket.org/galaxy/galaxy-central
> 
> Greg Von Kuster
> 
> 
> On Apr 30, 2012, at 2:46 PM, Shantanu Pavgi wrote:
> 
>> 
>> I am getting following "ValueError: too many values to unpack" from Python 
>> while trying to install a tool from toll_shed repository. 
>> 
>> {{{
>> 
>> URL: 
>> http://localhost:8080/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F&repo_info_dict=9e40c73c79ff0c9ab662043369229ec1cd73bf7c%3A7b2262616d5f746f5f6661737471223a205b22436f6e766572742042414d2066696c6520746f206661737471222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f627261642d636861706d616e2f62616d5f746f5f6661737471222c2022356139616461396133313931222c202230225d7d&includes_tools=True
>> File 
>> '/Users/shantanu/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py',
>>  line 364 in respond
>> app_iter = self.application(environ, detect_start_response)
>> File 
>> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py',
>>  line 98 in __call__
>> environ, self.app)
>> File 
>> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', 
>> line 539 in intercept_output
>> app_iter = application(environ, replacement_start_response)
>> File 
>> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', 
>> line 80 in __call__
>> return self.application(environ, start_response)
>> File 
>> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py',
>>  line 632 in __call__
>> return self.application(environ, start_response)
>> File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/base.py', line 
>> 160 in __call__
>> body = method( trans, **kwargs )
>> File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/__init__.py', 
>> line 173 in decorator
>> return func( self, trans, *args, **kwargs )
>> File 
>> '/Users/shantanu/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', 
>> line 291 in install_repository
>> description, repository_clone_url, changeset_revision = repo_info_tuple
>> ValueError: too many values to unpack
>> 
>> }}}
>> 
>> I am using galaxy-dist revision 40f1816d6857 on Mac OS 10.7 with Python 2.7. 
>>  I got the same error on CentOS 5.8 platform with Python 2.6 version for the 
>> same galaxy-dist revision. Am I missing anything in the configuration file? 
>> Any pointers? Appreciate all the help.  
>> 
>> --
>> Thanks,
>> Shantanu
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> 
>> http://lists.bx.psu.edu/
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Cutadapt

[Jennifer Jackson]

Hi Tilahun,

This tool is in the Galaxy Tool Shed 
(http://wiki.g2.bx.psu.edu/Tool%20Shed) so to use for now, use with a 
local/cloud instance (http://getgalaxy.org) would be the best route:

http://galaxy.psu.edu/search/getgalaxy/
search for "cutadapt"

I will also forward this to the galaxy-dev list where tools are 
discussed. Going forward, that would be the best place to make requests. 
We will open a ticket at bitbucket if the team decides this would be a 
good fit for the public server.

http://wiki.g2.bx.psu.edu/Support#Mailing_Lists

Thanks!

Jen
Galaxy team

 Original Message 
Subject:Cutadapt
Date:   Mon, 30 Apr 2012 13:49:53 -0500
From:   Tilahun Abebe 
To: Jennifer Jackson 



Hi Jennifer,

I want to use cutadapt to trim adapters on Galaxy main. It looks like it
is not listed under the tools. Can you add cutadapt to Galaxy?

Thanks for your help.

Tilahun Abebe
University of Northern Iowa


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] installing tool from ToolShed - Python Error 'too many values to unpack'

[Greg Von Kuster]

Hi Shantanu,

Your Galaxy instance is not current with the Galaxy central repository - are 
you tracking Galaxy dist?  In order to use the features that provide 
communication between Galaxy and a tool shed, you'll have use the latest code 
from Galaxy central - see https://bitbucket.org/galaxy/galaxy-central

Greg Von Kuster


On Apr 30, 2012, at 2:46 PM, Shantanu Pavgi wrote:

> 
> I am getting following "ValueError: too many values to unpack" from Python 
> while trying to install a tool from toll_shed repository. 
> 
> {{{
> 
> URL: 
> http://localhost:8080/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F&repo_info_dict=9e40c73c79ff0c9ab662043369229ec1cd73bf7c%3A7b2262616d5f746f5f6661737471223a205b22436f6e766572742042414d2066696c6520746f206661737471222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f627261642d636861706d616e2f62616d5f746f5f6661737471222c2022356139616461396133313931222c202230225d7d&includes_tools=True
> File 
> '/Users/shantanu/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py',
>  line 364 in respond
>  app_iter = self.application(environ, detect_start_response)
> File 
> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', 
> line 98 in __call__
>  environ, self.app)
> File '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', 
> line 539 in intercept_output
>  app_iter = application(environ, replacement_start_response)
> File 
> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', 
> line 80 in __call__
>  return self.application(environ, start_response)
> File 
> '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py',
>  line 632 in __call__
>  return self.application(environ, start_response)
> File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 
> in __call__
>  body = method( trans, **kwargs )
> File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 
> 173 in decorator
>  return func( self, trans, *args, **kwargs )
> File 
> '/Users/shantanu/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', 
> line 291 in install_repository
>  description, repository_clone_url, changeset_revision = repo_info_tuple
> ValueError: too many values to unpack
> 
> }}}
> 
> I am using galaxy-dist revision 40f1816d6857 on Mac OS 10.7 with Python 2.7.  
> I got the same error on CentOS 5.8 platform with Python 2.6 version for the 
> same galaxy-dist revision. Am I missing anything in the configuration file? 
> Any pointers? Appreciate all the help.  
> 
> --
> Thanks,
> Shantanu
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] installing tool from ToolShed - Python Error 'too many values to unpack'

[Shantanu Pavgi]

I am getting following "ValueError: too many values to unpack" from Python 
while trying to install a tool from toll_shed repository. 

{{{

URL: 
http://localhost:8080/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F&repo_info_dict=9e40c73c79ff0c9ab662043369229ec1cd73bf7c%3A7b2262616d5f746f5f6661737471223a205b22436f6e766572742042414d2066696c6520746f206661737471222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f627261642d636861706d616e2f62616d5f746f5f6661737471222c2022356139616461396133313931222c202230225d7d&includes_tools=True
File 
'/Users/shantanu/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py',
 line 364 in respond
  app_iter = self.application(environ, detect_start_response)
File 
'/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', 
line 98 in __call__
  environ, self.app)
File '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', 
line 539 in intercept_output
  app_iter = application(environ, replacement_start_response)
File '/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', 
line 80 in __call__
  return self.application(environ, start_response)
File 
'/Users/shantanu/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', 
line 632 in __call__
  return self.application(environ, start_response)
File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 
in __call__
  body = method( trans, **kwargs )
File '/Users/shantanu/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 
173 in decorator
  return func( self, trans, *args, **kwargs )
File 
'/Users/shantanu/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', 
line 291 in install_repository
  description, repository_clone_url, changeset_revision = repo_info_tuple
ValueError: too many values to unpack

}}}

I am using galaxy-dist revision 40f1816d6857 on Mac OS 10.7 with Python 2.7.  I 
got the same error on CentOS 5.8 platform with Python 2.6 version for the same 
galaxy-dist revision. Am I missing anything in the configuration file? Any 
pointers? Appreciate all the help.  

--
Thanks,
Shantanu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy parallel processing

[Nate Coraor]

Hi Kiran,

This is a question best directed to the galaxy-dev list, which I have CC'd.

On Apr 23, 2012, at 5:23 AM, Kiran Jaycee wrote:

> Dear Nate,
> 
> Im deploying Galaxy in the cluster environment with 8 nodes and 96 cores. Im 
> particularly interested in running NGS tools
> like BWA, BOWTIE etc in the cluster with Torque PBS. Just wondering if galaxy 
> is capable of parallel processing of data. 1 user running BWA in 10 cores in 
> which 10 instances of BWA in each core parallely processing the mapping of 
> 250 GB of reads data on to the reference genome.
> Is this possible?

The BWA tool itself supports running multithreaded, see the threads option in 
the wrapper.  Also, Galaxy can split a tool's inputs, run a tool many times, 
and merge the outputs.  I am not sure whether this is possible or implemented 
with BWA, but I would encourage you to look in to whether this would work for 
your needs.

> I have another issue as well. Im unable to finish FASTQ grooming of the reads 
> of size 250GB even after 2 days of processing. Is there a way to circumvent 
> the grooming step? or any way to make the grooming speed up?

Assuming this is on a local server, make sure you have set the config option 
set_metadata_externally = True.  You don't need to run the groomer if your 
fastq file is valid fastqsanger.

--nate

> 
> Your quick response will be really appreciated.
> 
> Thank you
> 
> Regards,
> Kiran Jaycee
> 
> Bioinformatician, BINET
> ELOGIC Technologies Pvt. Ltd.
> #1, 2nd Floor, 100 Ft. Ring Road, 
> Kathriguppe, Banashankari III Stage, 
> Bangalore - 560085 
> Phone   : +91 (0)80 4080 3829 
> Website : www.elogic.co.in
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] /bin/sh: samtools: not found-->>WORKAROUND

[Michael Moore]

Yeah, I was using ['printenv| mail  && samtools'] inside the
subprocess.Popen(['samtools'],... in upload.py.

Now right before in upload.py before the subprocess was called, I had
/usr/bin/ in my PATH, but inside the subprocess the story was different.  I
just used the symbolic link to access samtools from something that remained
in the PATH.  I have no idea how this could happen, but I have noticed that
galaxy does things with input and output and seems to manipulate the
environment heavily--but why should a child process have a different
environment when none was invoked?  I'll figure that out later.  I am still
trying to convert some software to run with galaxy.

On Mon, Apr 30, 2012 at 8:57 AM, Nate Coraor  wrote:

> On Apr 24, 2012, at 8:36 PM, Michael Moore wrote:
>
> > There is apparently a persistent problem with samtools which normally
> lives at /usr/bin/samtools.  I encountered a similar problem in Python when
> uploading BAM files.
> >
> > I did not resolve the problem.  I hacked for a while on binary.py in a
> lib/ subdirectory and used os.system to send myself mail describing the
> effective path at various points, and I added a missing
> >
> > logging.basicConfig()
> >
> > statement and scattered some log.WARNING statements strategically.  All
> this told me nothing.  So I made a few symlinks to samtools.  The one that
> got things working was
> >
> > ln -s /usr/bin/samtools /home/galaxy/bin/samtools
> >
> > so--worked around but not resolved.
>
> Hi Michael,
>
> For tools that output BAM, samtools needs to be in your $PATH, or has to
> be set up via the tool dependencies system.  See the following for details:
>
>http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies
>
> For SGE, you can modify the $PATH used on the cluster in ~/.sge_request or
> the file specified in the 'environment_setup_file' galaxy config option.
>
> --nate
>
> >
> > Michael
> >
> > On Tue, Apr 17, 2012 at 12:15 PM, zhengqiu cai 
> wrote:
> > Hi All,
> >
> > I submitted a job to convert sam to bam, and the job was running forever
> without outputing the result. I then checked the log, and it read:
> > Traceback (most recent call last):
> >  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
> line 336, in finish_job
> >drm_job_state.job_wrapper.finish( stdout, stderr )
> >  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/__init__.py", line
> 637, in finish
> >dataset.set_meta( overwrite = False )
> >  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/model/__init__.py", line
> 875, in set_meta
> >return self.datatype.set_meta( self, **kwd )
> >  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/datatypes/binary.py",
> line 179, in set_meta
> >raise Exception, "Error Setting BAM Metadata: %s" % stderr
> > Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found
> >
> > It means that the samtools is not in the PATH. I tried to set the PATH
> in a couple of methods according the Galaxy documentation:
> > 1. put the path in the env.sh in the tool directory and symbolink
> default to the tool directory, e.g. default ->
> =/mnt/galaxyTools/tools/samtools/0.1.18
> > 2. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in ~/.sge_request
> > 3. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in
> /path/sge_request
> >
> > none of them worked, and I got the above same problem.
> >
> > Then I checked the job log file in the job_working_directory, and it
> read:
> > Samtools Version: 0.1.18 (r982:295)
> > SAM file converted to BAM
> >
> > which shows that sge knows the PATH of samtools. To double check it, I
> added samtools index to Galaxy, and it worked well. I am very confused why
> SGE knows the tool path but cannot run the job correctly.
> >
> > The system I am using is ubuntu on EC2. I checked out the code from
> galaxy-dist on bitbucket. Other tools such as bwa and bowtie worked well
> using the same setting method(put env.sh in the tools directory to set the
> tool path)
> >
> > Thank you very much for any help or hints.
> >
> > Cai
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >  http://lists.bx.psu.edu/
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >  http://lists.bx.psu.edu/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Configuring Galaxy for FTP upload

[Nate Coraor]

On Apr 23, 2012, at 10:31 PM, Ciara Ledero wrote:

> Hi all,
> 
> According to Galaxy, uploading files larger than 2GB will surely fail. For 
> this reason, uploading via FTP is recommended. Unfortunately, I cannot 
> download/install an FTP server for Galaxy since I am working on a remote 
> server. I only have access to it via Xming. Is there anyway to allow FTP 
> upload, without using ProFTPd? Can I use Mozilla's fireFTP or Linux's own 
> FTP? (Actually, I tried to connect to Galaxy, but I was denied access.).
> 
> Thank you in advance for any help.

Hi Ciara,

ProFTPd is not required.  If your Galaxy users already have a way to access the 
server directly, that access method is fine.  You simply need to create a 
top-level directory under which you create directories for each user.  Then, 
set the top level directory as the value of 'ftp_upload_dir' in the config 
file, and set 'ftp_upload_site' to something other than the default 'None'.

e.g.:

/scratch/galaxy/upload
/scratch/galaxy/upload/us...@example.org
/scratch/galaxy/upload/us...@example.org

ftp_upload_dir = /scratch/galaxy/upload
ftp_upload_site = server.example.org

--nate

> 
> CL
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] /bin/sh: samtools: not found-->>WORKAROUND

[Nate Coraor]

On Apr 24, 2012, at 8:36 PM, Michael Moore wrote:

> There is apparently a persistent problem with samtools which normally lives 
> at /usr/bin/samtools.  I encountered a similar problem in Python when 
> uploading BAM files.
> 
> I did not resolve the problem.  I hacked for a while on binary.py in a lib/ 
> subdirectory and used os.system to send myself mail describing the effective 
> path at various points, and I added a missing 
> 
> logging.basicConfig() 
> 
> statement and scattered some log.WARNING statements strategically.  All this 
> told me nothing.  So I made a few symlinks to samtools.  The one that got 
> things working was
> 
> ln -s /usr/bin/samtools /home/galaxy/bin/samtools
> 
> so--worked around but not resolved.

Hi Michael,

For tools that output BAM, samtools needs to be in your $PATH, or has to be set 
up via the tool dependencies system.  See the following for details:

http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies

For SGE, you can modify the $PATH used on the cluster in ~/.sge_request or the 
file specified in the 'environment_setup_file' galaxy config option.

--nate

> 
> Michael
> 
> On Tue, Apr 17, 2012 at 12:15 PM, zhengqiu cai  
> wrote:
> Hi All,
> 
> I submitted a job to convert sam to bam, and the job was running forever 
> without outputing the result. I then checked the log, and it read:
> Traceback (most recent call last):
>  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 
> 336, in finish_job
>drm_job_state.job_wrapper.finish( stdout, stderr )
>  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/__init__.py", line 637, 
> in finish
>dataset.set_meta( overwrite = False )
>  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/model/__init__.py", line 875, 
> in set_meta
>return self.datatype.set_meta( self, **kwd )
>  File "/mnt/galaxyTools/galaxy-dist/lib/galaxy/datatypes/binary.py", line 
> 179, in set_meta
>raise Exception, "Error Setting BAM Metadata: %s" % stderr
> Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found
> 
> It means that the samtools is not in the PATH. I tried to set the PATH in a 
> couple of methods according the Galaxy documentation:
> 1. put the path in the env.sh in the tool directory and symbolink default to 
> the tool directory, e.g. default -> =/mnt/galaxyTools/tools/samtools/0.1.18
> 2. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in ~/.sge_request
> 3. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in /path/sge_request
> 
> none of them worked, and I got the above same problem.
> 
> Then I checked the job log file in the job_working_directory, and it read:
> Samtools Version: 0.1.18 (r982:295)
> SAM file converted to BAM
> 
> which shows that sge knows the PATH of samtools. To double check it, I added 
> samtools index to Galaxy, and it worked well. I am very confused why SGE 
> knows the tool path but cannot run the job correctly.
> 
> The system I am using is ubuntu on EC2. I checked out the code from 
> galaxy-dist on bitbucket. Other tools such as bwa and bowtie worked well 
> using the same setting method(put env.sh in the tools directory to set the 
> tool path)
> 
> Thank you very much for any help or hints.
> 
> Cai
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Error: Job output not returned from cluster

[Nate Coraor]

On Apr 25, 2012, at 8:25 AM, Louise-Amélie Schmitt wrote:

> Hi,
> 
> Thanks a lot it actually helped. It is not exactly as straightforward in 
> drmaa.py but somehow I could manage.
> 
> However, it was not the problem. For some reason, the user needs to write 
> files from the node to job_working_directory/00X// and the latter is not 
> world-writable. I had to make everyone chmod everything to 777 to make it 
> work. Did I miss something?

That doesn't seem right.  The subdirectory for your job underneath 
job_working_directory/00X/XXX/ should be chowned to the real user before the 
job is submitted and then chowned back to the galaxy user once the job is 
complete.

--nate

> 
> Best,
> L-A
> 
> 
> 
> Le 24/04/2012 15:17, Alban Lermine a écrit :
>> Hi L-A,
>> 
>> I run Galaxy as real user on our cluster with pbs (free version).
>> 
>> We first configure LDAP authentification for having email account
>> related to unix account (just cut the @curie.fr)
>> Then I have modify pbs.py (in
>> galaxy-dist/lib/galaxy/jobs/runners)
>> 
>> I have just disconnected the pbs submission through python library and
>> replace it by a system call (just like I send jobs to the cluster with
>> command line), here is the code used:
>> 
>>  galaxy_job_id = job_wrapper.job_id
>>  log.debug("(%s) submitting file %s" % ( galaxy_job_id, job_file ) )
>>  log.debug("(%s) command is: %s" % ( galaxy_job_id, command_line ) )
>> 
>> # Submit job with system call instead of using python PBS library -
>> Permit to run jobs as .. with sudo -u cmd prefix
>> 
>> galaxy_job_idSTR = str(job_wrapper.job_id)
>> galaxy_tool_idSTR = str(job_wrapper.tool.id)
>> galaxy_job_name =
>> galaxy_job_idSTR+"_"+galaxy_tool_idSTR+"_"+job_wrapper.user
>> torque_options = runner_url.split("/")
>> queue = torque_options[3]
>> ressources = torque_options[4]
>> user_mail = job_wrapper.user.split("@")
>> username = user_mail[0]
>> 
>> torque_cmd = "sudo -u username echo "+"\""+command_line+"\" | qsub
>> -o "+ofile+" -e "+efile+" -M "+job_wrapper.user+" -N "+galaxy_job_name+"
>> -q "+queue+" "+ressources
>> 
>> submit_pbs_job = os.popen(torque_cmd)
>> 
>> job_id = submit_pbs_job.read().rstrip("\n")
>> 
>> #Original job launcher
>> #job_id = pbs.pbs_submit(c, job_attrs, job_file, pbs_queue_name, None)
>> 
>> pbs.pbs_disconnect(c)
>> 
>> Second thing I have done is to wait error and output file from torque in
>> the finish_job function (if not, I never receive the output, seems to be
>> your problem..), here is the code used:
>> 
>> def finish_job( self, pbs_job_state ):
>> """
>> Get the output/error for a finished job, pass to
>> `job_wrapper.finish`
>> and cleanup all the PBS temporary files.
>> """
>> ofile = pbs_job_state.ofile
>> efile = pbs_job_state.efile
>> job_file = pbs_job_state.job_file
>> 
>> # collect the output
>> try:
>> 
>> # With qsub system call, need to wait efile and ofile creation
>> at the end of the job execution before reading them
>> 
>> efileExists = os.path.isfile(efile)
>> ofileExists = os.path.isfile(ofile)
>> efileExistsSTR = str(efileExists)
>> ofileExistsSTR = str(ofileExists)
>> 
>> while efileExistsSTR == "False":
>> time.sleep( 1 )
>> efileExists = os.path.isfile(efile)
>> efileExistsSTR = str(efileExists)
>> 
>> while ofileExistsSTR == "False":
>> time.sleep( 1 )
>> ofileExists = os.path.isfile(ofile)
>> ofileExistsSTR = str(ofileExists)
>> 
>> # Back to original code
>> 
>> ofh = file(ofile, "r")
>> efh = file(efile, "r")
>> stdout = ofh.read( 32768 )
>> stderr = efh.read( 32768 )
>> except:
>> stdout = ''
>> stderr = 'Job output not returned by PBS: the output
>> datasets were deleted while the job was running, the job was manually
>> dequeued or there was a cluster error.'
>> log.debug(stderr)
>> 
>> * Last step is to allow galaxy user to run sudo
>> 
>> 
>> Hope it can help you finding your problem..
>> 
>> See you,
>> 
>> Alban
>> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
> http://lists.bx.psu.edu/
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Running jobs as real user and extra_file_path

[Nate Coraor]

On Apr 25, 2012, at 9:39 AM, Louise-Amélie Schmitt wrote:

> Hi everyone,
> 
> I just wanted to ask how the extra_file_path is handled in case of job 
> running as the real user since the file_path is only writable by the galaxy 
> user. Any clue?

Hi L-A,

There are actually two dataset attributes for accessing the extra files 
directory, 'files_path' and 'extra_files_path'.

'extra_files_path' always points to the real path under the directory specified 
in the config file's 'file_path' option and should be used when providing a 
dataset's extra files directory as the input to a tool.

'files_path' points at the location that should be written to when used in a 
job so that the job's finish method can find the output files.  This will be 
under the job_working_directory and thus writable by the actual user.  So any 
tool outputs that use the extra files directory should use the output's 
'files_path' attribute.

Sorry for this confusion.  There was talk some time ago about merging both 
attributes to work correctly in all cases, but this was never done.

--nate

> 
> Thanks,
> L-A
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
> http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Error running job as real user: looking et the wrong dir?

[Nate Coraor]

On Apr 25, 2012, at 9:22 AM, Louise-Amélie Schmitt wrote:

> Hi everyone,
> 
> I'm (still) having issues with running jobs as the real user on our PBS Pro 
> cluster. When I try running a job, it ends up in error state and displays 
> the following error message:
> 
> tou!
>  ch: cannot
>  touch `/home/galaxy/.drmaa/9167860.pbs-master2.embl.de.started': Permission 
> denied
> /g/funcgen/galaxy-dev/database/pbs/galaxy_1516.sh: line 15: syntax error: 
> unexpected end of file
> /var/spool/PBS/mom_priv/jobs/9167860.pbs-master2.embl.de.SC: line 1: 
> /home/galaxy/.drmaa/9167860.pbs-master2.embl.de.exitcode: Permission denied
> 
> 
> I suspect this is due to some code executed as the real user (since the 
> permission is denied) but then why looking at /home/galaxy instead of 
> /home/realuser ?
> 
> If you have any idea about what's going on, please share, I'll be forever 
> grateful!

Hi L-A,

This path appears to be part of pbs-drmaa:

http://apps.man.poznan.pl/trac/pbs-drmaa

Namely, the user_state_dir option.  You may need to write out a pbs_drmaa.conf 
for each job or user and then set $PBS_DRMAA_CONF to point to that user's 
config file before initializing the session in scripts/drmaa_external_runner.py

You may also be able to get away with making the user_state_dir world writable, 
although this comes with security implications.

--nate

> 
> Thanks,
> L-A
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other

[Dorset, Daniel C]

Hi Ann,

Count me in. Great idea!

-Dan



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Stalled upload jobs under "Admin", "Manage jobs"

[Nate Coraor]

On Apr 26, 2012, at 11:03 AM, Hans-Rudolf Hotz wrote:

> 
> 
> On 04/26/2012 04:47 PM, Peter Cock wrote:
>> On Fri, Mar 16, 2012 at 11:00 AM, Peter Cock  
>> wrote:
>>> On Mon, Feb 13, 2012 at 5:02 PM, Nate Coraor  wrote:
 On Feb 10, 2012, at 6:47 AM, Peter Cock wrote:
 
> Hello all,
> 
> I've noticed we have about a dozen stalled upload jobs on our server
> from several users. e.g.
> 
> Job IDUserLast Update ToolState   Command LineJob 
> Runner  PID/Cluster ID
> 2352  21 hours agoupload1 upload  NoneNoneNone
> ...
> 2339  19 hours agoupload1 upload  NoneNoneNone
> 
> The job numbers are consecutive (2339 to 2352) and reflect a problem
> for a couple of hours yesterday morning. I believe this was due to the
> underlying file system being unmounted (without restarting Galaxy),
> and at the time restarting Galaxy fixed uploading files. Test jobs
> since then have completed normally - but these zombie jobs remain.
> 
> Using the "Stop jobs" option does not clear these dead upload jobs.
> 
> Restarting the Galaxy server does not clear them either.
> 
> This is our production server and was running galaxy-dist, changeset
> 5743:720455407d1c - which I have now updated to the current release,
> 6621:26920e20157f - which makes no difference to these stalled jobs.
> 
> Does anyone have any insight into what might be wrong, and how to get
> rid of these zombie tasks?
 
 Hi Peter,
 
 Are you using the nginx upload module?
 
 There's no way to fix these from within Galaxy, unfortunately.
 You'll have to update them in the database.
 
 --nate
>>> 
>>> Hi Nate,
>>> 
>>> Sorry for the delay - I must have missed your reply.
>>> 
>>> No, we're not using nginx here.
>>> 
>>> What should I edit in the database? Presumably rather than deleting
>>> these jobs I should set the state to finished with error?
>>> 
>>> (Is there any documentation about the Galaxy database schema,
>>> and the values of fields in it - or is that all considered to be an
>>> internal detail?)
>> 
>> Sorry to nag - my zombie jobs are still there and I'd like a little
>> guidance about how to delete them (e.g. which tables and what
>> status should I change them to).
>> 
> 
> Hi Peter
> 
> We had one such job which kept showing up in the "Admin/Manage jobs" page.
> 
> Have a look a the 'job' table. The 'state' is probably: "upload" for yours.
> 
> change the state to "error"  - this was the solution for us.
> 
> As always, be very careful when you directly access the MySQL or PostgreSQL 
> database.

Thanks Hans,

Indeed, it'd be:

update job set state='error' where state='upload';

But be aware that this would take out any current uploads (if there are any), 
so it may be useful to limit it to jobs older than a certain date or id, e.g.:

update job set state='error' where state='upload' and id<123456;

--nate

> 
> 
> Hope this helps
> Regards, Hans
> 
> 
> 
> 
> 
>> Thanks,
>> 
>> Peter
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> 
>>   http://lists.bx.psu.edu/
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
> http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?

[Ann Black-Ziegelbein]

I would like to thank everyone for their responses!  I am working on a 
few things to hopefully get us going in the right direction:


1) a brief online web survey for everyone to fill out that will help us 
gauge the background of those interested in participation and help steer 
content for the first call.  For example, what stage your local Galaxy 
deployment is in ( running for several years with 20+ users, no running 
local Galaxy but interested in hosting one, currently installing and 
configuring, etc).  How many people are you targeting using Galaxy?  
What problem areas have you hit? What customizations have you done? What 
areas are you interested in learning more about in these calls? etc.)   
The results of the survey will be published out to the group.


2) I am working on getting a teleconference with international numbers 
arranged.  I like the idea of having regional groups and maybe 
summaries/any presentation slides can be posted to do cross regional 
sharing.  At the same time - anyone interested (from any geographic 
region) is welcome to join - time zones may not be convenient, however, 
but we will do our best.


3) I will send out a poll to see if we can find the *best* time for our 
first call.  Yes, this will be difficult - but we will pick a time that 
works for the majority.


Stay tuned and looking forward to our first call.  Dave, as always, 
thanks for your help - I will be contacting you shortly.


Ann

On 4/30/2012 9:39 AM, Ryan Golhar wrote:
Agreed.  User driven is good.  Having developer(s) available to answer 
questions would be helpful...that's what I was alluding to...


On Sun, Apr 29, 2012 at 10:37 PM, Dave Clements 
mailto:cleme...@galaxyproject.org>> wrote:


Hi Ryan, Ann, and everyone else

I second what Nate says (I always do :-).  I too like that it is
user driven.  And, while I am not a developer, I do plan on being
on the call as often as possible.

I can also offer my support for logistical and any other support. 
Ann, please let me know if you want help with getting this going

and getting the word out.


On a related note,   With Matloob's email, we have at least 3
organizations in Australia and New Zealand that are interested. I
know that there are many more Galaxy installations in that part of
the world (and you have Enis and Ross in Australia now).  Would
this be a good opportunity to set up the first (to my knowledge)
regional user group?  If anyone thinks this is a good idea, please
respond (and create a new thread).


Finally, Ann, thanks for starting this.

Dave C



I also  that both Ross and Enis are in Australia now



On Sun, Apr 29, 2012 at 10:24 AM, Nate Coraor mailto:n...@bx.psu.edu>> wrote:

Hi Ryan,

I like that it's user-directed, and we could be there to
provide input, although if help is needed to organize the call
we could probably assist with that.  I plan to be on the call
as often as my schedule allows, and I believe some of the
other developers on the team would also be interested.

--nate

On Apr 28, 2012, at 8:49 AM, Ryan Golhar wrote:

> One question - Are the Galaxy developers involved in this or
is this for user's only?  It may be helpful to have developers
on the call to provide information that users do not
necessarily have.
>

-- 
http://galaxyproject.org/GCC2012


http://galaxyproject.org/
http://getgalaxy.org/
http://usegalaxy.org/
http://galaxyproject.org/wiki/




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?

[Ryan Golhar]

Agreed.  User driven is good.  Having developer(s) available to answer
questions would be helpful...that's what I was alluding to...

On Sun, Apr 29, 2012 at 10:37 PM, Dave Clements
wrote:

> Hi Ryan, Ann, and everyone else
>
> I second what Nate says (I always do :-).  I too like that it is user
> driven.  And, while I am not a developer, I do plan on being on the call as
> often as possible.
>
> I can also offer my support for logistical and any other support.  Ann,
> please let me know if you want help with getting this going and getting the
> word out.
>
>
> On a related note,   With Matloob's email, we have at least 3
> organizations in Australia and New Zealand that are interested. I know that
> there are many more Galaxy installations in that part of the world (and you
> have Enis and Ross in Australia now).  Would this be a good opportunity to
> set up the first (to my knowledge) regional user group?  If anyone thinks
> this is a good idea, please respond (and create a new thread).
>
>
> Finally, Ann, thanks for starting this.
>
> Dave C
>
>
>
> I also  that both Ross and Enis are in Australia now
>
>
>
> On Sun, Apr 29, 2012 at 10:24 AM, Nate Coraor  wrote:
>
>> Hi Ryan,
>>
>> I like that it's user-directed, and we could be there to provide input,
>> although if help is needed to organize the call we could probably assist
>> with that.  I plan to be on the call as often as my schedule allows, and I
>> believe some of the other developers on the team would also be interested.
>>
>> --nate
>>
>> On Apr 28, 2012, at 8:49 AM, Ryan Golhar wrote:
>>
>> > One question - Are the Galaxy developers involved in this or is this
>> for user's only?  It may be helpful to have developers on the call to
>> provide information that users do not necessarily have.
>> >
>>
> --
> http://galaxyproject.org/GCC2012 
> http://galaxyproject.org/
> http://getgalaxy.org/
> http://usegalaxy.org/
> http://galaxyproject.org/wiki/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Using REMOTE_USER with nginx

[Nate Coraor]

On Apr 29, 2012, at 1:50 AM, Peter van Heusden wrote:

> Hey there
> 
> The instructions on using REMOTE_USER with nginx are still a bit vague
> in the wiki, so let me share how I got this working with nginx's
> http_auth_pam module and our local Kerberos setup. Really simple actually:
> 
> First, I created a pam.d entry for nginx, as follows:
> 
> auth[success=1 default=ignore]pam_krb5.so minimum_uid=1000
> ignore_k5login
> authrequisitepam_deny.so
> authrequiredpam_permit.so
> 
> That can of course be adapted for your authentication scheme of choice.
> 
> The, after recompiling nginx to add the module (I actually used the
> source from the Ubuntu .deb and installed from this customised .deb), I
> added:
> 
>auth_pam "SANBI Galaxy (dev)";
>auth_pam_service_name "nginx";
>proxy_set_header REMOTE_USER $remote_user;
> 
> That auth_pam_service_name must be the name of the file you add in
> /etc/pam.d. So the complete location clause is now:
> 
>location / {
>auth_pam "SANBI Galaxy (dev)";
>auth_pam_service_name "nginx";
>proxy_set_header REMOTE_USER $remote_user;
>proxy_pass http://galaxy_app;
>proxy_set_header X-Forwarded-Host $host;
>proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>proxy_set_header X-URL-SCHEME https;
>}
> 
> Finally, set:
> 
> use_remote_user = True
> remote_user_maildomain = 
> 
> And restart nginx and galaxy, and you're done. Of course, since you're
> using Basic authentication, you should make sure that you are using ssl too.
> 
> If this all looks ok, maybe someone can update the wiki?

Hi Peter,

I've updated the wiki with your findings.

Thanks!
--nate

> 
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?

[Zhibin Lu]

Hi, 

We have two galaxy instances installed within our institute, and I am very 
interested in communicating with other galaxy users. Please count me in.

Zhibin Lu
Bioinformatics Support: Databases & Applications

Ontario Institute for Cancer Research   
MaRS Centre, South Tower
101 College Street, Suite 800
Toronto, Ontario, Canada M5G 0A3
 
Tel: 647-260-7944
Toll-free: 1-866-678-6427
www.oicr.on.ca


> Date: Fri, 27 Apr 2012 13:45:16 -0500
> From: annbl...@eng.uiowa.edu
> To: galaxy-dev@lists.bx.psu.edu
> Subject: [galaxy-dev] Interested in speaking with other institutions  
> deploying Galaxy locally?
> 
> Hi everyone -
> 
> Here at the University of Iowa we are working on deploying Galaxy 
> locally for campus wide access.  I am interested in forming a community 
> of other institutions trying to deploy Galaxy locally and mange/operate 
> it on a broad level.  Is anyone else?   If there is enough interest, 
> possibly we could have  a community conference call every other month to 
> have an open discussion on how we are all deploying galaxy, 
> customizations we are making, problems we are encountering, bugs, and 
> any add-on operations management for galaxy being developed, etc.
> 
> Would love to hear from others operating Galaxy or in process of 
> standing up a local deployment.
> 
> Thanks!
> 
> Ann Black-Ziegelbein
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
  ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Unable to set BAM Metadata

[Nate Coraor]

On Apr 29, 2012, at 9:19 PM, Ciara Ledero wrote:

> This is my code for running a samtools command internally, via Galaxy:
> 
> 
> open INP,"< $ARGV[0]"
>or die "Cannot open file: $!";
> 
> $file = "$ARGV[1]";
> 
> open OUT, "> $file"
>or die "Cannot open file: $!";
> 
> @out = `/home/applications/samtools-0.1.7a/samtools view -bS $ARGV[0] 2>&1`;
> 
> print OUT @out;
> 
> close INP;
> close OUT;
> 
> The 2>&1 is used to redirect the STDERR to STDOUT. When I run this in the 
> console, it gives a non-empty file. But when run it in Galaxy, Galaxy can no 
> longer find the path. This error message is given to me.
> 
> Traceback (most recent call last):
>  File "/home/applications/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 
> 126, in run_job
>job_wrapper.finish( stdout, stderr )
>  File "/home/applications/galaxy-dist/lib/galaxy/jobs/__init__.py", line 618, 
> in finish
>dataset.set_meta( overwrite = False )
>  File "/home/applications/galaxy-dist/lib/galaxy/model/__init__.py", line 
> 874, in set_meta
>return self.datatype.set_meta( self, **kwd )
>  File "/home/applications/galaxy-dist/lib/galaxy/datatypes/binary.py", line 
> 179, in set_meta
>raise Exception, "Error Setting BAM Metadata: %s" % stderr
> Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found
> 
> The weird thing is, when I remove the 2>&1 from the code and run it again in 
> Galaxy, I am able to get this diagnostic message:
> 
> [samopen] SAM header is present: 66338 sequences.
> 
> This is the reason why I added the 2>&1 in the first place, so that this 
> diagnostic message would be 'ignored' (since it is detected as an error).
> 
> Now, if the other code (the one without 2>&1) is able to produce the 
> aforementioned diagnostic message, then I assume that it is able to at least 
> execute the samtools command internally. But when I add the 2>&1, galaxy can 
> no longer find the command. Do I need to specify a path somewhere in the 
> galaxy-dist?

Hi Ciara,

This is most likely not the job, but Galaxy's metadata step, which uses 
samtools to create a BAM index (.bai) file.  If you put 
/home/applications/samtools-0.1.7a on your $PATH before you start Galaxy, that 
should solve this problem.

--nate

> 
> I apologize for the very long message.
> 
> Thanks in advance,
> 
> CL
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
> http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] What does this line of code mean

[Nate Coraor]

On Apr 29, 2012, at 9:26 PM, Ciara Ledero wrote:

> Hi all,
>  
> Can I ask what this line of code does?
>  
> job_wrapper.finish( stdout, stderr )
> It's the one from the local.py.

Hi Ciara,

This runs the JobWrapper.finish() method in 
galaxy-dist/lib/galaxy/jobs/__init__.py, which performs all the functions that 
have to be done once a tool has finished running.

--nate

>  
> Cheers,
>  
> CL
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Unable to set BAM Metadata

[Nate Coraor]

On Apr 29, 2012, at 9:19 PM, Ciara Ledero wrote:

> This is my code for running a samtools command internally, via Galaxy:
> 
> 
> open INP,"< $ARGV[0]"
> or die "Cannot open file: $!";
> 
> $file = "$ARGV[1]";
> 
> open OUT, "> $file"
> or die "Cannot open file: $!";
> 
> @out = `/home/applications/samtools-0.1.7a/samtools view -bS $ARGV[0] 2>&1`;
> 
> print OUT @out;
> 
> close INP;
> close OUT;
> 
> The 2>&1 is used to redirect the STDERR to STDOUT. When I run this in the 
> console, it gives a non-empty file. But when run it in Galaxy, Galaxy can no 
> longer find the path. This error message is given to me.
> 
> Traceback (most recent call last):
>   File "/home/applications/galaxy-dist/lib/galaxy/jobs/runners/local.py", 
> line 126, in run_job
> job_wrapper.finish( stdout, stderr )
>   File "/home/applications/galaxy-dist/lib/galaxy/jobs/__init__.py", line 
> 618, in finish
> dataset.set_meta( overwrite = False )
>   File "/home/applications/galaxy-dist/lib/galaxy/model/__init__.py", line 
> 874, in set_meta
> return self.datatype.set_meta( self, **kwd )
>   File "/home/applications/galaxy-dist/lib/galaxy/datatypes/binary.py", line 
> 179, in set_meta
> raise Exception, "Error Setting BAM Metadata: %s" % stderr
> Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found
> 
> The weird thing is, when I remove the 2>&1 from the code and run it again in 
> Galaxy, I am able to get this diagnostic message:
> 
> [samopen] SAM header is present: 66338 sequences.
> 
> This is the reason why I added the 2>&1 in the first place, so that this 
> diagnostic message would be 'ignored' (since it is detected as an error).
> 
> Now, if the other code (the one without 2>&1) is able to produce the 
> aforementioned diagnostic message, then I assume that it is able to at least 
> execute the samtools command internally. But when I add the 2>&1, galaxy can 
> no longer find the command. Do I need to specify a path somewhere in the 
> galaxy-dist?

Hi Ciara,

This is most likely not the job, but Galaxy's metadata step, which uses 
samtools to create a BAM index (.bai) file.  If you put 
/home/applications/samtools-0.1.7a on your $PATH before you start Galaxy, that 
should solve this problem.

--nate

> 
> I apologize for the very long message.
> 
> Thanks in advance,
> 
> CL
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Error running MACS in Galaxy

[Waldron, Michael H]

We get the following error when trying to run MACS in our local installation of 
Galaxy:

INFO  @ Mon, 30 Apr 2012 09:02:14:
# ARGUMENTS LIST:
# name = MACS_in_Galaxy
# ChIP-seq file = /proj/galaxy/000/dataset_552.dat
# control file = None
# effective genome size = 2.70e+09
# tag size = 36
# band width = 300
# model fold = 32
# pvalue cutoff = 1.00e-05
# Ranges for calculating regional lambda are : 1000,5000,1
INFO  @ Mon, 30 Apr 2012 09:02:14: #1 read tag files...
INFO  @ Mon, 30 Apr 2012 09:02:14: #1 read treatment tags...
Traceback (most recent call last):
  File "/nas02/apps/python-2.6.5/bin/macs", line 238, in 
main()
  File "/nas02/apps/python-2.6.5/bin/macs", line 53, in main
(treat, control) = _load_tag_files_options (options)
  File "/nas02/apps/python-2.6.5/bin/macs", line 211, in _load_tag_files_options
treat = load_tag_file(options.tfile,options.format)
  File "/nas02/apps/python-2.6.5/bin/macs", line 231, in load_tag_file
raise Exception("Format \"%s\" cannot be recognized!" % (format))
Exception: Format "SAM" cannot be recognized!

Does anyone know what causes this? We are using MACS-1.3.2.

Mike Waldron

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?

[Hans-Rudolf Hotz]

Hi Ann

I am definitively interested in any form of collaboration - thank you 
very much for bringing up this idea!


However, I also share some of the concerns which have been raised by 
others. Even in Europe, we sometimes struggle to coordinate a conference 
call with the Americas and very often the quality is so bad it is 
impossible to have a discussion. Also, looking at the success of your 
original e-mail, now imagine a conference call with more than 20 
participants...


You can call me 'old fashion' but I prefer face-to-face meetings. So I 
am looking very much forward to Chicago this summer. I am aware, due to 
costs and other reasons not everybody will be able to attend GCC.


Having regional Galaxy meetings is one solution. Together with the 
Galaxy developers, I intended to organize an European Galaxy workshop as 
part of ECCB2012 in Basel in September. Unfortunately, the ECCB 
organizers did approve our proposal. I will now organize a smaller 
(Swiss) workshop in fall which will be open to everybody. I'll keep the 
list posted as soon as it has been finalized.


Nevertheless, I hope we can create an even better community and I will 
make any effort to take part in any upcoming Galaxy conference calls.



Regards, Hans









On 04/27/2012 08:45 PM, Ann Black-Ziegelbein wrote:

Hi everyone -

Here at the University of Iowa we are working on deploying Galaxy
locally for campus wide access. I am interested in forming a community
of other institutions trying to deploy Galaxy locally and mange/operate
it on a broad level. Is anyone else? If there is enough interest,
possibly we could have a community conference call every other month to
have an open discussion on how we are all deploying galaxy,
customizations we are making, problems we are encountering, bugs, and
any add-on operations management for galaxy being developed, etc.

Would love to hear from others operating Galaxy or in process of
standing up a local deployment.

Thanks!

Ann Black-Ziegelbein
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] Merging BLAST database support into Galaxy?

[Peter Cock]

On Thu, Apr 26, 2012 at 10:40 PM, Edward Kirton  wrote:
> your suggestion for blastdbn and blastdbp sounds fine.
> it's okay if a few of our users need to edit the metadata of the dbs in
> their history. thanks for asking and doing this.

Great.

Perhaps you can throw some light on the peek issue I raised here:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-April/009525.html

Is looking at BLAST (nucleotide) databases in the history still working
fine on your local install? What about from the latest galaxy-central?

Thanks,

Peter

P.S. My branch is here - not really finished yet:
https://bitbucket.org/peterjc/galaxy-central/src/blastdb
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] How to customize Galaxy web interface?

[Pascal Maugeri]

Hi,

I m running my own Galaxy instance and I would like to do some simple
customization of the web interface such as adding my logos in the header
frame, changing background colors, etc.
I am having a look to static/ folders but I can't see clearly where the
header frame is defined.

Could you please tell me if there is any tutorial for doing so? Any
starting point? Or which folder/file(s) corresponds to the web interface in
the distribution?

Best Regards,
Pascal
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?

[Gianmauro Cuccuru]

On 04/27/2012 08:45 PM, Ann Black-Ziegelbein wrote:
> Hi everyone -
>
> Here at the University of Iowa we are working on deploying Galaxy
> locally for campus wide access.  I am interested in forming a
> community of other institutions trying to deploy Galaxy locally and
> mange/operate it on a broad level.  Is anyone else?   If there is
> enough interest, possibly we could have  a community conference call
> every other month to have an open discussion on how we are all
> deploying galaxy, customizations we are making, problems we are
> encountering, bugs, and any add-on operations management for galaxy
> being developed, etc.

I'm interested. We have different local Galaxy running and we are on the
way to connect them to our sequencing and genotyping platform.

Glad to partecipate,
Mauro

-- 
Gianmauro Cuccuru 
CRS4 Bioinformatica
POLARIS Edificio 3, Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it 
phone: +390709250428
email: gma...@crs4.it


>
> Would love to hear from others operating Galaxy or in process of
> standing up a local deployment.
>
> Thanks!
>
> Ann Black-Ziegelbein
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/