Re: [galaxy-dev] Public toolshed giving internal server error

2014-11-19 Thread Nate Coraor
Hi all, Sorry for the service interruption. The Tool Shed should be back now, and the underlying disk usage problem has been alleviated, so it shouldn't occur again. --nate On Wed, Nov 19, 2014 at 9:15 AM, Lukasse, Pieter pieter.luka...@wur.nl wrote: I am also having problems when using the

Re: [galaxy-dev] MarkupSafe egg missing? e.args[1].key != e.args[0].key

2014-11-18 Thread Nate Coraor
Peter, Unless you've made modifications to Galaxy that depend on external libraries, switching to a virtualenv for the server itself should be pretty safe. Tools themselves can still run without using the/any virtualenv, if desired. --nate On Tue, Nov 18, 2014 at 8:24 AM, Peter Cock

[galaxy-dev] Galaxy Mailing Lists are moving

2014-11-11 Thread Nate Coraor
Hello Galaxy Community, *Galaxy Mailing Lists are moving!* During the week of November 17-21, 2014, Galaxy mailing lists (including this one) will be migrated from lists.bx.psu.edu to lists.galaxyproject.org . This transition should be largely transparent to you, but there are a few things to

Re: [galaxy-dev] problems installing tool dependency

2014-09-24 Thread Nate Coraor
I've just looked at this as well, it appears that /pub4 is not exposed and does not allow you to change into it, but it does allow you to fetch if you request files under it via the full path. e.g.: nate@victory% ncftp ftp.sanger.ac.uk ... Logged in to ftp.sanger.ac.uk. ncftp / ls ls-lR.gz

Re: [galaxy-dev] Missing configuration and location files automatic creation

2014-09-22 Thread Nate Coraor
an eventually existing galaxy.ini and then create each default config files to the location pointed by galaxy.ini. -- Julien Seiler @julozi https://twitter.com/julozi Le 20 sept. 2014 à 03:10, Nate Coraor n...@bx.psu.edu a écrit : Hi Julien, What timing for this email - we have just made

Re: [galaxy-dev] Missing configuration and location files automatic creation

2014-09-19 Thread Nate Coraor
Hi Julien, What timing for this email - we have just made significant changes to the default config layout, and this includes no longer copying sample configs to their non-sample locations. The changes are in our default branch right now and will be part of the stable release scheduled for

Re: [galaxy-dev] find bam index files

2014-09-18 Thread Nate Coraor
Hi Ulf, Unfortunately, metadata files are assigned an id that is unrelated to the dataset id. To link the files you'll need to use the database. The numerical id you see in the filename is the id of the dataset in the `dataset` table. From this you can go backward to the

Re: [galaxy-dev] Queueing system and job parameters

2014-09-15 Thread Nate Coraor
this? My build is at least 3 months old, probably even more. Thanks, Prakash On Sep 11, 2014, at 1:17 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Prakash, You can use the relatively new job resource params feature to insert a standard set of params on any tool form. See the example

Re: [galaxy-dev] Keeping Galaxy up to date

2014-09-11 Thread Nate Coraor
Hi Sebastian, As for Galaxy Main (usegalaxy.org), our process is automated with Ansible, the playbook for which can be found here: https://github.com/galaxyproject/usegalaxy-playbook A bit of documentation on what you should look for (new versions of universe_wsgi.ini.sample,

Re: [galaxy-dev] ToolShed test failure: NotFound: cannot find 'ucsc_display_sites' while searching for 'APP.config.ucsc_display_sites'

2014-09-10 Thread Nate Coraor
Hi Peter, This was due to a bug I introduced last week, which I've just fixed in d1f6d05. Sorry for the trouble. --nate On Tue, Sep 9, 2014 at 9:50 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'm wondering why my samtools_depad repository tests have failed, and since I have not

Re: [galaxy-dev] Join tool failure

2014-09-05 Thread Nate Coraor
On Sep 4, 2014, at 2:44 PM, Iry Witham iry.wit...@jax.org wrote: Hi Team, I have upgraded to the latest galaxy distribution on my test server. Now I am running into issues and can't figure this one out. I am attempting to run the join.py tool to join two datasets at specified fields. I

Re: [galaxy-dev] problems with database migration 119 - 120

2014-08-26 Thread Nate Coraor
Hi Hans, I'll add to John's statement about the ordering - a different ordering than the model is entirely possible because alters to add columns always add them at the end. The tables for Galaxy Main are so old that they are in a very different order than most newer installations. As John

Re: [galaxy-dev] Galaxy and HTTPS User-authentification

2014-08-22 Thread Nate Coraor
On Aug 20, 2014, at 3:53 AM, Matthias Enders m.end...@german-seed-alliance.de wrote: Hi, I have a question regarding the user authentication of Galaxy. As to my knowledge galaxy uses http, also for the authentication, so the User Email and the password are send in clear text. As I

Re: [galaxy-dev] Database integrity error when installing new tools from toolshed

2014-08-22 Thread Nate Coraor
On Aug 21, 2014, at 3:48 PM, Michael Ta m...@pacificdx.com wrote: Hi, I updated a local Galaxy installation a while back and I followed the steps listed on one of the wiki pages. The update included changes to the database schema and I was careful to backup and restore it according to the

Re: [galaxy-dev] TypeError with 'dict'

2014-08-12 Thread Nate Coraor
Hi Martin, If there's a full traceback instead of just this fragment, that'd certainly help with determining exactly where the problem is coming from. --nate On Aug 12, 2014, at 4:17 AM, bjoern.gruen...@googlemail.com bjoern.gruen...@gmail.com wrote: Hi, Galaxy stores all information

Re: [galaxy-dev] Galaxy error

2014-08-12 Thread Nate Coraor
Hi Emily, We might be able to track this down on our side if you provide the URL and HTTP method that caused this error and the time it occurred at. Thanks, --nate On Aug 11, 2014, at 9:20 AM, John Chilton jmchil...@gmail.com wrote: Hello, Sorry for the delay - please keep all

Re: [galaxy-dev] Problems uploading files to a data library

2014-08-12 Thread Nate Coraor
On Tue, Aug 12, 2014 at 3:03 AM, Juan Vladimir de la Rosa Medina jvdelar...@hotmail.com wrote: Hi everyone, I recently have installed a local instance of galaxy in my computer, everything worked fine until I changed the path for *library_import_dir* in the *universe_wsgi.ini* file, to

Re: [galaxy-dev] can't install numpy from toolshed

2014-08-12 Thread Nate Coraor
On Aug 12, 2014, at 9:59 AM, Sajdak, Doris dj...@buffalo.edu wrote: I’m running Galaxy in a virtual environment as described here (OS is Debian 3.2.54-2): https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Use_a_clean_environment When I go to install

Re: [galaxy-dev] can't install numpy from toolshed

2014-08-12 Thread Nate Coraor
python interpreter (e.g. `/usr/bin/python2.7 /path/to/virtualenv.py /path/to/venvdir`). See the documentation for more: https://virtualenv.pypa.io/en/latest/virtualenv.html#environment-variables-and-configuration-files --nate Thanks, Dori From: Nate Coraor [mailto:n...@bx.psu.edu

[galaxy-dev] Galaxy Security Vulnerability

2014-07-31 Thread Nate Coraor
A security vulnerability was recently discovered by Inge Alexander Raknes that would allow a malicious person to execute arbitrary code on a Galaxy server. The vulnerability was in a method that uses Python pickle functionality to decode state information from tool forms. Because pickles can be

Re: [galaxy-dev] How to configure galaxy with a cluster

2014-07-24 Thread Nate Coraor
Hi Ben, Most of the tools you are looking for should be in the Galaxy Tool Shed: https://toolshed.g2.bx.psu.edu/ As for the earlier question regarding the resource selector drop down menu which I assume you are seeing on Galaxy Test, this is a feature currently available in the development

Re: [galaxy-dev] Providing BLAST db in a data library

2014-07-24 Thread Nate Coraor
On Jul 23, 2014, at 6:42 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Interesting hypothesis - you may well be right. Galaxy guys - who is the expert to talk to on this and/or where in the code should we be looking? Thanks, Peter I think there's a bit of a mixup here - Peter, I

Re: [galaxy-dev] Run Jobs as Real User - How to Configure it for TORQUE

2014-07-24 Thread Nate Coraor
On Jul 22, 2014, at 2:13 PM, John Chilton jmchil...@gmail.com wrote: Running jobs as the real user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user. That said, the DRMAA runner plugin is compatible with Torque, you just need to

Re: [galaxy-dev] Delete an account (Galaxy local instance)

2014-07-08 Thread Nate Coraor
On Jul 7, 2014, at 6:11 AM, Pat-74100 leonardsqual...@hotmail.com wrote: I've found it: I've modified it in the file universe_wgsi.ini and now I can delete user by the admin tool panel. The problem is: when I delete the account and purge it why it still remain in the list ? is it

Re: [galaxy-dev] Object-Store, setting filetypes crashes Galaxy

2014-06-12 Thread Nate Coraor
Hi Björn, I believe the problem is most likely this bug: https://trello.com/c/OHlQoCrx If you symlink /usr/local/galaxy/galaxy-dist/database/tmp to somewhere writeable (e.g. what you have new_files_path set to), that'd confirm it. I'm going to take a look at fixing this today. --nate On

Re: [galaxy-dev] Old Tool Shed URL http://community.g2.bx.psu.edu/ dead

2014-05-28 Thread Nate Coraor
Hi all, This has been fixed. Sorry for the trouble, I am not sure when it disappeared. --nate On Tue, May 27, 2014 at 11:19 AM, Greg Von Kuster g...@bx.psu.edu wrote: Peter, sorry, this one is out of my hands. I'm hoping Nate can answer this when he gets a chance. Greg Von Kuster On

Re: [galaxy-dev] User information when creating a new user

2014-05-15 Thread Nate Coraor
Hi Neil, The address fields can be populated from User - Preferences - Manage your information - User Addresses. You can also create custom forms - see the Manage form definitions link in the admin section. I believe it used to be possible to make custom User Information forms required at

Re: [galaxy-dev] drmaa

2014-05-06 Thread Nate Coraor
Hi Donny, You should be able to specify the queue using the nativeSpecification field of drmaa requests, e.g. in your job_conf.xml: destination id=batch runner=pbs_drmaa param id=nativeSpecification-q batch/param /destination Documentation on job_conf.xml's syntax by runner can be found

Re: [galaxy-dev] Help with cluster setup

2014-05-01 Thread Nate Coraor
April release. In addition, the tarball linked would pull from the default branch of Galaxy, which includes unstable changesets. --nate 2013-08-08 22:58 GMT+08:00 Nate Coraor n...@bx.psu.edu: On Aug 7, 2013, at 9:23 PM, shenwiyn wrote: Yes,and I also have the same confuse about

Re: [galaxy-dev] version of SAM-to-BAM (CloudMap)

2014-04-04 Thread Nate Coraor
Hi Xiaofei, You should be able to install 1.1.3 by selecting revision 3 of sam_to_bam from the Preview and install page (Admin - Search and browse tool sheds - Galaxy main tool shed - sam_to_bam - Preview and install), and then clicking Install to Galaxy. [image: Inline image 1] You can tell

Re: [galaxy-dev] Depth of Coverage (local galaxy __ CloudMap)

2014-04-04 Thread Nate Coraor
Hi Xiaofei, SQLite is not suitable for complex tasks like running workflows. Please switch to PostgreSQL: https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Switching_to_a_database_server --nate On Fri, Apr 4, 2014 at 1:35 PM, Wang, Xiaofei xfw...@ku.edu wrote: Dear

Re: [galaxy-dev] [CONTENT] Re: Re: Re: Re: Unable to remove old datasets

2014-04-04 Thread Nate Coraor
-- From: Nate Coraor n...@bx.psu.edu Date: Friday, March 28, 2014 1:56 PM To: Ravi Sanka rsa...@jcvi.org Cc: Carl Eberhard carlfeberh...@gmail.com, Peter Cock p.j.a.c...@googlemail.com, galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: [CONTENT] Re: Re: [galaxy-dev] Re

Re: [galaxy-dev] Galaxy packages for Bio-Linux - an update + Apache proxying

2014-04-04 Thread Nate Coraor
On Fri, 2014-03-28 at 18:49 +, Nate Coraor wrote: Hi Tim, I have recently been working on getting Galaxy Main's configs and server-modified files and directories out of the galaxy-dist directory, so our goals are aligning. Not everything can be moved without some trickery (e.g

Re: [galaxy-dev] Problems submitting using PBS Pro 12.1

2014-04-03 Thread Nate Coraor
Hi Luca, Could you send your job_conf.xml please? Also, is there any chance that you modified run.sh to set DRMAA_LIBRARY_PATH there (which would override what you set on the command line)? --nate On Thu, Apr 3, 2014 at 9:17 AM, Luca Toldo lucato...@gmail.com wrote: Dear All, after having

Re: [galaxy-dev] Problems submitting using PBS Pro 12.1

2014-04-02 Thread Nate Coraor
Hi Luca, Are you passing any params to the runner for inclusion as PBS submit parameters (param id=nativeSpecification in job_conf.xml)? --nate On Wed, Apr 2, 2014 at 12:59 PM, Luca Toldo lucato...@gmail.com wrote: Dear all, SUSE SLE 11 SP 1, with PBS Pro 12.1 client delivers

Re: [galaxy-dev] Problems submitting using PBS Pro 12.1

2014-04-02 Thread Nate Coraor
using it, he gets the same error. Cheers, Bjoern Am 02.04.2014 20:34, schrieb Nate Coraor: Hi Luca, Are you passing any params to the runner for inclusion as PBS submit parameters (param id=nativeSpecification in job_conf.xml)? --nate On Wed, Apr 2, 2014 at 12:59 PM, Luca Toldo lucato

Re: [galaxy-dev] Problems submitting using PBS Pro 12.1

2014-04-02 Thread Nate Coraor
and it constantly fails. Unfortunately I also got serious problem trying to compile the pbs-drmaa-1.0.17 as mentioned in my posting. What can I try next ? I was thinking to modify /jobs/runners/drmaa.py adding more logging ... Thankyou for your advice ! 2014-04-02 20:43 GMT+02:00 Nate Coraor n

Re: [galaxy-dev] [CONTENT] Re: Re: Unable to remove old datasets

2014-03-28 Thread Nate Coraor
Hi Ravi, If you take a look at the dataset's entry in the history_dataset_association table, is that marked deleted? admin_cleanup_datasets.py only marks history_dataset_association rows deleted, not datasets. Running the cleanup_datasets.py flow with -d 0 should have then caused the dataset to

Re: [galaxy-dev] [CONTENT] Re: Re: Re: Unable to remove old datasets

2014-03-28 Thread Nate Coraor
-- From: Nate Coraor n...@bx.psu.edu Date: Friday, March 28, 2014 9:59 AM To: Ravi Sanka rsa...@jcvi.org Cc: Carl Eberhard carlfeberh...@gmail.com, Peter Cock p.j.a.c...@googlemail.com, galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: [CONTENT] Re: [galaxy-dev] Re: Re: Unable

Re: [galaxy-dev] restarting Galaxy without affecting jobs

2014-03-28 Thread Nate Coraor
Hi David, Setting track_jobs_in_database = True should not be required, recovery is supposed to work either way. Does Galaxy lose all jobs, or just the ones that completed while Galaxy was restarting? Can you provide the output from the Galaxy log that shows an attempt to recover a job and all

Re: [galaxy-dev] Galaxy packages for Bio-Linux - an update + Apache proxying

2014-03-28 Thread Nate Coraor
Hi Tim, I have recently been working on getting Galaxy Main's configs and server-modified files and directories out of the galaxy-dist directory, so our goals are aligning. Not everything can be moved without some trickery (e.g. symlinks) but most paths, including the paths to shed_*.xml are

Re: [galaxy-dev] homebrew python

2014-03-28 Thread Nate Coraor
Hi Joshua, You may be able to trick Galaxy into using existing versions of OS X eggs, they are built for both 32 and 64-bit Intel, but should work fine with a single-arch build. If the attached patch works, let me know and I'll commit it. If you'd rather not mess with the Galaxy source, you

Re: [galaxy-dev] Question concerning the xml file for local tools with multiple output files.

2014-03-28 Thread Nate Coraor
Hi Lifeng, Another option would be the 'from_work_dir' option to the data tag. Have a look at the tophat repository in the Tool Shed for an example: http://toolshed.g2.bx.psu.edu/view/devteam/tophat --nate On Tue, Mar 25, 2014 at 11:29 AM, Hans-Rudolf Hotz hansrudolf.h...@fmi.chwrote:

Re: [galaxy-dev] jobs stuck in new state

2014-03-28 Thread Nate Coraor
Hi David, This is pretty common in the case of workflows. When a workflow step fails, the next job in the workflow will be set to the paused state and all jobs downstream of the paused job will remain in the new state until corrective action is taken. The current query for finding

Re: [galaxy-dev] ERROR executing tool

2014-03-28 Thread Nate Coraor
On Thu, Mar 27, 2014 at 7:13 AM, virginia dalla via virdalla...@hotmail.com wrote: Hi, I tried to GROOMER my fastq data in fastq data, and galaxy did not allowed me:Error executing tool: objectstore, __call_method failed: get_filename on , kwargs: {} could you please help me? Thank you

Re: [galaxy-dev] How to install bowtie2 tool in galaxy

2014-03-14 Thread Nate Coraor
On Wed, Mar 5, 2014 at 8:18 PM, Ravi Alla ravi.a...@berkeley.edu wrote: Hi Jennifer, Thank you for this information. I was able to troubleshoot the bowtie2_indices. Like Bjoern said the bowtie2 tool needed to be reinstalled because the tool_table does not load the correct indices. I am still

Re: [galaxy-dev] Tool Shed install best practise : Precompiled binaries vs local compile

2014-03-14 Thread Nate Coraor
Hi Peter, I just wanted to add my $0.02 USD to say that I mostly agree with this - I have long used binaries precompiled by the tool author on Main, especially for cases where, as you say, the compile-time dependency list is large and painful. The only gotcha here is to make sure that binaries

Re: [galaxy-dev] Tool Shed install best practise : Precompiled binaries vs local compile

2014-03-14 Thread Nate Coraor
On Fri, Mar 14, 2014 at 12:47 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Fri, Mar 14, 2014 at 4:41 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Peter, I just wanted to add my $0.02 USD to say that I mostly agree with this - I have long used binaries precompiled by the tool author on Main

Re: [galaxy-dev] Torque Exec Exception

2014-03-07 Thread Nate Coraor
Hello, Are you able to run `qstat -x` on the command line on the remote system? What's the output? --nate On Thu, Mar 6, 2014 at 1:01 PM, ngsf...@hygenomics.com ngsf...@hygenomics.com wrote: hi all: galaxy.jobs.runners.cli_job.torque WARNING 2014-03-07 11:23:33,764 No valid qstat XML

Re: [galaxy-dev] FTP password and web interface password

2014-03-05 Thread Nate Coraor
Hi Yec'han, Could you check that the 'password' column for the user in question in the 'galaxy_user' table in the database does not begin with $PBKDF2$? If not, do you have any debug logs from the FTP session and server that provide details on the failure? --nate On Wed, Mar 5, 2014 at 10:36

Re: [galaxy-dev] Uploaded files not previewing at random

2014-03-05 Thread Nate Coraor
Hi Ravi, Are your datasets stored on a shared filesystem (e.g. NFS)? If yes, you could try setting `retry_job_output_collection` to something 0 in universe_wsgi.ini? NFS attribute caching can produce behavior like this. --nate On Wed, Mar 5, 2014 at 11:46 AM, Sanka, Ravi rsa...@jcvi.org wrote:

Re: [galaxy-dev] Markup safe egg problem/fetching the wrong eggs?

2014-02-28 Thread Nate Coraor
Hi Joe, Something is automatically re-fetching the Python 2.6 eggs? Is the Galaxy server itself using 2.6 perhaps? --nate On Fri, Feb 28, 2014 at 3:42 AM, Joseph Ward j.x.w...@dundee.ac.uk wrote: Hi, I've got a strange problem with python paths/eggs. When I run any tool that produces

Re: [galaxy-dev] Mercurial wants to overwrite my job_conf.xml

2014-02-27 Thread Nate Coraor
Hi Carrie, .hgignore should not affect files that you have explicitly tracked (e.g. with `hg add job_conf.xml`) - if the file is tracked, Mercurial will attempt to update and patch the working copy if there are upstream changes. Also, it looks like your current tip includes neither .hgignore or

Re: [galaxy-dev] Mercurial wants to overwrite my job_conf.xml

2014-02-27 Thread Nate Coraor
(6dc34fa Refined cli runner..). Should I just pull 6dc and replace the files I need from bf3 manually? Thanks so much for the advice, Carrie Ganote From: Nate Coraor [n...@bx.psu.edu] Sent: Thursday, February 27, 2014 8:49 AM To: Ganote, Carrie L Cc

Re: [galaxy-dev] local toolshed with remote-user=true and require-user=falase

2014-02-26 Thread Nate Coraor
Hi Brad, If you're using HTTP Auth, that will be the case since HTTP Auth has no notion of sessions, the auth credentials must be provided with every request. For a more robust solution, you probably want to use an auth filter that creates an authentication session. Penn State uses Cosign for

Re: [galaxy-dev] Cufflinks doesn't stop

2014-02-26 Thread Nate Coraor
Hi Björn, Does stopping other, non-split jobs work correctly for you? --nate On Wed, Feb 26, 2014 at 7:30 AM, Björn Grüning bjoern.gruen...@gmail.comwrote: Hi Nate, I'm running changeset: 12641:cab3db6e1d59 and I can see the same behaviour. The blast job is not listed under jobs in

Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml

2014-02-26 Thread Nate Coraor
/ ncbi_tblastx_wrapper/ destination=24_cores_24G / tool id=toolshed.g2.bx.psu.edu/repos/peterjc/blast2go/ blast2go/0.0.6 destination=20G_memory/ Thanks, Bjoern Am 14.02.2014 16:37, schrieb Nate Coraor: Hi Björn, It should be fixed, the problem was with tools with uppercase

Re: [galaxy-dev] Cufflinks doesn't stop

2014-02-25 Thread Nate Coraor
Hi all, There was a regression introduced in the most recent stable release that was preventing jobs from being stopped, fixed here: https://bitbucket.org/galaxy/galaxy-central/commits/1298d3f6aca59825d0eb3d32afd5686c4b1b9294?at=stable You can pull from the stable branch of galaxy-central to

Re: [galaxy-dev] Error when installing package_galaxy_utils

2014-02-25 Thread Nate Coraor
- Original Message - From: Nate Coraor n...@bx.psu.edu To: Sarah Diehl di...@ie-freiburg.mpg.de Cc: galaxy-dev@lists.bx.psu.edu List galaxy-dev@lists.bx.psu.edu Sent: Friday, February 21, 2014 5:10:46 AM Subject: Re: [galaxy-dev] Error when installing package_galaxy_utils Hi Sarah

Re: [galaxy-dev] Error when installing package_galaxy_utils

2014-02-20 Thread Nate Coraor
Hi Sarah, This looks a lot like a transient filesystem error. If you attempt to reinstall the repository, do you get the same error? --nate On Thu, Feb 20, 2014 at 10:48 AM, Sarah Diehl di...@ie-freiburg.mpg.dewrote: Hi everyone, I just updated our local Galaxy installation to the latest

Re: [galaxy-dev] Database and FTP setup

2014-02-20 Thread Nate Coraor
Hi Turner, You should be connecting to the database that Galaxy is using, specified in the database_connection option in universe_wsgi.ini. If you have not yet started Galaxy at least once, then the tables will not have been created yet. Once you run Galaxy and the tables have been created, you

Re: [galaxy-dev] Database and FTP setup

2014-02-20 Thread Nate Coraor
7703 Floyd Curl Drive San Antonio, TX 78229 Email: conr...@livemail.uthscsa.edu Office: (210) 567-3947 Cell: (210) 954-0893 On Feb 20, 2014 10:23 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Turner, You should be connecting to the database that Galaxy is using, specified

Re: [galaxy-dev] Acceleration Card Optimization

2014-02-18 Thread Nate Coraor
Hi Adam, This is going to depend entirely on the tools you're using. I'd suggest running your typical workload and examining the I/O patterns to determine what makes the most sense. I'd think the temp directory, input dataset, and output dataset filesystems to be the most utilized. --nate On

Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml

2014-02-14 Thread Nate Coraor
think it's still not working? Or is that a regression. In particular shed_host/repos/owner/repo/tool_id is not working, and that is what is 90% needed, or? Thanks, Bjoern From: Nate Coraor [mailto:n...@bx.psu.edu] This should work, it looks like there may be a bug with handling tool

Re: [galaxy-dev] sysadmin help

2014-01-29 Thread Nate Coraor
Hi Donny, Thanks for the heads up about the broken link, I've fixed it. Other than that, are you running in to problems connecting Galaxy to your existing cluster? --nate On Tue, Jan 28, 2014 at 9:29 PM, Shrum, Donald C dcsh...@admin.fsu.eduwrote: Hi all, I could use a little basic help.

Re: [galaxy-dev] Can REMOTE_USER be changed to another header variable?

2014-01-03 Thread Nate Coraor
without issues. Thanks, Prakash On Jan 3, 2014, at 11:45 AM, Nate Coraor n...@bx.psu.edu wrote: Hi Prakash, This was not previously possible, but I have added a config option for it: https://bitbucket.org/galaxy/galaxy-central/commits/e92e13e9c103cc1f36dff65e1523479bf5cb17ed If you're

Re: [galaxy-dev] Can REMOTE_USER be changed to another header variable?

2014-01-03 Thread Nate Coraor
3, 2014, at 2:11 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Prakash, Could you send the output of `hg summary`? Thanks, --nate On Fri, Jan 3, 2014 at 1:41 PM, Velayutham, Prakash (Prakash) prakash.velayut...@cchmc.org wrote: Hi Nate, I just updated my copy and the changes you pushed

Re: [galaxy-dev] Local installation problem

2014-01-02 Thread Nate Coraor
, this is occurring because of a bug in the version of the Babel library Galaxy is using. To work around this, set the environment variable `LC_ALL` to a valid locale, e.g. 'en_US.UTF-8' or 'C', before starting Galaxy: % export LC_ALL='en_US.UTF-8' % sh run.sh --nate Thanks Cesar De: Nate Coraor n

Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

2013-12-20 Thread Nate Coraor
Hi Ben, The job running code is in lib/galaxy/jobs/. Galaxy jobs get a wrapper which include the start/finish methods, that's in __init__.py. handler.py is what dispatches jobs out to various runner plugins, finds new jobs to run, and generally controls the operation. runners/*.py are the

Re: [galaxy-dev] Sysadmin Question

2013-12-19 Thread Nate Coraor
Hi Donny, Galaxy expects data to be copied in to its data store (by default, galaxy-dist/database/files/), and it's not designed to access data in the way you're suggesting. However, there are some workarounds. First, if the data is added to a Data Library (an admin feature, although once a

Re: [galaxy-dev] a galaxy of issues; Job output not returned from cluster,drmaa getting wrong LSF job id and job.o files not being found and moving files occurring before upload completes.

2013-12-12 Thread Nate Coraor
Hi Starr, I'd suggest setting 'retry_job_output_collection' in universe_wsgi.ini to some value 0 (e.g. 5). You may also want to try remounting the filesystem on which your working directories are located with the `-noac` temporarily just to rule out that the problem is related to attribute

Re: [galaxy-dev] hierarchical ObjectStore - noatime

2013-12-03 Thread Nate Coraor
Hi Björn, atime is definitely not used by Galaxy, and we have generally disabled it on all filesystems we mount. --nate On Tue, Dec 3, 2013 at 8:10 AM, Dannon Baker dannon.ba...@gmail.com wrote: On Tue, Dec 3, 2013 at 5:49 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: The new

Re: [galaxy-dev] Local installation problem

2013-11-20 Thread Nate Coraor
Hi Cesar, The easiest way to avoid module version conflicts is to use virtualenv to set up a self-contained environment: http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Use_a_clean_environment --nate On Tue, Nov 19, 2013 at 4:02 PM, Cesar Rodriguez

Re: [galaxy-dev] secure Galaxy with SSL

2013-11-19 Thread Nate Coraor
On Tue, Nov 19, 2013 at 12:32 PM, Jingchao Zhang zh...@unl.edu wrote: Hi Peter, Thanks so much for your help! It is like you said a browser issue. I also noticed that the Galaxy main server (https://usegalaxy.org/) doesn't have this problem. And their USCS main table browser uses https

Re: [galaxy-dev] wigTobigwig error: broken pipe and len file not found

2013-11-18 Thread Nate Coraor
On Sat, Nov 16, 2013 at 12:27 PM, ruiwang.sz ruiwang...@gmail.com wrote: Hi All, I‘m seeing some weird error messages...I googled but didn't see anything useful: So, it is during the wigToBigwig conversion: Dataset generation errors Dataset 47: Wig/BedGraph-to-bigWig on data 43 Tool

Re: [galaxy-dev] Setting a queue for torque submission

2013-11-15 Thread Nate Coraor
Xian and I were able to resolve this on IRC, the problem is is that no default queue is specified in the PBS client (intentionally) and the `-q` param was improperly handled in the pbs job runner (it expected param id=“destination” instead). This has been fixed in the stable branch in

Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries

2013-11-14 Thread Nate Coraor
Bash is easily obtained on these systems and I think the extra functionality available in post-Bourne shells ought to be allowed. I also question how many people are going to run tools on *BSD since most underlying analysis tools tend to only target Linux. That said, shell code should be

Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries

2013-11-14 Thread Nate Coraor
On Nov 14, 2013, at 12:21 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Nov 14, 2013 at 5:13 PM, Nate Coraor n...@bx.psu.edu wrote: Bash is easily obtained on these systems and I think the extra functionality available in post-Bourne shells ought to be allowed. I also question how

Re: [galaxy-dev] Fw: No peek issue and datasets wrongly reported as Empty

2013-11-08 Thread Nate Coraor
On Nov 7, 2013, at 2:45 PM, Jean-Francois Payotte wrote: Dear Galaxy developers, I know I am not the only one with this issue, as over time I've stumbled on a few mailing-list threads with other users having the same problem. And I know the recommended solution is to use the -noac mount

Re: [galaxy-dev] Security vulnerability in Galaxy filtering tools

2013-11-05 Thread Nate Coraor
• News/2013_04_08_Galaxy_Security_Release 1.3k - rev: 3 (current) last modified: 2013-04-08 16:56:41 escape does not bring up anything. thank you very much, ido On Nov 5, 2013, at 12:45 AM, Nate Coraor n...@bx.psu.edu wrote: A security vulnerability was recently discovered

[galaxy-dev] Security vulnerability in Galaxy filtering tools

2013-11-04 Thread Nate Coraor
A security vulnerability was recently discovered by John Chilton with Galaxy's Filter data on any column using simple expressions and Filter on ambiguities in polymorphism datasets tools that can allow for arbitrary execution of code on the command line. The fix for these tools has been

Re: [galaxy-dev] jobid exception

2013-10-16 Thread Nate Coraor
Hi Rémy, Do you have some process that restarts Galaxy if it dies? It should not do this itself. Is there anything being logged right before the server restarts that indicates the problem? --nate On Oct 16, 2013, at 4:10 AM, Rémy Dernat wrote: Hi, About the last updates on DRMAA: now

Re: [galaxy-dev] AttributeError: type object 'InvalidJobException' has no attribute 'name'

2013-10-14 Thread Nate Coraor
On Oct 14, 2013, at 6:07 AM, Peter Cock wrote: On Fri, Oct 11, 2013 at 9:12 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Oct 10, 2013 at 8:20 PM, Nate Coraor n...@bx.psu.edu wrote: Hi all, The recent changes to the DRMAA runner are for better handling of job-ending conditions

Re: [galaxy-dev] RFC: remove trailing semicolons from command line - Broken bowtie2_wrapper on some SGE systems

2013-10-11 Thread Nate Coraor
Hi Björn, Are you certain that SGE is responsible? I ran in to the same thing with bowtie2 with other runners. The double semicolon is invalid syntax in Bourne-compatible shells and so the tool ought to be fixed not to produce them (it happens under certain combinations of options). Have

Re: [galaxy-dev] UCSC Main table browser invalid link

2013-10-10 Thread Nate Coraor
On Oct 9, 2013, at 5:40 PM, Gonsky, Rivkah, Ph.D wrote: Link doesn’t work Hi Rivkah, If you're referring to the Galaxy instance at usegalaxy.org, this has been fixed. Thanks, --nate Rivkah Gonsky Inflammatory Bowel and Immunobiology Research Institute Davis 4063 8700 Beverly Blvd

Re: [galaxy-dev] AttributeError: type object 'InvalidJobException' has no attribute 'name'

2013-10-10 Thread Nate Coraor
Hi all, The recent changes to the DRMAA runner are for better handling of job-ending conditions under slurm, but it looks like SGE has different behavior when a job finishes. I'll provide a fix for this shortly, in the meantime, it's fine to use a slightly older version of drmaa.py. --nate

Re: [galaxy-dev] 'DECLARE CURSOR ... FOR UPDATE/SHARE is not supported'

2013-09-11 Thread Nate Coraor
Hi Sergei, Your current Galaxy database migration revision (13) is extremely old. How old was your previous Galaxy installation, and are you certain that the dump you created was complete, and fully restored? To solve the original 'DECLARE CURSOR ... FOR UPDATE' problem, you'd need to update

Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml

2013-09-11 Thread Nate Coraor
Hi Simon, This should work, it looks like there may be a bug with handling tool shed IDs when determining job destinations. You are actually supposed to be able to use any of: shed_host/repos/owner/repo/tool_id/version shed_host/repos/owner/repo/tool_id tool_id I'll take a look at this

Re: [galaxy-dev] 'DECLARE CURSOR ... FOR UPDATE/SHARE is not supported'

2013-09-11 Thread Nate Coraor
On Sep 11, 2013, at 9:16 AM, hogart wrote: Hi Nate, Yes, db schema is old, but the upgrading (manage_db.sh upgrade) to the current version resulted to the error, as mentioned above. I don't remember the release number of the previous version of Galaxy, but it was installed in the March

Re: [galaxy-dev] Bismark error

2013-09-04 Thread Nate Coraor
penalty. --nate Bests, Nikos 2013/9/4 Nate Coraor n...@bx.psu.edu On Sep 4, 2013, at 9:17 AM, Nikos Sidiropoulos wrote: Hi Bjørn that does not look like a bismark error. Is it happen with other tools as well? No, I have only experienced it with Bismark. Not sure, sorry

Re: [galaxy-dev] Bismark error

2013-09-04 Thread Nate Coraor
On Sep 4, 2013, at 9:17 AM, Nikos Sidiropoulos wrote: Hi Bjørn that does not look like a bismark error. Is it happen with other tools as well? No, I have only experienced it with Bismark. Not sure, sorry. But you should migrate to the new job configuration, better sooner than

Re: [galaxy-dev] Local installation: Uploading fastq file takes ages - why?

2013-08-29 Thread Nate Coraor
On Aug 14, 2013, at 8:39 AM, Boaz Shaanan wrote: Hi, On my local galaxy installation (updated to the latest version yesterday), it takes ages (hours!) to upload a fastq files. I attach a report from the the computer on which galaxy is installed. The fastq file is uploaded from

Re: [galaxy-dev] Help with cluster setup

2013-08-29 Thread Nate Coraor
#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_worker = 5 Where cn01 and cn02 are cluster nodes echo $DRMAA_LIBRARY_PATH /usr/local/lib/libdrmaa.so On 8 August 2013 16:58, Nate Coraor n...@bx.psu.edu wrote: On Aug 7, 2013, at 9:23 PM, shenwiyn wrote

Re: [galaxy-dev] Postgresql database cleaning

2013-08-29 Thread Nate Coraor
On Aug 26, 2013, at 5:03 AM, Christophe Antoniewski wrote: Hi everybody, The python scripts to clean histories, datasets, users etc.. are fine... However, the records are not really removed from the postgresql database and as a result, this one gets bigger and bigger with unused records.

Re: [galaxy-dev] Postgresql database cleaning

2013-08-29 Thread Nate Coraor
On Aug 29, 2013, at 11:50 AM, Nate Coraor wrote: On Aug 26, 2013, at 5:03 AM, Christophe Antoniewski wrote: Hi everybody, The python scripts to clean histories, datasets, users etc.. are fine... However, the records are not really removed from the postgresql database and as a result

Re: [galaxy-dev] tool_dependencies.xml format

2013-08-27 Thread Nate Coraor
On Aug 26, 2013, at 11:59 AM, James Taylor wrote: On Mon, Aug 26, 2013 at 11:48 AM, John Chilton chil...@msi.umn.edu wrote: I think it is interesting that there was push back on providing infrastructure (tool actions) for obtaining CBL from github and performing installs based on it because

Re: [galaxy-dev] how to add settings of TMPDIR=/scratch for sge/drmaa in universe_wsgi.ini

2013-08-15 Thread Nate Coraor
On Aug 14, 2013, at 9:46 PM, tin h wrote: Hello gurus, I made some slight progress... If I specify a handlers section into the job_conf.xml file with: handlers handler id=main/ /handlers Then galaxy would at least start. However, it is not very functional, I

Re: [galaxy-dev] Jobs remain in queue until restart

2013-08-14 Thread Nate Coraor
On Aug 2, 2013, at 1:06 PM, Thon de Boer wrote: I did some more investigation of this issue I do notice that my 4 core, 8 slot VM machine has a load of 32, with only my 4 handler processes running (Plus my web server), but not even getting more than 10% of the CPU each. There seems to

Re: [galaxy-dev] Upstart script to manage a multi instance load balanced installation

2013-08-14 Thread Nate Coraor
unpack to a directory in /tmp/ env PYTHON_EGG_CACHE=/tmp/galaxy_eggs/ respawn script exec python ./scripts/paster.py serve universe_wsgi.ini --server-name=${SERVER_NAME} end script On Thu, Aug 8, 2013 at 1:52 PM, Nate Coraor n...@bx.psu.edu wrote: On Jul 15, 2013, at 6:29

Re: [galaxy-dev] support pbkdf2 in proftpd 1.3.5rc3

2013-08-09 Thread Nate Coraor
server, but now that I'm aware that ProFTPD has PBKDF2 support, I will put this on my to-do list for next week to test. --nate On Thu, Aug 8, 2013 at 8:45 PM, Nate Coraor n...@bx.psu.edu wrote: On Jul 26, 2013, at 3:51 PM, Leon Mei wrote: Dear galaxy developers, We have tried today

  1   2   3   4   5   6   7   8   >