Re: [galaxy-dev] Galaxy not killing split cluster jobs

2012-07-10 Thread Scott McManus
A suggested change will be coming down the pipe shortly, but it's good to hear that it will be useful! -Scott - Original Message - > On Tue, May 1, 2012 at 3:46 PM, Dannon Baker > wrote: > > I'll take care of it. Thanks for reminding me about the TODO! > > > > This seems to have reach

Re: [galaxy-dev] egg distribution error when running galaxy-central

2012-08-01 Thread Scott McManus
Clare- Have you resolved this yet? I had seen similar issues with respect to pycrypto before I used virtualenv, but I don't believe that I saw it afterwards. I'll take a look and see if I can reproduce it. I only have obvious things to recommend right now. For example, make sure that the environ

Re: [galaxy-dev] Local instance and storage space to provide

2012-08-01 Thread Scott McManus
Hi Sarah- I'm sure others have experiences that could help, so I hope they will chime in. Nate put together some slides two years ago on production setups, including some architecture for the server main.g2.bx.psu.edu: http://wiki.g2.bx.psu.edu/Events/GDC2010?action=AttachFile&do=get&target=GDC20

Re: [galaxy-dev] PBS Server Throwing Errors

2012-08-01 Thread Scott McManus
Hi Dan- Thanks again for your great hospitality at UCI! It looks like you've done most of the sane things, and a quick check looks like the PBS egg is requesting protocol version 1 while the server supports protocol version 2. One possibility is that there is a protocol mismatch, which could be

Re: [galaxy-dev] PBS Server Throwing Errors

2012-08-01 Thread Scott McManus
I've been told by Galaxy folks to keep this on galaxy-dev. My apologies for any confusion. -Scott - Original Message - > > Hi Dan- > > Thanks again for your great hospitality at UCI! > > It looks like you've done most of the sane things, and a quick > check looks like the PBS egg is r

Re: [galaxy-dev] egg distribution error when running galaxy-central

2012-08-01 Thread Scott McManus
I haven't been able to reproduce this yet with the instructions you gave, but I'm not using the same environment. Can you give me an idea of what tools you're using outside of SciPy/NumPy/Enthought stuff? There is the possibility that the virtualenv.py script isn't being sourced correctly. We ca

Re: [galaxy-dev] [galaxy-user] Fwd: Local instance and storage space to provide

2012-08-01 Thread Scott McManus
Sorry - I fouled up and should have kept this on galaxy-dev. -Scott - Original Message - > > I'm forwarding this to galaxy-user instead of galaxy-dev. > > -Scott > > - Forwarded Message - > From: "Scott McManus" > To: "Sarah Mama

Re: [galaxy-dev] Strange "Could not moveŠ." error

2012-08-02 Thread Scott McManus
Hi Iry- A call to shutil.move is failing, and this could be because file-copying semantics are different on your test instance's OS and/or filesystem than they are on your development instance's OS and/or filesystem. What OS and file system are you using on each? We can also try writing a sm

Re: [galaxy-dev] Strange "Could not moveŠ." error

2012-08-02 Thread Scott McManus
Ok - that makes sense. So the from_work_dir attribute is part of the tool's element under , and Jeremy fixed a timing issue. Thanks, Jeremy! -Scott - Original Message - > Iry, > This is probably an issue with using the from_work_dir attribute. > Prior to changeset 524cfb8ca67d, t

Re: [galaxy-dev] Local Galaxy concept system: hardware spec questions

2012-08-13 Thread Scott McManus
Hey Sebastian- It may help to consider other pieces aside from compute nodes that you will need, such as nodes for proxies and databases, networking gear (such as switches and cables), and so on. http://usegalaxy.org/production has some details, and there are high-level pieces explained at http

Re: [galaxy-dev] code example for improved error handling

2012-08-17 Thread Scott McManus
I'm writing documentation now - I'll have something this afternoon. Sorry for the delay. -Scott - Original Message - > No, I call the executable directly from the xml. It kept failing > although it seemed to finish the job and I realized that on success > the executable prints a summa

Re: [galaxy-dev] code example for improved error handling

2012-08-17 Thread Scott McManus
Please see http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax . There is a section for ", , and tag sets". The documentation applies to the latest galaxy-dist, though most of what's mentioned (aside from updating stdout and stderr with warning messages) is supported in galaxy-centra

Re: [galaxy-dev] [galaxy-user] LimitInternalRecursion

2012-08-22 Thread Scott McManus
It's the last rewrite rule that's causing the infinite recursion: RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] Check out this link: http://httpd.apache.org/docs/current/mod/mod_rewrite.html#rewriterule Look for the table of given rules and resulting substitutions - the proxying appear

Re: [galaxy-dev] Galaxy with LSF scheduler

2012-08-24 Thread Scott McManus
I haven't worked with an LSF cluster before, but it looks like it has DRMAA bindings. That means that you could try setting up a DRMAA galaxy runner : http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster -Scott - Original Message - > Hello, > Could you please tell me if it's p

Re: [galaxy-dev] Galaxy wiki is throwing errors during edit commit

2012-08-27 Thread Scott McManus
I found that the problem happened if I used the UI for editing text. What I did instead was edit the wiki using the text-based mode instead. If you have set your preferences to use the UI for editing by default, then you will need to change it back to use the text-based editing instead; I foun

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Scott McManus
I'll check it out. Thanks. - Original Message - > Hi all (and in particular, Scott), > > I've just updated my development server and found the following > error when running jobs on our SGE cluster via DRMMA: > > galaxy.jobs.runners.drmaa ERROR 2012-09-18 09:43:20,698 Job wrapper > fini

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Scott McManus
I have to admit that I'm a little confused as to why you would be getting this error at all - the "job" variable is introduced at line 298 in the same file, and it's used as the last variable to check_tool_output in the changeset you pointed to. (Also, thanks for pointing to it - that made inves

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Scott McManus
le in my recently merged > code > either (line ~1045), while JobWrapper's does around line 315. > > cheers, > jorrit > > > > > On 09/18/2012 03:55 PM, Scott McManus wrote: > > I have to admit that I'm a little confused as to why you would > >

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Scott McManus
Ok - that change was made. The difference is that the change is applied to the task instead of the job. It's in changeset 7713:bfd10aa67c78, and it ran successfully in my environments on local, pbs, and drmaa runners. Let me know if there are any problems. Thanks again for your patience. -Scott

Re: [galaxy-dev] DRMAA: TypeError: check_tool_output() takes exactly 5 arguments (4 given)

2012-09-18 Thread Scott McManus
Sorry - that's changeset 7714:3f12146d6d81 -Scott - Original Message - > > Ok - that change was made. The difference is that the change > is applied to the task instead of the job. It's in changeset > 7713:bfd10aa67c78, and it ran successfully in my environments > on local, pbs, and drma

Re: [galaxy-dev] Setting error levels in

2012-09-20 Thread Scott McManus
You should be able to use the following: Square brackets are not supported in the range. However, note that many shells will not return an exit code that is less than 0. Instead, the lowest 8 bits (i.e., value mod 256) will be returned. So returning -1 will give you -1 mod 256 = 255, r

Re: [galaxy-dev] Setting error levels in

2012-09-20 Thread Scott McManus
My apologies: Again, note that the second range will not be useful as OS X and Linux (or at least the distros I've used) will never return a value less than 0. Sorry. -Scott - Original Message - > > You should be able to use the following: > > > > > > > Square bracke

Re: [galaxy-dev] Setting error levels in

2012-09-20 Thread Scott McManus
The wiki markup was also fixed. The "//" is not supported. -Scott - Original Message - > My apologies: > > > > > > > Again, note that the second range will not be useful as OS X and > Linux (or at least the distros I've used) will never return a value > less than 0. > > Sorry.

Re: [galaxy-dev] Improved error logging in TaskWrapper

2012-09-21 Thread Scott McManus
Thanks, Peter! Those are good suggestions. I'll look into it soon. -Scott - Original Message - > Hi all, > > I've been running into some sporadic errors on our Cluster while > using the latest development Galaxy, and the error handling has > made this quite difficult to diagnose. > > F

Re: [galaxy-dev] Galaxy not recording return codes (error levels)

2012-09-21 Thread Scott McManus
Just to be clear, whenever a tool's rules are triggered there should be messages prepended to stdout and/or stderr. If a tool defines a regular expression and the regular expression matches on stdout, a message should be prepended to stdout. However, these will be empty if tools without section

Re: [galaxy-dev] Using an unique database for tow differents versions of Galaxy

2012-09-21 Thread Scott McManus
Maybe a better question is why you would want two separate versions of galaxy running to begin with. There could be another way to solve your problem. -Scott - Original Message - > The main problem with different versions of galaxy is that the > database schema that they expect (as appli

Re: [galaxy-dev] Galaxy not recording return codes (error levels)

2012-09-21 Thread Scott McManus
Excellent. :) I had considered adding an exit code column but had avoided touching the database at first. Migration could be a little tricky. If we assume that there are few (or even zero) tools checking stdout/stderr/exit codes, then we could assume that a job failed if it had any stderr text

Re: [galaxy-dev] Galaxy not recording return codes (error levels)

2012-09-21 Thread Scott McManus
Good - after taking a step back I thought it seemed better to leave everything null rather than guess at an exit code. Does anyone have any objections to adding an exit code column with null values for previously-executed jobs? -Scott - Original Message - > Or allow the error code f

Re: [galaxy-dev] How to rotate Galaxy log file

2012-09-25 Thread Scott McManus
Lukasz- How are you stopping the process? It's possible that the python task for galaxy is still running. I would recommend trying to send a "kill" signal to that process. Something like "kill -9 PID", where PID is the process id for galaxy, should work. -Scott - Original Message - > O

Re: [galaxy-dev] unhandled exception

2012-10-03 Thread Scott McManus
The galaxy-bugs mailing list is a better destination for bugs. I'm looking at that code right now, so I'll poke around and see what's wrong. (I'll move this over to galaxy-bugs, too.) -Scott - Original Message - > (with galaxy-central from a few days ago) > URL: http://galaxy/data

Re: [galaxy-dev] "No module named controllers.library_common" error on fresh Galaxy install from 4-Oct-2012

2012-10-05 Thread Scott McManus
That's been fixed in a galaxy-central commit last night; the webapps were moved around, and this didn't point to the new webapps directory. See changeset 210c39f4bf7f or later. In particular, the problem you're seeing had to do with the templates/library/common/library_common.mako pointing t

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

2012-10-18 Thread Scott McManus
This was fixed in galaxy-central last week and should be part of galaxy-dist. What happened is that controllers were migrated and a couple of modules were left out. See changeset 210c39f4bf7f or later, which I believe was fixed on October 4th. In particular, the problem you're seeing had to do w

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

2012-10-18 Thread Scott McManus
Excuse me - I meant to say that this will part of the upcoming galaxy-dist. -Scott - Original Message - > > This was fixed in galaxy-central last week and should be part of > galaxy-dist. What happened is that controllers were migrated and a > couple of modules were left out. See chang

Re: [galaxy-dev] Corner case in task splitter - merging zero files

2012-10-18 Thread Scott McManus
Hey Peter- Thanks - I'll look into it. If you're able to reproduce the problem easily and wouldn't mind crafting a pull request, then it would be much appreciated. Otherwise I'll put this on my to-do list to be done soon. I or someone else may want to revisit the exception handling to prevent th

Re: [galaxy-dev] Corner case in task splitter - merging zero files

2012-10-19 Thread Scott McManus
Thanks, Peter! I'll get to it this afternoon EDT. -Scott - Original Message - > On Thu, Oct 18, 2012 at 5:19 PM, Scott McManus > wrote: > > > > Hey Peter- > > > > Thanks - I'll look into it. If you're able to reproduce the problem >

Re: [galaxy-dev] Corner case in task splitter - merging zero files

2012-10-19 Thread Scott McManus
t; > On Thu, Oct 18, 2012 at 5:19 PM, Scott McManus > > wrote: > > > > > > Hey Peter- > > > > > > Thanks - I'll look into it. If you're able to reproduce the > > > problem > > > easily > > > and wouldn't

Re: [galaxy-dev] Failed to run display.py

2012-10-23 Thread Scott McManus
Hi Tom- Are you still having this problem? If so, then could you try your request again and return what you're seeing from the console's logging output for that request? If not, then anything that you may have done to get around this problem would be appreciated. -Scott - Original Mes

Re: [galaxy-dev] How to add the option "-c enabled" to the qsub command?

2012-11-05 Thread Scott McManus
I recommend checking out John Chilton's dynamic job runner first: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010080.html -Scott - Original Message - > Hi, guys, > I would like to enable checkpoint and restart for some galaxy jobs. > In order to do that, I need to add "-c

Re: [galaxy-dev] Think I found a bug

2012-11-06 Thread Scott McManus
That sounds perfectly reasonable. I'll make the change. Thanks, Juan! -Scott - Original Message - > Hi galaxy devs, > We are in the final stages of deploying a new Galaxy instance. We > downloaded the galaxy-dist version a month ago aprox. We realized > that, when launching a job an

Re: [galaxy-dev] Syntax error (!) in latest pull?!

2012-11-06 Thread Scott McManus
Well, lib/galaxy/datatypes/registry.py hasn't changed in over two months. Out of curiousity, I tried downloading a fresh copy of galaxy-central (without any initial configuration or database, either) and didn't encounter the same problem. I even tried messing with that line of code and didn't

Re: [galaxy-dev] Think I found a bug

2012-11-07 Thread Scott McManus
It should have been fixed yesterday - if a job is running locally and the user cancels it, then the job will be stopped immediately. Thanks, Juan! -Scott - Original Message - > Ok, thanks for the response! > On 6 November 2012 12:50, Dannon Baker < dannonba...@me.com > wrote: > >

Re: [galaxy-dev] Can't edit galaxy workflow

2012-11-14 Thread Scott McManus
Make sure that you are pulling from galaxy-central and not galaxy-dist. I believe something like: hg pull -r rev# -u http://bitbucket.org/galaxy/galaxy-central should work. The problem may be that you are pointing to galaxy-dist by default, which will not know about that revision (yet). -Scott

Re: [galaxy-dev] Can't edit galaxy workflow

2012-11-26 Thread Scott McManus
Bitbucket will only show the first 7 characters when browsing through commits. Try this: https://bitbucket.org/galaxy/galaxy-central/changeset/5013377e0bf7a656ea593098f1d1b38f3d6928c6 -Scott - Original Message - > Hi, > Can you please send the entire link to access this changeset? I