Re: [galaxy-dev] writing datatypes

2014-07-17 Thread John Chilton
Even more out of office than normal so maybe I don't have the throughput to process this but it sounds like it won't work then. If the new types aren't going to be loaded than we cannot evolve the datatypes with new functionality in new repositories. Perhaps I am missing something, but in the abs

Re: [galaxy-dev] datatype dependencies

2014-07-18 Thread John Chilton
My understanding of the code is that tool shed dependencies (or local dependencies) will not be available to tool shed datatypes (for sniffing for instance). Sorry. If you want to hack up your local instance to resolve dependencies during the sniffing process that may be possible - my guess is you

Re: [galaxy-dev] API v/s twill based testing

2014-07-18 Thread John Chilton
I think the confusion over two different test methods has reached a high level and Dave and I agreed previously that we should try to make the API driven version the default during this release cycle - but it is not our call :). Greg - is there anything else you feel that needs to happen to at lea

Re: [galaxy-dev] writing datatypes

2014-07-18 Thread John Chilton
Does the current implementation really handle datatypes in reproducible manner - if I have a repo which in revision 1 defines foo1 as a text subtype, foo2 as a tabular type and foo3 as a new type in foo.py and then in revision 2 foo1 is defined as a binary subtype , foo2 and foo3 disappear and foo4

Re: [galaxy-dev] Once-run galaxy archives

2014-07-22 Thread John Chilton
On Tue, Jul 22, 2014 at 7:51 AM, Peter Cock wrote: > On Tue, Jul 22, 2014 at 1:15 PM, Eric Rasche wrote: >> Hi Peter, >> >> On July 22, 2014 3:15:41 AM CDT, Peter Cock >> wrote: >>> >>>Given how close you can get now for minimal effort, >>>this seem unnecessary. >>> >>>http://blastedbio.blogspo

Re: [galaxy-dev] Run Jobs as Real User - How to Configure it for TORQUE

2014-07-22 Thread John Chilton
Running jobs as the "real" user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user. I have created a Trello card to add this functionality: https://trello.com/c/OddS8bMP Would be happy to field pull requests to add this - because I doubt

Re: [galaxy-dev] getting API key from user/pass

2014-07-22 Thread John Chilton
Hey Eric, This is not possible (easily anyway) in the latest release of blend4j. It was on the TODO list though so I have added the functionality in the following commit: https://github.com/jmchilton/blend4j/commit/f92909fbda3616da09614b65810ebd86ce496b19 So instead of using GalaxyInstanceFactor

Re: [galaxy-dev] getting API key from user/pass

2014-07-22 Thread John Chilton
the time being. Hmm... that is too bad. They should do that - seems like Android would be a key platform for a web service client library. Oh well. -John > > Thanks for being aware of this issue. > > Cheers, > Eric > > 22.07.2014, 21:10, "John Chilton" : >> Hey

Re: [galaxy-dev] Galaxy error

2014-07-22 Thread John Chilton
Hello, Thanks for the bug report. Is this on the main public server (usegalaxy.org) or a local instance at the University of Iowa? -John On Wed, Jul 16, 2014 at 10:39 AM, Beck, Emily A wrote: > > Hello, > > I have repeatedly gotten the following error message when attempting to > use both the

Re: [galaxy-dev] Once-run galaxy archives

2014-07-23 Thread John Chilton
Problem with automation is I could create dozens of templates over the next several years and consume less time in aggregate than it would take me to automate this. Nonetheless, there is a documentation component here that is important so I did enough to document - if someone wants to automate from

Re: [galaxy-dev] bioblend problem trying to use run_tool function

2014-07-24 Thread John Chilton
there but no values are there. > > Could you give me a use case, so I can mime it to my project ? > > Thanks in advance, > -- > Olivia Doppelt-Azeroual > CIB, Institut Pasteur, Paris > > Le 27/02/14 14:46, John Chilton a écrit : > >> I think you are right

Re: [galaxy-dev] How can I populate a data_collection output ?

2014-07-29 Thread John Chilton
Hello Julie, Right now Galaxy can implicitly create collections by running a tool that takes in a single input or a paired dataset over say a list of inputs or a list of paired inputs but tools cannot explicitly create dataset collections. I think Galaxy's current capabilities add a lot of usefu

Re: [galaxy-dev] Uploads with embedded citations causing red error on Tool Shed

2014-07-30 Thread John Chilton
Hey Peter, Opps sorry about that and thanks for the bug report. The tool shed code should be fixed with https://bitbucket.org/galaxy/galaxy-central/commits/38ba45d6ba5be65b3b743fc08739e16cd6e0ac8f - it is in next-stable so I think the tool shed should pick up that fix at next tool shed update. -J

Re: [galaxy-dev] Uploads with embedded citations causing red error on Tool Shed

2014-07-30 Thread John Chilton
ing > my tool once the fix is live on the Tool Shed? > > i.e. Was it a harmless warning? > > Peter > > > On Wed, Jul 30, 2014 at 12:12 PM, John Chilton wrote: >> Hey Peter, >> >> Opps sorry about that and thanks for the bug report. The tool shed >> cod

Re: [galaxy-dev] How can I populate a data_collection output ?

2014-07-30 Thread John Chilton
, Jul 29, 2014 at 4:43 PM, David Kelly wrote: > Hi John, > > Is there any documentation that explains how dataset collections work? I'd > like to learn more about it. > > Thanks, > David > > > On Tue, Jul 29, 2014 at 3:20 PM, John Chilton wrote: >> >

Re: [galaxy-dev] Secure Passwords

2014-07-30 Thread John Chilton
I don't believe this is possible and I am aware of not active plans to work on this. But if you are willing to hack on Galaxy I think you could enforce arbitrary constraints pretty easily (if you know Python) by modifying the validate_password function in lib/galaxy/security/validate_user_input.py.

Re: [galaxy-dev] Providing BLAST db in a data library

2014-07-30 Thread John Chilton
Thanks for tracking down the problem - it sounds like it is a Galaxy bug then so I have created a Trello card (https://trello.com/c/bNEKfOWR). -John On Wed, Jul 30, 2014 at 7:06 AM, Peter Cock wrote: > On Wed, Jul 30, 2014 at 11:52 AM, Ulf Schaefer > wrote: >> Dear Nate, dear Peter >> >> Again

Re: [galaxy-dev] TestToolShed failure, Exception: History in error state.

2014-08-01 Thread John Chilton
On Thu, Jul 31, 2014 at 12:29 PM, Peter Cock wrote: > On Thu, Jul 31, 2014 at 5:21 PM, bjoern.gruen...@googlemail.com > wrote: >> Hi Peter, >> >> >> 2014-07-31 10:57 GMT+02:00 Peter Cock : >> >>> Hi Dave, >>> >>> You are right that on closer inspection I've mixed tool_dependencies.xml >>> and rep

Re: [galaxy-dev] new_file_path questions

2014-08-01 Thread John Chilton
On Fri, Aug 1, 2014 at 4:16 AM, Geert Vandeweyer wrote: > Hi, > > I'm running into issues with our galaxy installation regarding the > new_file_path setting: > > 1/ clean up : > > Can I safely delete all contents in the tmp folder using a cronjob, or > manually when galaxy is (not) runnin

Re: [galaxy-dev] error using get( $__user_id__ ) in xml file in new Galaxy

2014-08-05 Thread John Chilton
Hmm... The following should work for the last at least three releases I think: ${__user__.api_keys[0].key}. Not sure why get/find are unavailable on the User model class in the cheetah template now. I think this version is a little better. Is this okay? If not I can create a Trello card to investi

Re: [galaxy-dev] job not starting on cluster

2014-08-05 Thread John Chilton
Nothing comes to mind immediately, can you post a link to your job_conf.xml file? What is your `cleanup_job` option set to in universe_wsgi.ini? Can you try changing it to 'never' and see if the jobs continue? -John On Tue, Aug 5, 2014 at 5:54 PM, Langhorst, Brad wrote: > I’m not aware of any r

Re: [galaxy-dev] Galaxy Security Vulnerability

2014-08-05 Thread John Chilton
This bug predates Galaxy's transition to mercurial - so I would definitely encourage applying the patch by hand and restarting Galaxy. -John On Tue, Aug 5, 2014 at 9:13 PM, Dooley, Damion wrote: > Does this apply to all past galaxy installs? I have an older galaxy site > I've been wanting to p

Re: [galaxy-dev] condor compute cluster

2014-08-06 Thread John Chilton
Jobs run as the condor user? That is surprising - Galaxy isn't running as the condor user is it? Is your intention to have the jobs run as the 'galaxy' user or to run as the LDAP authenticated end-user submitting the job? If it is to run as the Galaxy user - I think it would be best to have condo

Re: [galaxy-dev] bug?

2014-08-10 Thread John Chilton
Doubt this is job metrics related - fairly certain the problematic line is: https://bitbucket.org/galaxy/galaxy-central/src/default/lib/galaxy/webapps/galaxy/controllers/dataset.py?at=default#cl-112 It tried to determine if a dataset id is encoded or not by hitting the database. It could possible

Re: [galaxy-dev] Galaxy error

2014-08-11 Thread John Chilton
City, IA 52242 > Lab: (319)335-3430 > ---------- > *From:* John Chilton [jmchil...@gmail.com] > *Sent:* Tuesday, July 22, 2014 4:06 PM > *To:* Beck, Emily A > *Cc:* galaxy-...@bx.psu.edu > *Subject:* Re: [galaxy-dev] Galaxy error > > Hello, > > Thanks for the bug report. Is thi

Re: [galaxy-dev] Bug report: within

2014-08-12 Thread John Chilton
I have applied your patch here (https://bitbucket.org/galaxy/galaxy-central/commits/19309418c86450b7f39691460eb430e100883cb4) and one correction to James' previous comment - multi-page tools actually do work through the API - this was required for multi-page tool testing via the API. -John On Tue

Re: [galaxy-dev] TypeError with 'dict'

2014-08-12 Thread John Chilton
Nate is right - full traceback would help. I am not sure it is the only problem - but these outputs do not look right: Galaxy isn't going to expand variables ($Project.input / $Project.samples) in the from_work_dir attribute so those are going to be taken as literal strings. That dire

Re: [galaxy-dev] Installed tool invisible in Galaxy

2014-08-12 Thread John Chilton
Hey Peter, When you are deleting and reinstalling the tool - can you confirm for me that you are definitely deleting the tool and not just deactivating it? I don't know where to hack around from there - I think repositories are tracked in the database (tool_shed_repository) table, in shed_tool_co

Re: [galaxy-dev] Determining datatype inheritance in tool XML Cheetah

2014-08-12 Thread John Chilton
Fun question! I have opened a pull request with my answer - https://bitbucket.org/galaxy/galaxy-central/pull-request/457/allow-cheetah-tool-templates-to-reason/diff. There are three different hacks you can use right now... here is a diff against tools/filters/catWrapper.xml I was using the to test

Re: [galaxy-dev] FW: Problems uploading files to a data library

2014-08-12 Thread John Chilton
You don't need a tool to upload - you need 'upload1' (i.e. tools/data_source/upload.xml) and it cannot come from the tool shed - this will make its id something other than upload1. You can make some modifications to it, relocate it, etc... but that needs to be there in some form. In universe_wsgi.

Re: [galaxy-dev] Installed tool invisible in Galaxy

2014-08-12 Thread John Chilton
along that chain to remove an entry from > tool_shed_repository. > > The tool and its type dependency are attached. > > On 12/08/2014 16:43, John Chilton wrote: >> Hey Peter, >> >> When you are deleting and reinstalling the tool - can you confirm for >> me that

Re: [galaxy-dev] Problem selecting datasets with a specified datatype

2014-08-13 Thread John Chilton
When Galaxy cannot find the extension/short name corresponding to a datatype it treats the input as a 'data' the root datatype meaning anything can connect to it. So I think the problem here is not so much your tool format as your datatype - I don't think Galaxy can see the lefse_res datatype. You

Re: [galaxy-dev] Internal Server Error when trying to install a tool from the Tool shed

2014-08-13 Thread John Chilton
I am not sure what the problem is - more details would really help. For instance - What version of Galaxy are you using? What repository are you trying to install? Do other repositories work? Does the Galaxy log have a more complete stack trace? Any additional details would be most helpful. Thank

Re: [galaxy-dev] unable to stob jobs

2014-08-13 Thread John Chilton
When was the last time you updated Galaxy - we fixed a bug that would be causing this behavior several releases ago. Otherwise - additional information would help - like are you using the DRMAA job runner or the newer specialized Slurm job runner and are there any details in the Galaxy log that mi

Re: [galaxy-dev] Upload file not working with new install

2014-08-15 Thread John Chilton
Okay - I just checked out a clean Galaxy without modifications and I don't see any problems with uploads. So this is likely something to do with your setup - if I had to guess I would guess that your job_conf.xml configuration is somehow incorrect - the job is getting creating but nothing is runnin

Re: [galaxy-dev] How set the working and temp directories in external drive

2014-08-15 Thread John Chilton
So you want temp directories and working directories to sit on the external drive but not Galaxy's files: Lets say this external disk is mounted as /mnt/gxwork. I would create two directories under that maybe /mnt/gxwork/tmp and /mnt/gxwork/working. Setting the working directory is the easier par

Re: [galaxy-dev] Problem selecting datasets with a specified datatype

2014-08-15 Thread John Chilton
efse program, it > generates output of the type lefse_res. > > So why > > > > > >label="Input 2"/> > > > > does not filter only those datasets? > > Thanks!! > > George > > > > > > On Wed, Aug 13, 2014 at 1:33 P

Re: [galaxy-dev] unable to stob jobs

2014-08-15 Thread John Chilton
I am using Galaxy at an all out > pace and I was able to kill jobs this morning. > Thanks again. > Mike > > On 8/15/14 9:32 AM, "John Chilton" wrote: > >>Very interesting - whenever you get a chance I would try to kill >>simple jobs when Galaxy is not under l

Re: [galaxy-dev] Internal Server Error when trying to install a tool from the Tool shed

2014-08-15 Thread John Chilton
hg update latest_2014.08.11 To make sure you have the latest and greatest Galaxy and then retry? If you still have problems and it is the same strack trace - please let me know. Next up would be to eliminate the proxy theory - not sure how to do that though :(. -John On Wed, Aug 13, 2014 at 4:3

Re: [galaxy-dev] TypeError with 'dict'

2014-08-15 Thread John Chilton
Hello Martin, You have found something of a bug - there are problems when the output name matches an input name - in this case your conditional name "Project" matches your output name "Project" . I think you can just fix this by calling the output Project_output or the conditional which_project or

Re: [galaxy-dev] Upload file not working with new install

2014-08-18 Thread John Chilton
for submit failed: 15007: No permission > > > Iry > > On 8/15/14 12:11 PM, "John Chilton" wrote: > >>Okay - I just checked out a clean Galaxy without modifications and I >>don't see any problems with uploads. So this is likely something to do >>with

Re: [galaxy-dev] Internal Server Error when trying to install a tool from the Tool shed

2014-08-19 Thread John Chilton
new install on one of >> our other servers and see if the problem occurs there or not. >> >> Will update you when we do this. >> >> Thanks. >> Audrey >> >> >> On Fri, Aug 15, 2014 at 8:20 PM, John Chilton wrote: >>> >>> Okay - I am

Re: [galaxy-dev] redirecting to unnamed history

2014-08-19 Thread John Chilton
Interesting. I have never seen that behavior before - any chance you can try this patch and see if it fixes the problem? https://gist.github.com/jmchilton/b9ae7588de4b21e59cec -John On Tue, Aug 19, 2014 at 1:24 PM, Michael Mason wrote: > Hi I am running a work flow on data in a history with 54

Re: [galaxy-dev] How to build an application that uses a set of files?

2014-08-21 Thread John Chilton
Are these 17 files fairly dynamic - i.e. are they going to depend on previous steps in a history or user uploads - or are they literally just 17 more or less fixed files associated with that tool and every user should see roughly the same 17 options. -John On Thu, Aug 21, 2014 at 3:03 PM, George

Re: [galaxy-dev] How to build an application that uses a set of files?

2014-08-21 Thread John Chilton
dynamic. > Say all the files of tabular type that the User has at the moment. > > > On Thu, Aug 21, 2014 at 12:28 PM, John Chilton wrote: >> >> Are these 17 files fairly dynamic - i.e. are they going to depend on >> previous steps in a history or user uploads - or

Re: [galaxy-dev] Problems downloading files via bioblend

2014-08-22 Thread John Chilton
Yes sorry about this - Eric reported this same problem this week. bioblend is not using API endpoints to download datasets and this is a problem for proxies with REMOTE_USER configured since most such proxies are only configured to allow through requests on /api and this is definitely not the route

Re: [galaxy-dev] problems with database migration 119 -> 120

2014-08-26 Thread John Chilton
Well it looks like the migration file has these columns listed in a different order than the mapping Galaxy uses - and the order yours appeared in were the ones from Galaxy's mapping file. So somehow Galaxy is automatically creating those tables prior to running the migration based on the code in G

Re: [galaxy-dev] Galaxy and object stores

2014-08-26 Thread John Chilton
Thanks Enis, just to elaborate on Pulsar - I suspect it would work with something like configuring Galaxy with S3 object store right now - but it would do so by having Galaxy cache the data locally and then Pulsar would negotiate the transfer with Galaxy (many different ways this could occur depend

Re: [galaxy-dev] Grouping of tools inside of toolbox does not work

2014-08-28 Thread John Chilton
Thanks for the bug report. I would say that sounds like it should be easy to fix - but the last time there was a tool grouping bug - it took multiple of us hours (days in some cases I think) of investigation to figure it out - so I have created a Trello card to track progress on this https://trello

Re: [galaxy-dev] "when else" in ? RE: refresh_on_change : is this a valid attribute? Any other ideas/options??

2014-09-02 Thread John Chilton
Galaxy's handling of dynamic options has some restrictions for sure outlined here and it is true that conditional test parameters cannot be set at runtime in workflow execution (the workflow form doesn't have the ability to restructure itself based on the conditional the way the tool form does). A

Re: [galaxy-dev] Error with functional tests on cluster

2014-09-02 Thread John Chilton
This looks like some script is correct locally but not on the cluster. It looks like the script has some error (like are modules available locally but not on the cluster). At any rate this error is being reported on standard error and the script is continuing to run (hence the datasets being writte

Re: [galaxy-dev] API for multi-file upload

2014-09-02 Thread John Chilton
Hello Neil, The newer uploader targets the tool's API with in browser JavaScript - so bioblend say targets the same API with its tool client (https://github.com/afgane/bioblend/blob/master/bioblend/galaxy/tools/__init__.py#L104). I imagine you could get a similar degree of parallelism by splitti

Re: [galaxy-dev] directory as an input file

2014-09-03 Thread John Chilton
Collection are one potential answer for how users can specify the set of stuff that belongs in the directory. For explicitly dealing with applications that consume directories - I think it is best to just create the directory and link in files (if possible) before the tool runs. mkdir input_dir; #

Re: [galaxy-dev] Error executing workflow via API

2014-09-08 Thread John Chilton
There are definitely limitations to the size of workflows that can be executed right now but I feel like that problem should be getting better not worse so this is a little confusing. Did something besides the Galaxy version change (like proxy settings, timeouts, etc...?). The client side error in

Re: [galaxy-dev] Keeping Galaxy up to date

2014-09-09 Thread John Chilton
Hello Thomas, After public debate on Trello, there have been a couple tools that have indeed disappeared completely but nearly all tools that have gone away have been migrated to the tool shed and can be recovered with the migration scripts that are included with each new release (https://wiki.g

Re: [galaxy-dev] How to escape % characters in tool parameters?

2014-09-09 Thread John Chilton
Funny - I just exchanged an e-mail with someone about this tool this morning, the Galaxy-P dbbuilder tool escapes exactly this santizing. Here is the relevant portion: The full tool is on the tool shed. https://tool

Re: [galaxy-dev] Workflow step IDs: out of order?

2014-09-15 Thread John Chilton
I am not certain order_index solves all of the problems but I agree it is a step forward and in the latest galaxy distribution the default is to use 'order_index' instead of the old database id *IF* instead of specifying inputs via the 'ds_map' parameter you specify the parameter as 'inputs'. ('inp

Re: [galaxy-dev] API : Max retries exceeded

2014-09-16 Thread John Chilton
Is there some small difference between your test and production instance? It seems like you client is trying to connect to port 8080 in production? Is this accessible? The error makes me think this is not a Galaxy problem but a firewall or proxy issue. Is your production instance served out on port

Re: [galaxy-dev] Problem writing functional tool tests with 'repeat' & 'conditional' tags

2014-09-17 Thread John Chilton
Hello Peter, For both of your questions it would be very helpful to know what version of Galaxy you are using (which tagged version or output of hg id -i). I noticed you have min on your repeat - this did not work for the older Twill test runner historically (and may still not) but it isn't a

Re: [galaxy-dev] Error report via API?

2014-09-25 Thread John Chilton
I don't believe this is currently available via the API - but we would like to move everything to the API so I don't see why this wouldn't be included (maybe with some rate limit-ting :)). I have created a Trello card: https://trello.com/c/W2PEpe8V This could potentially be fairly straight forwar

Re: [galaxy-dev] Question re: email alerts on workflow errors

2014-09-25 Thread John Chilton
I don't think anything like this is currently possible - we are working on some structural changes to workflow evaluation that should allow for the addition of stuff like this more easily (right now after all the jobs are queued up by Galaxy - to some large approximation Galaxy doesn't really reaso

Re: [galaxy-dev] sendmail from addr

2014-09-25 Thread John Chilton
Hey Evan, The following diff *should* add a new config option in universe_wsgi.ini called no_reply_email_from that can be used to set from when sending these activation emails. If you are feeling brave, try it out and let me know how it goes. I have not recreated this problem (read I have not test

Re: [galaxy-dev] Internal Server error when attemptng to use Admin Tab

2014-10-01 Thread John Chilton
Hello David, We don't have a workaround yet - but we (Dan did most of the investigating) understand the problem I think. You are using a MySQL database I assume? (Ignore the rest of this e-mail if not). I think MySQL has lower limits on the size of BLOBs that can be stored in the database than pos

Re: [galaxy-dev] Get Data - url upload (connection timed out)

2014-10-01 Thread John Chilton
I doubt there is a config option somewhere that is going to fix this. Can your compute nodes download external data outside of Galaxy? (One could submit a test script to the cluster with a download and see if it succeeds.) If you give me the URL and let me know what kind of cluster you are running

Re: [galaxy-dev] Login with a system user

2014-10-01 Thread John Chilton
I don't know how one would do this precisely but I believe it is possible at the proxy level - i.e. setting up Apache to handle this and authenticate with system resources. I believe Tim Booth's work on integrating galaxy into biolinux uses system users for authentication by default. The following

Re: [galaxy-dev] 'KeyError' in workflow extraction when using newer tool version

2014-10-01 Thread John Chilton
Sorry this has been sitting in my inbox for a while - because I wanted to try to fix it before the next release but I don't think it is going to happen. Generically - it doesn't surprise me that Galaxy can handle certain parameters changing between versions but not conditionals - they are handled

Re: [galaxy-dev] Simple standard for API use of a global user/key that all loaded tools can draw upon?

2014-10-01 Thread John Chilton
I don't know about a standard way of doing - I am not sure about Galaxy officially supporting this but you can add whatever properties you want in universe_wsgi.ini (which will be renamed config/galaxy.ini in coming release). These can be accessed within tool cheetah blocks as $__app__.config.confi

Re: [galaxy-dev] Internal Server error when attemptng to use Admin Tab

2014-10-02 Thread John Chilton
> As it happens, we also had a few cases with multiple instances of tool-shed > tools being installed. Since this server is brand new, it was fine to delete > them. I suppose that maybe we are now fitting in MySQL BLOBs now?? > > thanks again - > David > > > > On

Re: [galaxy-dev] Simple standard for API use of a global user/key that all loaded tools can draw upon?

2014-10-02 Thread John Chilton
at you would intend to do). Thanks. -John On Wed, Oct 1, 2014 at 1:26 PM, John Chilton wrote: > I don't know about a standard way of doing - I am not sure about > Galaxy officially supporting this but you can add whatever properties > you want in universe_wsgi.ini (which will b

Re: [galaxy-dev] Job wrapper object

2014-10-06 Thread John Chilton
Yes - I am with Iyad - having more context would probably help :). You can inject job_wrapper related stuff into tool execution in a very round about way perhaps by creating a dynamic job destination that consumes the job_wrapper and then creates a job destination with enviornment variables set to

Re: [galaxy-dev] no universe_wsgi.ini & can't install tools from the standard toolshed because: "Initializing repository installation failed"

2014-10-09 Thread John Chilton
Thanks for the screenshot. That page will often and regrettably report errors when there are none. I think the advice I have been given by people who know the tool shed better is to click install and then leave that page and go to the list of installed repositories to track progress. Did that sam

Re: [galaxy-dev] Download tool-dependency with using a proxy and functional tests http 403 error

2014-10-09 Thread John Chilton
Hi Marcus, There was some discussion of proxies a while ago - and it looks like people were able to get it to work, does this advice help any? http://dev.list.galaxyproject.org/How-to-install-tool-shed-behind-proxy-with-authentication-td4663218.html As for the install and test framework - I am p

Re: [galaxy-dev] HOWTO share tool parameter settings?

2014-10-09 Thread John Chilton
How would this be different then the third option you listed? You want it to work with all tools and you as the developer want to be able to construct these files without needing a dummy tool to produce the values? How would imagine these setting files would be disseminated to users and then sele

Re: [galaxy-dev] failure to export workflows

2014-10-12 Thread John Chilton
I was unable to reproduce this problem. Can you supply more details - what path did you take through the UI to get that URL? Is that restricted to just some workflows or do all workflows have this problem as far as you can tell? I guess ultimately the reason that workflow isn't downloading is that

Re: [galaxy-dev] failure to export workflows

2014-10-13 Thread John Chilton
? Is that > not the case anymore ? > > Thanks again, > Wolfgang > > > On 10/13/2014 02:34 AM, John Chilton wrote: >> >> I was unable to reproduce this problem. Can you supply more details - >> what path did you take through the UI to get that URL? Is that >>

Re: [galaxy-dev] error running API with example_watch_folder.py

2014-10-14 Thread John Chilton
When you say "p.s. there is nothing in the log file."? Does you mean the Galaxy log file? If there are jobs running and completing there must be something in the log file right? Do you mean there are just not tracebacks indicating errors in the logs? My guess is the response is just taking a whil

Re: [galaxy-dev] UCSC Main table browser

2014-10-15 Thread John Chilton
I believe there is a fix in the latest update to Galaxy for this thanks to Nate. https://bitbucket.org/galaxy/galaxy-central/commits/579d211 The previous change to that file was in 2011 - so you are likely to be able to apply the commit to any release of Galaxy you may be running. -John On Tue,

Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path

2014-10-15 Thread John Chilton
JJ, Arg this is a mess. I am very sorry about this - I still don't understand extra_files_path versus files_path myself. There are open questions on Peter's blast repo and no one ever followed up on my object store questions about this with Bjoern's issues a couple release cycles ago. We need to g

Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path

2014-10-15 Thread John Chilton
thinking about this and respond again. -John On Wed, Oct 15, 2014 at 9:39 AM, John Chilton wrote: > JJ, > > Arg this is a mess. I am very sorry about this - I still don't > understand extra_files_path versus files_path myself. There are open > questions on Peter's b

Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path

2014-10-15 Thread John Chilton
r_only=True, > extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) > > I'm trying to see when self._extra_files_path gets set. Otherwise, would > this return the path relative to the current file location of dataset? > > > > > On 10/15/14, 9:36 AM, J

Re: [galaxy-dev] snpeff tool for Galaxy extra_files_path

2014-10-15 Thread John Chilton
On Wed, Oct 15, 2014 at 1:47 PM, Björn Grüning wrote: > Hi John, > > glad to see this gets some attention! > > Am 15.10.2014 um 19:05 schrieb John Chilton: >> Hey JJ, >> >> Opened a pull request to stable with my best guess at the right to >> proceed and h

Re: [galaxy-dev] Help with Galaxy server migration

2014-10-16 Thread John Chilton
Hmm hopefully someone more knowledgeable about the tool shed than me responds also but I had a couple quick thoughts. The first is a warning - workflows may break. At very least workflows that depend on the previous instance having gone through tool migrations instead of tool install. My under

Re: [galaxy-dev] question about GALAXY_SLOTS

2014-10-17 Thread John Chilton
In theory GALAXY_SLOTS is now always set unless Galaxy is pretty old - I still think it is important for tools to specify defaults using the \${GALAXY_SLOTS:-4} idiom so that tools themselves don't have explicit dependencies on the Galaxy runtime and in case new resource managers or job runners com

Re: [galaxy-dev] strange issue with .RData files

2014-10-22 Thread John Chilton
Hey Pieter, Sorry I am swamped right now so I don't have time to dig into this in detail - but I have encountered this before with datatypes that are compressed - zipped, gzipped, etc Galaxy will attempt to decompress them in order to figure out what they are. I believe this is what is happe

Re: [galaxy-dev] Determining galaxy api path automatically from a tool via dynamic_options?

2014-10-22 Thread John Chilton
Well host is probably not enough - it doesn't seem like a good idea to assume http right? There is the prefix and (trans.request.scheme) that could be missing. Is trans.request.base what you want? Maybe trans.request.path_url? It should be possible to recover that from trans I guess - though I don'

Re: [galaxy-dev] Citations tag not working

2014-10-24 Thread John Chilton
This feature was added with the August 2014 release and it looks like you are still on the June release (latest_2014.06.02). You will need to upgrade to the August or October release (remember to run tool migrations). (This feature touched a number of different components and so I don't have a smal

Re: [galaxy-dev] Dependencies not working on toolshed?

2014-10-24 Thread John Chilton
On Fri, Oct 24, 2014 at 10:18 AM, Lukasse, Pieter wrote: > Hi , > > > > I am trying to get a dependency to work in practice (when running a tool > after the installation). But I keep getting the following message when > running the tool: > > > > > > [sshexec] galaxy.tools.deps DEBUG 2014-10-24 1

Re: [galaxy-dev] Stop autoselecting in drop-down menus

2014-10-29 Thread John Chilton
On Wed, Oct 29, 2014 at 7:45 AM, Aleksey Jironkin < aleksey.jiron...@phe.gov.uk> wrote: > > Hi, > > I was wondering if there is a way to prevent auto-selecting an option > for Running a workflow? Quiet often users (especially novices) would > re-run workflows without checking much the data that goe

Re: [galaxy-dev] Test failure, JSONDecodeError: Unpaired high surrogate

2014-10-31 Thread John Chilton
Ugh... sorry I missed this e-mail. I don't have any answers yet but I have created a Trello card to track the issue. https://trello.com/c/3yVLp8RA Thanks for the detailed error report. -John On Fri, Oct 31, 2014 at 12:16 PM, Peter Cock wrote: > I have "solved" this by commenting out the appare

Re: [galaxy-dev] job_working / tmp folders

2014-11-02 Thread John Chilton
I am not aware of any other way new_file_path is accessed. Is it old jobs that were queued up prior to the switch? I think job_working_directory might actually be affected by the settings in your object store configuration. Have you configured an object store? In either case - knowing the specific

Re: [galaxy-dev] Unable to connect two tools in workflow editor

2014-11-02 Thread John Chilton
Can you review this dev thread http://dev.list.galaxyproject.org/Problem-selecting-datasets-with-a-specified-datatype-td4665366.html? In particular can you go to the "View data types registry" option in the admin menu and see if Galaxy thinks it knows about this datatype? Let me know if Galaxy thi

Re: [galaxy-dev] Configuring r script galaxy tool to install on testtoolshed with dependencies in the same repository

2014-11-02 Thread John Chilton
I think a working example that uses this pattern is Bjoern's glimmer tool: https://github.com/bgruening/galaxytools/tree/master/glimmer3 The only obvious difference I see that he sets the environment variable to $REPOSITORY_INSTALL_DIR in tool_dependencies.xml ($REPOSITORY_INSTALL_DIR). Does chan

Re: [galaxy-dev] bowtie2: no stderr output

2014-11-02 Thread John Chilton
It looks like this was a problem for awhile but isn't anymore on main https://trello.com/c/yFeXvhUg. It doesn't look like there has been any changes to the bowtie2 wrapper since it was migrated to the tool shed so I am not sure why it isn't working for you? As an admin can you open a successful b

Re: [galaxy-dev] error running cuffdiff

2014-11-02 Thread John Chilton
Hello Liz, I wish I had some better news - upgrading might help - we have definitely made modifications to that file since then - but not to address this issue I don't think. What would be really interesting is to see the Galaxy logs around the time of that error - I wonder if there would be som

Re: [galaxy-dev] Integrating Docker-based tools within Galaxy failed

2014-11-06 Thread John Chilton
I think you want your job_conf.xml destination to have the parameter true not true Does this help any? -John On Thu, Nov 6, 2014 at 1:35 AM, Weiyan Shen wrote: > Hi, > I integrated Docker-based tools‍ SMALT whin my local Galaxy by the > guid:https://github.com/apetkau/galaxy-hackathon-2014/t

Re: [galaxy-dev] ulimit problems after update

2014-11-10 Thread John Chilton
Hmm... this is probably a script that works fine on your login node but not on your worker nodes or vise versa? The fact that it is writing to standard error and happens for each new shell is probably what is causing Galaxy jobs to fail - Galaxy is thinking the underlying applications are writing c

Re: [galaxy-dev] Use of gzipped files in Galaxy Unit Tests

2014-11-10 Thread John Chilton
I have no issue with supporting this in general - but the implementation I think is a bit more tricky than it would seem. The test framework doesn't know if Galaxy would uncompress zipped files or not - I think the only way to reason about that in the abstract is to actually upload the file to Gala

Re: [galaxy-dev] Jobs stuck in "new" state - Data Library datasets to blame?

2014-11-10 Thread John Chilton
Hello Lance, I cannot think of a good way to rescue these jobs. If you are curious about the code where jobs are selected for execution - I would check out the job handler (lib/galaxy/jobs/handler.py) - see __monitor_step for instance. It seems like to prevent this from happening in the futur

Re: [galaxy-dev] Error in creating admin user during tool shed bootstrap

2014-11-13 Thread John Chilton
Hey Bruno, So Greg has moved on new exciting things and I am not sure if there is anyone who now routinely does the tool shed bootstrapping workflow that can help. I tried the bootstrapping process and it worked fine with an sqlite database - maybe it is not compatible with postgres? Want to

Re: [galaxy-dev] Dataset collection generation from a tool

2014-11-13 Thread John Chilton
Yup - sorry :(. Hopefully by the end of the year I will have something committed on that - it is a lot of work to do though. -John On Wed, Nov 12, 2014 at 4:25 PM, Aaron Petkau wrote: > I think I may have just found the answer to my question in this trello card > https://trello.com/c/ndVQmt3G/18

  1   2   3   4   5   >