Re: [galaxy-dev] Download URL not accepted in tool_dependencies.xml
Thanks, then I will run toolshed from a separate galaxy-central clone if that's advisable - different from the Galaxy service. I prefer to keep galaxy-dist on the stable branch. In this process, I will move the tools from the old toolshed to my new toolshed running from galaxy-central. When I have time, I will upload everything to the test-toolshed. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 05:11 PM, Bjoern Gruening wrote: Hi Joachim, I had such problems if my XML file is invalid. Than I get immediately the 'installed:' text and can click install again, without any change. Also I would really recommend to reset your toolshed. These white ghosts can have unpredictable effects. If you can upload your tools to the test tool shed I can try to install it debug it. Please make sure that you run the latest Galaxy central version. Greg and Dave invested a lot of time to get rid of white ghosts. But these changes are only in central. Cheers, Bjoern Still having a hard time and some strange situations. It is almost too much to report via email. Basically, the tool_dependencies.xml get recognised sometimes. Please, watch this video to see what I mean (1'30''), and pause as you like: https://dl.dropboxusercontent.com/u/18352887/toolshedissues.ogv I am afraid I cannot invest this amount of time to make my repo's public-proof. There are weird things happening which make no sense to me. And debugging of tool_dependencies.xml is very hard, since I find almost no logs of this process. Joachim. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 02:53 PM, Dave Bouvier wrote: Joachim, For sourceforge URLs, you can safely omit the question mark and everything after it, since that's part of the browser download load balancing process. When downloading with wget, curl, or python's urllib, downloads.sourceforge.net does its own behind-the-scenes load balancing. As for hyphens, I don't believe they need to be escaped, since they're not XML entities like the ampersand is. The following download_by_url action should work: action type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip/action --Dave B. On 09/30/2013 08:45 AM, Joachim Jacob | VIB | wrote: Hi all, Not fixed yet. Somehow the download does not work: I have escaped the ampersands and the hyphens. The download seems to have to proceed, since I receive no error from the download step. The next step in the dependencies file gives an error: mv: cannot stat transpose.zip': No such file or directory Cheers, J Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 02:06 PM, Peter Cock wrote: You must escape any ampersand in the XML as amp; and then it should work. Peter On Mon, Sep 30, 2013 at 12:51 PM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi all, The download URL seems not to be accepted in tool_dependencies.xml. It is an URL from sourceforge, the direct link to a package. The error upon uploading the tool to my toolshed: ** Metadata may have been defined for some items in revision '5665a799775d'. Correct the following problems if necessary and reset metadata. tool_dependencies.xml - Exception attempting to parse /mnt/toolsheddb/database/000/repo_26/tool_dependencies.xml: not well-formed (invalid token): line 6, column 149 ** The tool_dependencies.xml file: ** 1 ?xml version=1.0? 2 tool_dependency 3 package name=transpose version=2.0.0 4install version=1.0 5 actions 6 action type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip?r=ts=1380535239use_mirror=surfnet/action 7action type=shell_commandmkdir bin/action 8 action type=shell_commandunzip transpose-2.0.zip/action action type=shell_commandcd transpose-2.0/src/action action type=shell_commandgcc transpose.c -o transpose/action action type=move_file source$INSTALL_DIR/transpose-2.0/src/transpose/source destination$INSTALL_DIR/bin/destination /action action type=shell_commandchmod +x $INSTALL_DIR/bin/transpose/action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme Compiling transpose and putting in the path. /readme /package /tool_dependency ** Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your
Re: [galaxy-dev] multiple input files
Hi Ulf, What I do: 1. make a history, doing the steps you want to do on one input file 2. create a workflow of that history 3. assemble all input files in one history 4. run the workflow and select the multiple input files to run the workflow on. 5. Optionally: send the results to a new history, for every input file you will get a new history, properly named. Hope this helps, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 10/01/2013 02:27 PM, Ulf Schaefer wrote: Dear all We frequently find ourselves in situations where a tool needs to be run with a lot of input files. For example, run the GATK UnifiedGenotyper with easily dozens of bam files. Using the repeat in this case requires quite a bit of clicking. Is there a more conventient way fo doing this? Maybe similar to the multi-file-select that is possible for workflow inputs? I saw some older discussions on this or similar issues, but I am a bit lost what the current official stable proposed solution for this is. Thanks for your help Ulf ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] multiple input files
Sorry, my answer doesn't fit your question. :-) J Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 10/01/2013 03:08 PM, Joachim Jacob | VIB | wrote: Hi Ulf, What I do: 1. make a history, doing the steps you want to do on one input file 2. create a workflow of that history 3. assemble all input files in one history 4. run the workflow and select the multiple input files to run the workflow on. 5. Optionally: send the results to a new history, for every input file you will get a new history, properly named. Hope this helps, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 10/01/2013 02:27 PM, Ulf Schaefer wrote: Dear all We frequently find ourselves in situations where a tool needs to be run with a lot of input files. For example, run the GATK UnifiedGenotyper with easily dozens of bam files. Using the repeat in this case requires quite a bit of clicking. Is there a more conventient way fo doing this? Maybe similar to the multi-file-select that is possible for workflow inputs? I saw some older discussions on this or similar issues, but I am a bit lost what the current official stable proposed solution for this is. Thanks for your help Ulf ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Download URL not accepted in tool_dependencies.xml
Hi all, The download URL seems not to be accepted in tool_dependencies.xml. It is an URL from sourceforge, the direct link to a package. The error upon uploading the tool to my toolshed: ** Metadata may have been defined for some items in revision '5665a799775d'. Correct the following problems if necessary and reset metadata. tool_dependencies.xml - Exception attempting to parse /mnt/toolsheddb/database/000/repo_26/tool_dependencies.xml: not well-formed (invalid token): line 6, column 149 ** The tool_dependencies.xml file: ** 1 ?xml version=1.0? 2 tool_dependency 3 package name=transpose version=2.0.0 4install version=1.0 5 actions 6 action type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip?r=ts=1380535239use_mirror=surfnet/action 7action type=shell_commandmkdir bin/action 8 action type=shell_commandunzip transpose-2.0.zip/action action type=shell_commandcd transpose-2.0/src/action action type=shell_commandgcc transpose.c -o transpose/action action type=move_file source$INSTALL_DIR/transpose-2.0/src/transpose/source destination$INSTALL_DIR/bin/destination /action action type=shell_commandchmod +x $INSTALL_DIR/bin/transpose/action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme Compiling transpose and putting in the path. /readme /package /tool_dependency ** Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Messed up a tool installation from a local toolshed
Hi Greg, Sorry to dig up this unsolved thread from a few months back. Recap of the bug: I have developed a small tool. I have put it in my local Toolshed, and installed it. After applying some changes to the tool, I 'hg push'ed the changes to my repository on my toolshed. I uninstalled the tool, and reinstalled it. Since I work with job handlers, I have to restart Galaxy. Since I had about ~4 committed changes, I have repeated these steps about 4 times. The bug: the tool appears multiple times in the tool management window. 2 times as new, and 1 time as uninstalled (since I uninstalled it). The tool is still available in the toolbox, although it should not. The 2 installed tools have an option 'install', but when I click this I get an internal server error 500. Galaxy version: * [galaxy@galaxy galaxy-dist]$ hg tip changeset: 10411:c42567f43aa7 tag: tip user:greg date:Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy. * As per your request: the web log of this internal server error. 193.191.128.2 - - [30/Sep/2013:13:15:19 +0200] GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.bits.vib.be/repository_ids=968205d9ed31733dchangeset_revisions=e08a6d9e97a3 HTTP/1.1 500 - http://galaxy.bits.vib.be/admin_toolshed/browse_repositories; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/28.0.1500.71 Chrome/28.0.1500.71 Safari/537.36 Debug at: http://galaxy.bits.vib.be/_debug/view/1380535486 The postgres database shows: galaxydb= select id, tool_shed, name, owner, installed_changeset_revision, status, error_message from tool_shed_repository where name = 'transpose'; id | tool_shed | name| owner | installed_changeset_revision | status| error_message +--+---+-+--+-+--- 45 | toolshed.bits.vib.be | transpose | joachim | cee0157acee1 | Uninstalled | 44 | toolshed.bits.vib.be | transpose | joachim | 2fa0462b9303 | New | 43 | toolshed.bits.vib.be | transpose | joachim | e08a6d9e97a3 | New | (3 rows) How should I proceed ? This is the second time I experience this. On a last note: I have noted also that I managed to get a repository with different branches in this tool shed repository, by mixing 'hg push' and uploading a tarball. There seems no easy way to fixing this. Kind regards, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 06/14/2013 04:15 PM, Greg Von Kuster wrote: Hello Joachim, What version of Galaxy are your running? What does the tool shed paster go show for the 500 Internal Server Error? Also, run the following sql manually in your Galaxy database and send the results. select id, tool_shed, name, owner, installed_changeset_revision, status, error_message from tool_shed_repository; Thanks, Greg Von Kuster On Jun 14, 2013, at 3:41 AM, Joachim Jacob joachim.ja...@gmail.com mailto:joachim.ja...@gmail.com wrote: Hi all, I have developed a tool in our local Galaxy, and imported it in a local toolshed. I next removed the 'dev' version of the tool in Galaxy, and installed it now via het Tool Shed Manager. I got an error that the dependencies could not be installed. If figured it out that it (needed to add an extra shell_command in the tool_dependencies) (and the mount point on my system needs to have exec permissions of course). So I updated the tool in the toolshed. BUT, then I uninstalled the tool from Galaxy, and reinstalled it from the Tool Shed ... instead of just updating. Anyway, it has happened. Now I have the tool listed in my 'Install Tool Shed Repositories', and when I click on it, it says 'this tool is not installed'. The only action I can apply on this installed tool shed repository is 'Install' (...). But clicking on that 'install' button, leads me to an error page (http error 500). ** URL: http://localhost/admin_toolshed/prepare_for_install?tool_shed_url=http://localhost:9009/repository_ids=84cbaf0b6bf7a1a9changeset_revisions=fc2339abcab7 Module weberror.evalexception.middleware:364 in respond view app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in
Re: [galaxy-dev] Messed up a tool installation from a local toolshed
See attachment. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 02:15 PM, Peter Cock wrote: On Mon, Sep 30, 2013 at 1:05 PM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi Greg, Sorry to dig up this unsolved thread from a few months back. Recap of the bug: I have developed a small tool. I have put it in my local Toolshed, and installed it. After applying some changes to the tool, I 'hg push'ed the changes to my repository on my toolshed. I uninstalled the tool, and reinstalled it. Since I work with job handlers, I have to restart Galaxy. Since I had about ~4 committed changes, I have repeated these steps about 4 times. The bug: the tool appears multiple times in the tool management window. 2 times as new, and 1 time as uninstalled (since I uninstalled it). The tool is still available in the toolbox, although it should not. The 2 installed tools have an option 'install', but when I click this I get an internal server error 500. Galaxy version: * [galaxy@galaxy galaxy-dist]$ hg tip changeset: 10411:c42567f43aa7 tag: tip user:greg date:Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy. * As per your request: the web log of this internal server error. 193.191.128.2 - - [30/Sep/2013:13:15:19 +0200] GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.bits.vib.be/repository_ids=968205d9ed31733dchangeset_revisions=e08a6d9e97a3 HTTP/1.1 500 - http://galaxy.bits.vib.be/admin_toolshed/browse_repositories; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/28.0.1500.71 Chrome/28.0.1500.71 Safari/537.36 Debug at: http://galaxy.bits.vib.be/_debug/view/1380535486 The postgres database shows: galaxydb= select id, tool_shed, name, owner, installed_changeset_revision, status, error_message from tool_shed_repository where name = 'transpose'; id | tool_shed | name| owner | installed_changeset_revision | status| error_message +--+---+-+--+-+--- 45 | toolshed.bits.vib.be | transpose | joachim | cee0157acee1 | Uninstalled | 44 | toolshed.bits.vib.be | transpose | joachim | 2fa0462b9303 | New | 43 | toolshed.bits.vib.be | transpose | joachim | e08a6d9e97a3 | New | (3 rows) How should I proceed ? This is the second time I experience this. On a last note: I have noted also that I managed to get a repository with different branches in this tool shed repository, by mixing 'hg push' and uploading a tarball. There seems no easy way to fixing this. Kind regards, Joachim Hmm - do these New entries show up in white on the Manage installed tool shed repositories admin page? It sounds like a case of what was nicknamed white ghosts, some causes of which have been fixed - but I don't think there was any automated removal of old white ghosts (I still have two on my test system). Peter attachment: Selection_221.png___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Download URL not accepted in tool_dependencies.xml
Still having a hard time and some strange situations. It is almost too much to report via email. Basically, the tool_dependencies.xml get recognised sometimes. Please, watch this video to see what I mean (1'30''), and pause as you like: https://dl.dropboxusercontent.com/u/18352887/toolshedissues.ogv I am afraid I cannot invest this amount of time to make my repo's public-proof. There are weird things happening which make no sense to me. And debugging of tool_dependencies.xml is very hard, since I find almost no logs of this process. Joachim. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 02:53 PM, Dave Bouvier wrote: Joachim, For sourceforge URLs, you can safely omit the question mark and everything after it, since that's part of the browser download load balancing process. When downloading with wget, curl, or python's urllib, downloads.sourceforge.net does its own behind-the-scenes load balancing. As for hyphens, I don't believe they need to be escaped, since they're not XML entities like the ampersand is. The following download_by_url action should work: action type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip/action --Dave B. On 09/30/2013 08:45 AM, Joachim Jacob | VIB | wrote: Hi all, Not fixed yet. Somehow the download does not work: I have escaped the ampersands and the hyphens. The download seems to have to proceed, since I receive no error from the download step. The next step in the dependencies file gives an error: mv: cannot stat transpose.zip': No such file or directory Cheers, J Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/30/2013 02:06 PM, Peter Cock wrote: You must escape any ampersand in the XML as amp; and then it should work. Peter On Mon, Sep 30, 2013 at 12:51 PM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi all, The download URL seems not to be accepted in tool_dependencies.xml. It is an URL from sourceforge, the direct link to a package. The error upon uploading the tool to my toolshed: ** Metadata may have been defined for some items in revision '5665a799775d'. Correct the following problems if necessary and reset metadata. tool_dependencies.xml - Exception attempting to parse /mnt/toolsheddb/database/000/repo_26/tool_dependencies.xml: not well-formed (invalid token): line 6, column 149 ** The tool_dependencies.xml file: ** 1 ?xml version=1.0? 2 tool_dependency 3 package name=transpose version=2.0.0 4install version=1.0 5 actions 6 action type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip?r=ts=1380535239use_mirror=surfnet/action 7action type=shell_commandmkdir bin/action 8 action type=shell_commandunzip transpose-2.0.zip/action action type=shell_commandcd transpose-2.0/src/action action type=shell_commandgcc transpose.c -o transpose/action action type=move_file source$INSTALL_DIR/transpose-2.0/src/transpose/source destination$INSTALL_DIR/bin/destination /action action type=shell_commandchmod +x $INSTALL_DIR/bin/transpose/action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme Compiling transpose and putting in the path. /readme /package /tool_dependency ** Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ZFS storage recommendations
- tank/galaxydb snapdev hidden default * Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/10/2013 11:29 PM, Guest, Simon wrote: Hi Joachim, At AgResearch we are using ZFS for our HPC storage, which is used by our internal Galaxy instance. Currently we are running on FreeNAS (FreeBSD derivative), but we are in transition to ZFS on Linux. We export the filesystem over NFS (10Gb Ethernet), but not the database (PostgreSQL). In general, you want block storage for a database, so I suggest you look for a solution other than NFS to host that. Our experience with ZFS has been very positive. However, FreeNAS is not really suited to our needs - it's more of a storage appliance, probably great for a home NAS. Hence the planned transition. I strongly recommend you follow the discussion on the ZFS discuss mailing list. There's a lot to learn about ZFS configuration, much more than you will glean from few posts here. http://zfsonlinux.org/lists.html cheers, Simon -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev- boun...@lists.bx.psu.edu] On Behalf Of Joachim Jacob | VIB | Sent: Tuesday, 10 September 2013 10:29 p.m. To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] ZFS storage recommendations Hi all, I am performing some tests to move my galaxy database to ZFS. Does anybody have experience with ZFS on linux, and some recommendations/experiences to optimize performance? The purpose is to share the database over NFS to the Galaxy VM. Thanks, Joachim. -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deseq2 wrapper question
Hi Rui, I should provide a 'sample metadata table', a tabular file with one column the names of your samples, the other column the associated metadata (e.g. type). This is how I've done it in my DESeq2 wrapper. :-) Feel free to check the interface at http://toolshed.bits.vib.be/view/joachim/deseq2 Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 01:15 AM, ruiwang.sz wrote: Hi All, we are making a wrapper for deseq2, and there is a step where control/experimental conditions need to be determined. For example, it would be a column like the following in the input file Type CTC CTC LM LM PT PT and we want the select list to contain CTC LM PT In other words, we want to filter out the 'Type' and have only distinct ones in the select list. We could make it in two steps, generate a intermediate file for this but it would be nice that we could directly retrieve the distinct types to make the list. I took a look at the tool config wiki, but didn't see anything (maybe because it was a quick scan). Does anyone have any tip on this? we'll really appreciate. Thanks, Rui ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deseq2 wrapper question
It is not necessarily bad. If we all test each others tools (dependencies, interface, output,...) we might get to the best of all worlds :-) (I have planned to test Björns wrapper) One of the points of differentiation is also how 'integrated' a tool needs to be: e.g. Ross' tool for diff expression (combining a complete pipelin), and my tool just on DE with DESeq2. I like to keep things granular on the level of the tools. Cheers, Joachim Joachim Jacob. Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 10:56 AM, Peter Cock wrote: Wow - that makes at least five Deseq2 wrappers for Galaxy available or in progress :( Bjoern's wrapper on the (test) tool shed, http://testtoolshed.g2.bx.psu.edu/view/bgruening/deseq2 Ross' combined wrapper for edgeR, DESeq2 and voom in one tool: http://testtoolshed.g2.bx.psu.edu/view/fubar/differential_count_models Joachim's wrapper on the Tool Shed http://toolshed.bits.vib.be/view/joachim/deseq2 Vipin's which is soon to be released to the (Test?) Tool Shed, https://github.com/ratschlab/oqtans_tools/tree/master/DESeq2 And Rui's is working on one too. This does seem like duplicated effort a source of confusion for end users and Galaxy administrators (a problem not unique to deseq2, but affecting many Galaxy wrappers). I appreciate there will be different needs, and one wrapper may not suit all, but I would prefer if the default behaviour for Galaxy tool wrapper authors was to collaborate on one good wrapper rather than writing competing ones. When I started work on a new wrapper I tried to announce this on the mailing list to find out if anyone else was already tackling the same tool - and that seemed to work quite well. Of course, the volume of emails on galaxy-dev has grown quite a lot over the last few years so that may not be as effective, but the archives should be searchable. Perhaps we need to improve communication in some way? One option would be to encourage greater use of the Test Tool Shed for works in progress to give them visibility? You could even have the Tool Shed itself require a search step before creating a new repository to avoid accidental duplication of effort? Or maybe a wiki page of wrappers in progress? (And maybe we should split this into a new thread) Regards, Peter On Wed, Sep 11, 2013 at 8:59 AM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi Rui, I should provide a 'sample metadata table', a tabular file with one column the names of your samples, the other column the associated metadata (e.g. type). This is how I've done it in my DESeq2 wrapper. :-) Feel free to check the interface at http://toolshed.bits.vib.be/view/joachim/deseq2 Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 01:15 AM, ruiwang.sz wrote: Hi All, we are making a wrapper for deseq2, and there is a step where control/experimental conditions need to be determined. For example, it would be a column like the following in the input file Type CTC CTC LM LM PT PT and we want the select list to contain CTC LM PT In other words, we want to filter out the 'Type' and have only distinct ones in the select list. We could make it in two steps, generate a intermediate file for this but it would be nice that we could directly retrieve the distinct types to make the list. I took a look at the tool config wiki, but didn't see anything (maybe because it was a quick scan). Does anyone have any tip on this? we'll really appreciate. Thanks, Rui ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
The latest code from the 'stable' branch of Galaxy is preferred. See the news brief accompanying the update (release_2013.08.12) on http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12 You need mercurial (http://mercurial.selenic.com/) for installing Galaxy. Report bugs at https://trello.com/b/75c1kASa/galaxy-development Cheers Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/11/2013 03:55 AM, test galaxy wrote: Hi everyone, I want to install my local Galaxy with less bugs,and I don't know which version is more stable and work well.Could anyone give same advise?Thank you very much. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ZFS storage recommendations
Thanks Nate. Compression is currently with lzjb. Will do some testing with dd. Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On Wed 11 Sep 2013 04:03:55 PM CEST, Nate Coraor wrote: Hi all, I'm a big fan of ZFS, we have long used it behind Galaxy Main. Some of our older servers are (still) Solaris, and the newest is FreeBSD. I've lately been using SmartOS for virtualization and while it has a drawback as a fileserver (since currently the nfs server can only run in the global zone, which is not ideal on SmartOS), there are other illumos derivatives that would probably be great for this task (e.g. OmniOS). Native ZFS in the OS in which it is developed is a win for me, especially when you are serving via simple NFS. For more complex network filesystems, Linux is probably preferable. I considered a separate ZIL and L2ARC for the latest ZFS server, but DTrace revealed that I probably would not see much of a performance benefit with our usage patterns. The memory usage you're seeing is to be expected - it will pretty much consume whatever is available for caching, but it's available to be freed if needed for something else. I wouldn't suggest rsync for performance testing. I typically do things like timed writing of blocks read from /dev/zero using dd, so that the source filesystem and checksumming algorithm can be taken out of the equation. And dedup/compression will of course cause a significant write penalty. If you can suffer the decreased space optimization, lzjb performs significantly better than gzip. gzip-1 is a nice compromise between the default gzip level and lzjb, as well. --nate On Sep 11, 2013, at 3:45 AM, Joachim Jacob | VIB | wrote: Thank you all for the reactions. Some details about my current ZFS and Galaxy setup: - Galaxy runs as a single virtual machine, with currently 20 cores, 80GB RAM. Will be 32 cores and about 160GB RAM soon. - The postgres database is on the virtual machine itself. - The 'files' and 'job_working_dir' are on an NFS exported directory, hosted on the host machine of the guest. - The NFS exported directory is an raidz1 dataset. - The raidsz1 runs on 7 550GB SAS disks, which are (unfortunately) controlled by a RAID hardware controller, but passed as RAID0 (JBOD not available). So raidz1 runs on 7 RAID0 disks (with settings in the hardware controller PERC H700: no read ahead, write through, 8 KB stripe size, disk cache policy enabled). - Compression and deduplication is enabled. - The directory on which the zfs dataset is mounted, is exported using the native linux NFS daemon to the Galaxy virtual machine. The 'zfs sharenfs' did not work (ownerships not set correctly - perhaps need some more investigation, but I found several times reports about sharenfs option in ZFS in linux is not behaving well...). The numbers: - my initial files database (ext4 on RAID5) is 3.0TB in size. On ZFS, with compression and deduplication, this database is *1.8TB *(-40%). - Did not yet provide a SLOG to host the ZIL and a L2ARC, since I have a cleared picture about the performance I can get. Would you advise preference over ZIL on a SLOG, or go for a SSD to host the L2ARC. - The cost for this performance of ZFS in storage is RAM: currently continuously using*284GB RAM* for ZFS! - The write and read speed is from the Galaxy VM over NFS is *~40MB/s and ~100MB/s* (tested by simply copying over rsync - I still need to check the presentation and scripts of Anne Black-Ziegelbein). This is a 66% decrease in previously achieved write and read speed (ext4 on hardware RAID5), but I feel that the benefits (deduplication, backing up via snapshots, data integrity,) outweigh this IO performance. (I am setting this ZFS up on a new server (well, actually 2 years old now, has served on another project well)) Currently our Galaxy uses this zfs with success! For your interest, my settings on the 'galaxydb' zfs tank below. (I was wondering if here some more wizardry can be applied). [root@r910bits ~]# *zfs get all tank/galaxydb* NAME PROPERTY VALUE SOURCE tank/galaxydb type filesystem - tank/galaxydb creation Mon Sep 9 12:44 2013 - tank/galaxydb used 1.81T - tank/galaxydb available 1.66T - tank/galaxydb referenced1.81T - tank/galaxydb compressratio 1.66x - tank/galaxydb mounted yes- tank/galaxydb quota none default tank/galaxydb reservation none default tank/galaxydb recordsize128K default tank/galaxydb mountpoint/mnt/galaxydb local tank/galaxydb sharenfs rw=@galaxy local tank/galaxydb checksum on default tank/galaxydb compression lzjb
Re: [galaxy-dev] API question: is it possible to start exporting histories
OK, I would like to see that happen too :-) I have been thinking about possible 'elegant' solutions to enable this exporting. I am very enthousiastic about Bittorrent Sync, which could perhaps be an efficient and secure way to export the histories (http://labs.bittorrent.com/experiments/sync.html). The Galaxy session would then show the bittorrent sync 'shared key' in the session, which the user can copy paste in their BitTorrent Sync client, after which syncing happens. Once syncing is done, connection can quit and exported directory deleted from Galaxy. One difficult thing at the client side is firewall configuration :-( Thanks, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/06/2013 05:41 PM, Jeremy Goecks wrote: Is it currently possible to export histories via the API? Not possible yet but is definitely something we'd like to implement and/or see from a community contribution. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tool's toolshed page does not get updated after uploading new version
Hi all, My issue: I have put a new version of a tool in our local toolshed (via 'hg push'). Now, going to the tool's page, still the old version of the README is displayed. When browsing the repository, the new (last uploaded) version is there. In addition, the tool's page show the latest revision, but displays the former version. Did I go wrong somewhere or is this a bug? [ I don't know whether this is known already - Trello is annoyingly slow, and there is no easy way to swiftly search in the non-existing Trello list Toolshed ( sorry, bear with me please, I needed to point out my issues with Trello at least once ;-) ] Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Solved Tool's toolshed page does not get updated after uploading new version
Hmm, now the tool's page is updated. Problem solved. Perhaps a browser cache issue or something alike. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/06/2013 10:22 AM, Joachim Jacob | VIB | wrote: Hi all, My issue: I have put a new version of a tool in our local toolshed (via 'hg push'). Now, going to the tool's page, still the old version of the README is displayed. When browsing the repository, the new (last uploaded) version is there. In addition, the tool's page show the latest revision, but displays the former version. Did I go wrong somewhere or is this a bug? [ I don't know whether this is known already - Trello is annoyingly slow, and there is no easy way to swiftly search in the non-existing Trello list Toolshed ( sorry, bear with me please, I needed to point out my issues with Trello at least once ;-) ] Thanks, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] API question: is it possible to start exporting histories
Hi all, Is it currently possible to export histories via the API? I have one user who want to download and store her data. Total sizes of all her histories combined is about ~700GB. I think this is a typical API job: to list all her histories, start 'exporting to file' all of them, and finally download all exported histories to a disk. But looking at the current API code, I doubt whether this can be done. Hence my question. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Trackster: alignment is not showing up as 'squish'
Hi all, I have a problem that the mappings of an experiment suddenly don't show up anymore in Trackster, only when I choose 'squish' as visualisation format. 'Coverage' and 'dense' provide the expected output. So 'squish' shows nothing. But 'Pack' shows only the names, without the bars. Note: 'squish' and 'pack' used to work. Currently I am on 2013_08_12. Other mappings (from other experiments) show up as expected. The 2 cases side by side in this screenshot (thanks to the scratchbook functionality!): https://dl.dropboxusercontent.com/u/18352887/Selection_151.png Thanks for any help, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trackster: alignment is not showing up as 'squish'
Update: the 'dense' option does not work neither. So in summary: the alignment can only be viewed in 'coverage' mode. Thanks, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/05/2013 09:52 AM, Joachim Jacob | VIB | wrote: Hi all, I have a problem that the mappings of an experiment suddenly don't show up anymore in Trackster, only when I choose 'squish' as visualisation format. 'Coverage' and 'dense' provide the expected output. So 'squish' shows nothing. But 'Pack' shows only the names, without the bars. Note: 'squish' and 'pack' used to work. Currently I am on 2013_08_12. Other mappings (from other experiments) show up as expected. The 2 cases side by side in this screenshot (thanks to the scratchbook functionality!): https://dl.dropboxusercontent.com/u/18352887/Selection_151.png Thanks for any help, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ProFTPD Config Issues
Hi Adam, I remembered having a hard time getting this to work. After it worked, networkadmins decided to disallow ftp connections at our university due to security concerns... I do not remember the solution (...), but I can share my config files (I am using apache instead of nginx): This is my working 'proftp.conf': # This is the ProFTPD configuration file # Server Config - config used for anything outside a VirtualHost or Global context ServerNameBITS Galaxy FTP ServerIdenton Welcome to BITS Galaxy FTP server. ServerTypestandalone #ServerAdminroot@localhost #Umask0066 SyslogFacility DAEMON SyslogLevel debug DefaultRoot~ !adm CreateHome on 700 uid 600 gid 601 PassivePorts39000 4 MaxInstances30 Usergalaxy Group galaxy #UseFtpUsers off #AuthPAMoff # Don't do reverse DNS lookups (hangs on DNS problems) UseReverseDNSoff MaxInstances20 # Define the log formats LogFormatdefault%h %l %u %t \%r\ %s %b LogFormatauth%v [%P] %h %t \%r\ %s # General database support (http://www.proftpd.org/docs/contrib/mod_sql.html) #LoadModule mod_sql.c # Support for base-64 or hex encoded MD5 and SHA1 passwords from SQL tables #LoadModule mod_sql_passwd.c # Postgresql support (requires proftpd-postgresql package) # (http://www.proftpd.org/docs/contrib/mod_sql.html) # LoadModule mod_sql_postgres.c Global # Allow users to overwrite files and change permissions AllowOverwrite on AllowStoreRestart on # Bar use of SITE CHMOD Limit SITE_CHMOD DenyAll /Limit # Bar use of RETR (download) since this is not a public file drop Limit RETR DenyAll /Limit SQLLogFile /var/log/proftpd/proftpd.log # get FTP connection over SSH DefaultServeron SFTPEngine on SFTPLog /var/log/proftpd-sftp.log Port 8822 # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex RequireValidShell off # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxydb@127.0.0.1 galaxyftp ** SQLAuthTypesSHA1 SQLAuthenticate users # An empty directory in case chroot fails SQLDefaultHomedir /mnt/galaxytemp/ftptmp SQLDefaultGID 601 SQLDefaultUID 600 SQLMinID 95 # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT email,password,'600','601','/mnt/galaxydb/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U' /Global Good luck, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 09/05/2013 02:46 AM, Panzer, Adam wrote: Hello Devs, I've been trying for a while now to get ProFTPD working for our local instance. I have: 1) Modified pg_hba.conf 2) Set up the ProFTPD config files 3) Disabled PBKDF2 password encryption by adding use_pbkdf2 = false to universe_wsgi.ini I've read through the posts from others with this issue and made sure to avoid some of the common pitfalls like forgetting to install/enable mod_sql.c, mod_sql_postgres.c, and LoadModule mod_sql_passwd.c or having an incorrect/999 uid/gid in proftpd.conf. Even so, when I attempt to access the server through FileZilla, it continues to kick back login errors for all users: Status:Resolving address of localhost Status:Connecting to 127.0.0.1:21... Status:Connection established, waiting for welcome message... Response:220 ProFTPD 1.3.4a Server (Jay2) [:::127.0.0.1] Command:USER panze...@kids.wustl.edu Response:331 Password required for panze...@kids.wustl.edu Command:PASS *** Response:530 Login incorrect. Error:Critical error Error:Could not connect to server I have included the contents of the relevant config files and logs below. The proftpd log says that the user name is wrong, but the sqllog seems to indicate a successful hit on the user but a failed password authentication (again, despite disabling PBKDF2). I would be most grateful if someone could tell me what I'm doing wrong. Thanks, Adam - PACKAGE VERSIONS - PostgreSQL 9.2 Nginx 1.1.19 ProFTPD 1.3.4a - PG_HBA.CONF - local all all trust hostall all 127.0.0.1/32trust hostall all ::1/128
Re: [galaxy-dev] UGENE tools integration into Galaxy web-platform
Hi Peter, I have used UGene before: it is a nice tool. I am not sure I have understood your question. The easiest way to integrate your tool in Galaxy, is that your tool can be driven by the command line (no GUI). The tool and parameters should be configured by a tool config file, as you mentioned. The other way around (let UGene use Galaxy tools) is that you use Galaxy's API to submit jobs and return the results. Plenty of documentation can be found on Galaxy's wiki pages! wiki.galaxyproject.org Hope this helps you a bit further, Joachim Jacob. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 08/05/2013 12:19 PM, Пётр Леонтьев wrote: Hello, Galaxy team. I am writing you to find out the right way about how can be other bioinformatics tools integrated into your web-platform. I will explain more detail: we have in our bioinformatics software UGENE a tool Workflow designer. It has different elements for simple sequences operations as well as MSA algorithms, NGS analysis and so on. You can create new scheme in the workflow designer, set parameters that will be shown at new Galaxy tool window and then just generate tool config file. In Galaxy you can run this tool and get results at history panel and use it futher (in other Galaxy tools). I attached simple scheme example and showed how to generate config for it. Also there is a picture of our scheme in Galaxy. Exact answer is very interesting for us and we will hope to hear good response. I will be grateful to you if this letter will be sent to correct address if that is not so. Kind Regards, Peter Leontev, UGENE team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Disable cheetah filtering
Hi all, A Cheetah question: when I enter ; as a value in a text box of a tool (stored in $exp), Cheetah prints this as X, and passed this string to the tool. How to disable this behaviour in Cheetah? I have tried to play with #filter directives, but to no avail. Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Disable cheetah filtering
Thanks both of you! Okay, so I was looking in the wrong direction. This solved the issue. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/24/2013 11:13 AM, Björn Grüning wrote: Hi Joachim, try that one: http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set Cheers, Bjoern Hi all, A Cheetah question: when I enter ; as a value in a text box of a tool (stored in $exp), Cheetah prints this as X, and passed this string to the tool. How to disable this behaviour in Cheetah? I have tried to play with #filter directives, but to no avail. Thanks, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Disable cheetah filtering
Okay, normally I would have forked the galaxy-dist and do a pull request with my change. But the change I want to commit is in the tool 'add_value', which is in the toolshed. Seems like a safer approach to me to contribute to the toolshed. How can I commit my changes to that tool? Can somebody of the devteam grant me acces? Do I send the diff by email :-) ? Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/24/2013 11:17 AM, Peter Cock wrote: On Wed, Jul 24, 2013 at 9:40 AM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Hi all, A Cheetah question: when I enter ; as a value in a text box of a tool (stored in $exp), Cheetah prints this as X, and passed this string to the tool. How to disable this behaviour in Cheetah? I have tried to play with #filter directives, but to no avail. Thanks, Joachim That sounds like the Galaxy character sanitation making the change, things like pipes etc can have unexpected results in command lines. Have a look at the sanitizer tag for use in the tool's XML file. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Disable cheetah filtering
Hmm, seems that collaborating on tools is kind of a burden... Three places to notify this small change you say? (trello, galaxy-dev and direct emailing to the authors.) I have opted for Trello, card #1013 Cheers, J Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/24/2013 12:45 PM, Peter Cock wrote: On Wed, Jul 24, 2013 at 11:25 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi, Okay, normally I would have forked the galaxy-dist and do a pull request with my change. But the change I want to commit is in the tool 'add_value', which is in the toolshed. Seems like a safer approach to me to contribute to the toolshed. How can I commit my changes to that tool? Can somebody of the devteam grant me acces? Do I send the diff by email :-) ? Yes, please send the patch to the mailing list and/or the devteam (toolshed should have a contact button). Also try to fill a trello card :) Thanks! Bjoern In general the upstream repository for a given tool will depend on the tool author's preferences. They may not even use a separate repository and work directly with the ToolShed in some cases. In other cases you might be able to contribute directly via a github pull request (planned for the NCBI BLAST+ tools), or a bitbucket pull request (the Galaxy team like bitbucket). The 'add_value' tool is under 'devteam' and it was original in the main Galaxy repository under BitBucket - I don't know where it lives now outside the tool shed: http://toolshed.g2.bx.psu.edu/view/devteam/add_value Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Testing with repeat parameters
Oops, I was replying via theGalaxy Development List Archive http://dev.list.galaxyproject.org/ interface, strange it has put the reply off list... So no functional test for my tool ( despite my enthousiasm :-) ), since the repeat tags must be provided in the test. Thanks, Joachim. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/16/2013 01:46 PM, Peter Cock wrote: Off list? They don't work yet, bar the simplest case of 0 or 1 repeats using a single parameter perhaps. Peter On Tuesday, July 16, 2013, wrote: Hi Peter, I was wondering if you have details on how to include tests for repeat tag sets? Cheers, Joachim quote author='Peter Cock' Hello all, Should unit tests with repeat parameters work? If so, how do I give each repeat group in the XML for the test? Sample tool here: http://testtoolshed.g2.bx.psu.edu/view/peterjc/venn_list inputs param name=main_lab size=30 type=text value=Venn Diagram label=Plot title/ conditional name=universe param name=type_select type=select label=Implicit or explicit full ID list? option value=explicitExplicit/option option value=implicitImplicit (use union of sets below)/option /param when value=explicit param name=main type=data format=tabular,fasta,fastq,sff label=Full dataset (with all identifiers) help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ /when when value=implicit/ /conditional repeat name=sets min=1 max=3 title=Sets param name=set type=data format=tabular,fasta,fastq,sff label=Members of set help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ param name=lab size=30 type=text value=Group label=Caption for set/ /repeat /inputs The first test attempts to use the repeat set/lab once - guessing at how to give repeat parameters in a test: test param name=type_select value=explicit/ param name=main value=venn_list.tabular ftype=tabular/ param name=main_lab value=Some Proteins/ param name=set value=rhodopsin_proteins.fasta/ param name=lab value=Rhodopsins/ output name=PDF file=venn_list1.pdf ftype=pdf/ /test $ ./run_functional_tests.sh -id venn_list ... python /mnt/galaxy/galaxy-central/tools/plotting/venn_list.py /mnt/galaxy/galaxy-central/database/files/000/dataset_1.dat tabular Some Proteins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Rhodopsins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Group /mnt/galaxy/galaxy-central/database/files/000/dataset_3.dat ... This seems to have resulted in two repeat entries, and so fails. What I was expecting was: python /mnt/galaxy/galaxy-central/tools/plotting/venn_list.py /mnt/galaxy/galaxy-central/database/files/000/dataset_1.dat tabular Some Proteins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Rhodopsins /mnt/galaxy/galaxy-central/database/files/000/dataset_3.dat Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ /quote Quoted from: http://dev.list.galaxyproject.org/Testing-with-repeat-parameters-tp4659840.html _ Sent from http://dev.list.galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Testing with repeat parameters
OK, it seems that the current state of the functional test framework 'simulates' one click on the 'repeat' button. You just provide one set of param names contained within the repeat block. Luckily, this is enough for my tool now. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/16/2013 01:49 PM, Joachim Jacob | VIB | wrote: Oops, I was replying via theGalaxy Development List Archive http://dev.list.galaxyproject.org/ interface, strange it has put the reply off list... So no functional test for my tool ( despite my enthousiasm :-) ), since the repeat tags must be provided in the test. Thanks, Joachim. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/16/2013 01:46 PM, Peter Cock wrote: Off list? They don't work yet, bar the simplest case of 0 or 1 repeats using a single parameter perhaps. Peter On Tuesday, July 16, 2013, wrote: Hi Peter, I was wondering if you have details on how to include tests for repeat tag sets? Cheers, Joachim quote author='Peter Cock' Hello all, Should unit tests with repeat parameters work? If so, how do I give each repeat group in the XML for the test? Sample tool here: http://testtoolshed.g2.bx.psu.edu/view/peterjc/venn_list inputs param name=main_lab size=30 type=text value=Venn Diagram label=Plot title/ conditional name=universe param name=type_select type=select label=Implicit or explicit full ID list? option value=explicitExplicit/option option value=implicitImplicit (use union of sets below)/option /param when value=explicit param name=main type=data format=tabular,fasta,fastq,sff label=Full dataset (with all identifiers) help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ /when when value=implicit/ /conditional repeat name=sets min=1 max=3 title=Sets param name=set type=data format=tabular,fasta,fastq,sff label=Members of set help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ param name=lab size=30 type=text value=Group label=Caption for set/ /repeat /inputs The first test attempts to use the repeat set/lab once - guessing at how to give repeat parameters in a test: test param name=type_select value=explicit/ param name=main value=venn_list.tabular ftype=tabular/ param name=main_lab value=Some Proteins/ param name=set value=rhodopsin_proteins.fasta/ param name=lab value=Rhodopsins/ output name=PDF file=venn_list1.pdf ftype=pdf/ /test $ ./run_functional_tests.sh -id venn_list ... python /mnt/galaxy/galaxy-central/tools/plotting/venn_list.py /mnt/galaxy/galaxy-central/database/files/000/dataset_1.dat tabular Some Proteins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Rhodopsins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Group /mnt/galaxy/galaxy-central/database/files/000/dataset_3.dat ... This seems to have resulted in two repeat entries, and so fails. What I was expecting was: python /mnt/galaxy/galaxy-central/tools/plotting/venn_list.py /mnt/galaxy/galaxy-central/database/files/000/dataset_1.dat tabular Some Proteins /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat fasta Rhodopsins /mnt/galaxy/galaxy-central/database/files/000/dataset_3.dat Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ /quote Quoted from: http://dev.list.galaxyproject.org/Testing-with-repeat-parameters-tp4659840.html _ Sent from http://dev.list.galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use
Re: [galaxy-dev] repeat tag enhancement?
I like your proposal. I don't think it is currently possible to add - say - 5 repeat blocks at once. Feel free to add and describe your suggested enhancement on Trello, so I can vote for it :-) http://galaxyproject.org/trello Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/16/2013 01:34 PM, Berner, Thomas wrote: Hey guys, i was searching the mailing list and wiki for some hints if there is a possibility to enhance the repeat tag. We want to give the option to our users to add more than one additional set of the contained parameters at one go e.g. 10 or 20 or 50, because it can be very frustrating to click the “add new dataset” button 50 times. Is there any option to do this with a text field or preset like “add 20 new datasets” or something like that? I would be very thankful for an advice. Greetings, Thomas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Messed up a tool installation from a local toolshed
-jun.-2013 16:15 Onderwerp: Re: [galaxy-dev] Messed up a tool installation from a local toolshed Aan: Joachim Jacob joachim.ja...@gmail.com mailto:joachim.ja...@gmail.com Cc: galaxy-...@bx.psu.edu mailto:galaxy-...@bx.psu.edu Hello Joachim, What version of Galaxy are your running? What does the tool shed paster go show for the 500 Internal Server Error? Also, run the following sql manually in your Galaxy database and send the results. select id, tool_shed, name, owner, installed_changeset_revision, status, error_message from tool_shed_repository; Thanks, Greg Von Kuster Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 06/14/2013 09:38 AM, Joachim Jacob | VIB | wrote: Hi all, I have developed a tool in our local Galaxy, and imported it in a local toolshed. I next removed the 'dev' version of the tool in Galaxy, and installed it now via het Tool Shed Manager. I got an error that the dependencies could not be installed. If figured it out that it (needed to add an extra shell_command in the tool_dependencies) (and the mount point on my system needs to have exec permissions of course). So I updated the tool in the toolshed. BUT, then I uninstalled the tool from Galaxy, and reinstalled it from the Tool Shed ... instead of just updating. Anyway, it has happened. Now I have the tool listed in my 'Install Tool Shed Repositories', and when I click on it, it says 'this tool is not installed'. The only action I can apply on this installed tool shed repository is 'Install'. Clicking on that 'install' button, leads me to an error page (http error 500). ** URL: http://localhost/admin_toolshed/prepare_for_install?tool_shed_url=http://localhost:9009/repository_ids=84cbaf0b6bf7a1a9changeset_revisions=fc2339abcab7 Module weberror.evalexception.middleware:364 in respond view app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view body = method( trans, **kwargs ) Module galaxy.web.framework:221 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:825 in prepare_for_install view raw_text = common_util.tool_shed_get( trans.app, tool_shed_url, url ) Module tool_shed.util.common_util:110 in tool_shed_get view response = urlopener.open( uri ) Module urllib2:397 in open view response = meth(req, response) Module urllib2:510 in http_response view 'http', request, response, code, msg, hdrs) Module urllib2:435 in error view return self._call_chain(*args) Module urllib2:369 in _call_chain view result = func(*args) Module urllib2:518 in http_error_default view raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error The same happens when I go to the Tool Shed, and try to install the tool from there: Galaxy detects that the tool has been installed before, and points me to that 500 error page. Any help is heartily appreciated, Thanks, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Possible bug in managing tool shed repositories
Hi all, Before posting on Trello, perhaps someone can confirm that it is a bug: 1. In the admin menu, go to 'Manage installed tool shed repositories' 2. click on a tool that you have installed with dependencies (!) 3. In the top right 'repository actions', choose 'Manage tool dependencies' 4. In the top right 'Actions', choose 'Manage repository'. 5. An error similar like this one appears: Error Traceback: View as: Interactive | Text | XML (full) â ValueError: invalid literal for int() with base 10: 'a9975dcda42e2b00' URL: http://localhost/admin_toolshed/manage_repository?id=3f428053038c8626b08a92869ee1362d132430354fe813e0 Module weberror.evalexception.middleware:364 in respond view app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view body = method( trans, **kwargs ) Module galaxy.web.framework:221 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:561 in manage_repository view repository = suc.get_installed_tool_shed_repository( trans, repository_id ) Module tool_shed.util.shed_util_common:513 in get_installed_tool_shed_repository view return trans.sa_session.query( trans.model.ToolShedRepository ).get( trans.security.decode_id( id ) ) Module galaxy.web.security:60 in decode_id view return int( self.id_cipher.decrypt( obj_id.decode( 'hex' ) ).lstrip( ! ) ) ValueError: invalid literal for int() with base 10: 'a9975dcda42e2b00' All tools with dependencies seem to have this behaviour. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Messed up a tool installation from a local toolshed
Hi all, I have developed a tool in our local Galaxy, and imported it in a local toolshed. I next removed the 'dev' version of the tool in Galaxy, and installed it now via het Tool Shed Manager. I got an error that the dependencies could not be installed. If figured it out that it (needed to add an extra shell_command in the tool_dependencies) (and the mount point on my system needs to have exec permissions of course). So I updated the tool in the toolshed. BUT, then I uninstalled the tool from Galaxy, and reinstalled it from the Tool Shed ... instead of just updating. Anyway, it has happened. Now I have the tool listed in my 'Install Tool Shed Repositories', and when I click on it, it says 'this tool is not installed'. The only action I can apply on this installed tool shed repository is 'Install'. Clicking on that 'install' button, leads me to an error page (http error 500). ** URL: http://localhost/admin_toolshed/prepare_for_install?tool_shed_url=http://localhost:9009/repository_ids=84cbaf0b6bf7a1a9changeset_revisions=fc2339abcab7 Module weberror.evalexception.middleware:364 in respond view app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view body = method( trans, **kwargs ) Module galaxy.web.framework:221 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:825 in prepare_for_install view raw_text = common_util.tool_shed_get( trans.app, tool_shed_url, url ) Module tool_shed.util.common_util:110 in tool_shed_get view response = urlopener.open( uri ) Module urllib2:397 in open view response = meth(req, response) Module urllib2:510 in http_response view 'http', request, response, code, msg, hdrs) Module urllib2:435 in error view return self._call_chain(*args) Module urllib2:369 in _call_chain view result = func(*args) Module urllib2:518 in http_error_default view raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error The same happens when I go to the Tool Shed, and try to install the tool from there: Galaxy detects that the tool has been installed before, and points me to that 500 error page. Any help is heartily appreciated, Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installing Galaxy EC2
Hi Daniel,
Make also sure you use correct firewall settings ('security groups') in
EC2. They are described the Galaxy wiki
(http://wiki.galaxyproject.org/CloudMan/AWS/GettingStarted?highlight=%28security%29|%28group%29),
but there are some small differences with the setting applied by
CloudLaunch
(http://dev.list.galaxyproject.org/Cloudman-missing-security-rules-in-wiki-page-td4659826.html#a4659902)
Set by CloudLaunch (during my latest launch):
Ports ProtocolSource CloudMan
1-65535 tcp sg-61ba790a
*1-65535 udp sg-61ba790a *
*-1 icmpsg-61ba790a *
20-21 tcp 0.0.0.0/0 http://0.0.0.0/0
22 tcp 0.0.0.0/0 http://0.0.0.0/0
80 tcp 0.0.0.0/0 http://0.0.0.0/0
3-30100 tcp 0.0.0.0/0 http://0.0.0.0/0
42284 tcp 0.0.0.0/0 http://0.0.0.0/0
Cheers,
Joachim
Joachim Jacob
Contact details: http://www.bits.vib.be/index.php/about/80-team
On 06/10/2013 10:54 PM, Daniel Blankenberg wrote:
Hi Daniel,
This should work fine. You'll need to make sure that you have the host
setting in your universe_wsgi.ini like:
host = 0.0.0.0
and that the port 8080 is open/accessible in the security group
settings for the instance in AWS.
Thanks for using Galaxy,
Dan
On Jun 10, 2013, at 4:42 PM, Daniel Gorelik wrote:
Hello all,
I have a question: does anyone know if it is possible to install
Galaxy on an EC2 instance as you would on a local machine, not using
CloudMan, and then view it in the web browser by going to
public_ip:8080? I have been trying and it has not been working for me.
Thanks so much!
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Trackster error: not able to find the chrom len file
Dear all, I updated to the latest Galaxy stable code, and updated the datatypes_conf.xml. Trackster displays an error after opening a saved view: *** Error: The requested genome file (/home/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/dm3.len) could not be opened. Exiting! sort: write failed: standard output: Broken pipe sort: write error Couldn't open /home/galaxy/galaxy-dist/tool-data/shared/ucsc/chrom/dm3.len , No such file or directory *** In my universe_wsgi.ini I point to the correct directory where I keep my chrom len files. # Directory where chrom len files are kept, currently mainly used by trackster len_file_path = /mnt/referencedata/trackster Thanks for any advice, Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Cann't restart server after updating to the latest version
Hi Karen, Can you first switch to the user under which Galaxy runs, and go to the home directory of Galaxy: over there, run ./run.sh and watch which error messages appear. This should give you more information. It is recommended after a update to run Galaxy first like this. Kind regards, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 06/10/2013 10:33 AM, Karen Chait wrote: Hello, I upgraded to the latest version running: hg pull and then hg update. When trying to stop and start galaxy I got the following message: $ galaxy stop Stopping galaxy... *No PID file exists in paster.pid* done. $ galaxy start Starting galaxy... Entering daemon mode ...done. But the galaxy status remains ‘stopped’. Same happens when running restart to the server. Do you know what is the problem and how to overcome it? Thanks Karen ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Status of Galaxy Cloud Instance dev
Hi all, I am planning to launch a Galaxy Cloud Instance, to run for about one month. But I believe I've read somewhere a new image of Galaxy on AWS is being prepared right now. Can somebody update me on the status? Perhaps it is smart to wait a few days with launching through CloudLaunch until the new release is there? Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem. In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe. (BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way) Thanks, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 07:43 PM, Dave Bouvier wrote: Joachim, I believe the behavior you're experiencing may be due to an issue that was fixed between 9320:47ddf167c9f1 and the current stable branch. I would recommend updating to the most recent stable branch of galaxy-central. --Dave B. On 5/29/13 09:30:37.000, Joachim Jacob | VIB | wrote: Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy? Version info of the Tool Shed: * [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ * Version info of the Galaxy integrating the tool shed repository: * parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) * Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote: Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
Hi Peter, Yes, but your NCBI-blast executables were always packed in a tar.gz. The BLAT executable I want to use is just downloadable as such, without extracting needed. That's where it goes wrong. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On Thu 30 May 2013 11:52:54 AM CEST, Peter Cock wrote: On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | joachim.ja...@vib.be wrote: Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem. In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe. I don't understand why that issue is open, this is possible. https://trello.com/card/-/506338ce32ae458f6d15e4b3/584 e.g. I've successfully tested installing the pre-compiled NCBI BLAST+ binaries as an alternative to local compilation: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus See revision 19:c1a6e5aefee0 where I switched back to compiling: http://testtoolshed.g2.bx.psu.edu/repository/view_changeset?ctx_str=c1a6e5aefee0id=c1542e8b1988898c (BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way) Yes - sadly the best we can do as non-members is filter cards (pop out panel on right hand side) but that only looks at the titles. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy? Version info of the Tool Shed: * [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ * Version info of the Galaxy integrating the tool shed repository: * parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) * Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote: Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Creating tool shed repository: installation of dependencies fails.
The problem is most likely in ~/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py, line 100. 1. current_dir = os.path.abspath( os.path.join( work_dir, dir ) ) 2. while dir is set a couple of lines before as: dir = workdir (line 78). workdir is the temporary directory. 3. current_dir is then used by common_util.move_file on line 111. This explains the duplication in the path that I observed. It does not yet explain why the temporary directory is created in the wrong location. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote: Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote: Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: * Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat * Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. * ?xml version=1.0? tool_dependency package name=blat version=1.0.0 install version=1.0 actions action type=download_by_urlhttp://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat/action action type=move_file sourceblat/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install readme The BLAT binary for linux x86_64 is retrieved from UCSC. /readme /package /tool_dependency * Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] selecting multiple inputs for workflows not possible anymore
Allright. I just pulled in all changesets from stable to my cloud Galaxy, restarted, and everything works, at first glance. Thanks for your support! J Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/16/2013 05:21 PM, Dannon Baker wrote: Sorry, what I was saying was that you *can* pull these changes without leaving stable using 'hg pull -b stable http://bitbucket.org/galaxy/galaxy-central'. (Actually, if you're already on stable the -b is unnecessary -- you'll get the extra changesets but they won't be activated when you update) On Thu, May 16, 2013 at 11:08 AM, Joachim Jacob | VIB | joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: OK. So I believe I can wait for the next stable release then. I assume a do not want to go the default branch. thanks, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 tel:%2B32%209%20244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/16/2013 05:01 PM, Dannon Baker wrote: Your instance isn't quite up-to-date enough for the fix -- it's around 9549:47ddf167c9f1 -central / 9452:94caae7433a7 grafted in -stable. On Thu, May 16, 2013 at 10:56 AM, Joachim Jacob | VIB | joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: Running the Cloud Galaxy: galaxy@ip-10-39-130-141:/mnt/galaxyTools/galaxy-central$ hg summary parent: 9232:75f09617abaa release_2013.04.01 Merge next-stable to stable for release. branch: stable commit: 2 modified, 9 unknown (new branch head) update: 30 new changesets (update) galaxy@ip-10-39-130-141:/mnt/galaxyTools/galaxy-central$ hg branch stable Our production Galaxy: [galaxy@galaxy galaxy-dist]$ hg summary parent: 9292:2cc8d10988e0 security_2013.04.08 controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 267 unknown (new branch head) update: 11 new changesets (update) [galaxy@galaxy galaxy-dist]$ hg branch stable Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 tel:%2B32%209%20244.66.34 tel:%2B32%209%20244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/16/2013 04:35 PM, Dannon Baker wrote: I should have read the rest of my email before replying -- so you did update Galaxy for your cloud instance. What exact revision/branch are you running now? There was a brief period when select2 was enabled for the workflow page and it broke things. This should be fixed in -stable (and, of course, tip of -central). On Thu, May 16, 2013 at 10:12 AM, Joachim Jacob | VIB | joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: Hi all, I just found out that I cannot select multiple inputs in workflows anymore in our local Galaxy. This option used to work on these workflows in the past. I only get now a dropdown box, without the little icon at the top of the input list to select multiple inputs. I am a little puzzled of what's wrong. Any light in the dark? I have repeated these steps in a freshly started cloud galaxy instance, updated to the latest release, and imported that workflow. In that instance, I can select multiple files: but it is not anymore with the multiple selection, but in a 'dropdown-checkbox-list' (hope this is clear). The problem is here that only one input gets processed, even the input list contains multiple items. Every item that is checked gets into the inputbox. A second issue is that this entry (already in the input list) is not removed from the checkbox-list: so you can select
[galaxy-dev] Workflow: multiple input errors
Hi all, I have fired up an cloud instance of Galaxy, 2 persistent nodes, scalable up to 10. I have updated the code to the latest stable, after issues with not being able to select multiple input for workflows (more background: http://dev.list.galaxyproject.org/selecting-multiple-inputs-for-workflows-not-possible-anymore-td4659827.html) Running a workflow on multiple inputs (#=58) gives me - after a very long page loading - this error: Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910 Module galaxy.web.framework.middleware.error:*149* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#app_iter *=* self*.*application*(*environ*,* sr_checker*)*| Module paste.recursive:*84* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*application*(*environ*,* start_response*)*| Module paste.httpexceptions:*633* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*application*(*environ*,* start_response*)*| Module galaxy.web.framework.base:*128* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*handle_request*(* environ*,* start_response *)*| Module galaxy.web.framework.base:*184* in |handle_request| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#body *=* method*(* trans*,* kwargs *)*| Module galaxy.webapps.galaxy.controllers.workflow:*1443* in |run| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#job*,* out_data *=* tool*.*execute*(* trans*,* step*.*state*.*inputs*,* history*=*target_history*)*| Module galaxy.tools:*2342* in |execute| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*tool_action*.*execute*(* self*,* trans*,* incoming*=*incoming*,* set_output_hid*=*set_output_hid*,* history*=*history*,* kwargs *)*| Module galaxy.tools.actions:*397* in |execute| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#job*.*add_input_dataset*(* name*,* dataset *)*| Module galaxy.model:*247* in |add_input_dataset| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*input_datasets*.*append*(* JobToInputDatasetAssociation*(* name*,* dataset *)* *)*| Module ?:*4* in |__init__| Module sqlalchemy.orm.state:*82* in |initialize_instance| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* manager*.*events*.*original_init*(mixed*[**1**:**]**,* kwargs*)*| Module galaxy.model:*450* in |__init__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*dataset *=* dataset| Module sqlalchemy.orm.attributes:*150* in |__set__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*impl*.*set*(*instance_state*(*instance*)**,* instance_dict*(*instance*)**,* value*,* None*)*| Module sqlalchemy.orm.attributes:*590* in |set| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#value *=* self*.*fire_replace_event*(*state*,* dict_*,* value*,* old*,* initiator*)*| Module sqlalchemy.orm.attributes:*610* in |fire_replace_event| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#value *=* ext*.*set*(*state*,* value*,* previous*,* initiator *or* self*)*| Module sqlalchemy.orm.unitofwork:*69* in |set| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#sess*.*add*(*newvalue*)*| Module sqlalchemy.orm.session:*1091* in |add| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_save_or_update_state*(*state*)*| Module sqlalchemy.orm.session:*1100* in |_save_or_update_state| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_save_or_update_impl*(*state*)*| Module sqlalchemy.orm.session:*1267* in |_save_or_update_impl| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_update_impl*(*state*)*| Module sqlalchemy.orm.session:*1259* in |_update_impl| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_attach*(*state*)*| Module sqlalchemy.orm.session:*1286* in |_attach| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*(*mapperutil*.*state_str*(*state*)**,* state*.*key*)*| *InvalidRequestError: Can't attach instance HistoryDatasetAssociation at 0x8591c50; another instance with key (class 'galaxy.model.HistoryDatasetAssociation', (200,)) is already present in this session.* extra
Re: [galaxy-dev] Workflow: multiple input errors
It is most likely due to the cluster configuration, since on our non-cluster production machine doesn't have this behaviour. Cheers, J Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/17/2013 10:16 AM, Joachim Jacob | VIB | wrote: Hi all, I have fired up an cloud instance of Galaxy, 2 persistent nodes, scalable up to 10. I have updated the code to the latest stable, after issues with not being able to select multiple input for workflows (more background: http://dev.list.galaxyproject.org/selecting-multiple-inputs-for-workflows-not-possible-anymore-td4659827.html) Running a workflow on multiple inputs (#=58) gives me - after a very long page loading - this error: Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910 Module galaxy.web.framework.middleware.error:*149* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#app_iter *=* self*.*application*(*environ*,* sr_checker*)*| Module paste.recursive:*84* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*application*(*environ*,* start_response*)*| Module paste.httpexceptions:*633* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*application*(*environ*,* start_response*)*| Module galaxy.web.framework.base:*128* in |__call__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*handle_request*(* environ*,* start_response *)*| Module galaxy.web.framework.base:*184* in |handle_request| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#body *=* method*(* trans*,* kwargs *)*| Module galaxy.webapps.galaxy.controllers.workflow:*1443* in |run| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#job*,* out_data *=* tool*.*execute*(* trans*,* step*.*state*.*inputs*,* history*=*target_history*)*| Module galaxy.tools:*2342* in |execute| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* self*.*tool_action*.*execute*(* self*,* trans*,* incoming*=*incoming*,* set_output_hid*=*set_output_hid*,* history*=*history*,* kwargs *)*| Module galaxy.tools.actions:*397* in |execute| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#job*.*add_input_dataset*(* name*,* dataset *)*| Module galaxy.model:*247* in |add_input_dataset| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*input_datasets*.*append*(* JobToInputDatasetAssociation*(* name*,* dataset *)* *)*| Module ?:*4* in |__init__| Module sqlalchemy.orm.state:*82* in |initialize_instance| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#*return* manager*.*events*.*original_init*(mixed*[**1**:**]**,* kwargs*)*| Module galaxy.model:*450* in |__init__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*dataset *=* dataset| Module sqlalchemy.orm.attributes:*150* in |__set__| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*impl*.*set*(*instance_state*(*instance*)**,* instance_dict*(*instance*)**,* value*,* None*)*| Module sqlalchemy.orm.attributes:*590* in |set| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#value *=* self*.*fire_replace_event*(*state*,* dict_*,* value*,* old*,* initiator*)*| Module sqlalchemy.orm.attributes:*610* in |fire_replace_event| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#value *=* ext*.*set*(*state*,* value*,* previous*,* initiator *or* self*)*| Module sqlalchemy.orm.unitofwork:*69* in |set| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#sess*.*add*(*newvalue*)*| Module sqlalchemy.orm.session:*1091* in |add| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_save_or_update_state*(*state*)*| Module sqlalchemy.orm.session:*1100* in |_save_or_update_state| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_save_or_update_impl*(*state*)*| Module sqlalchemy.orm.session:*1267* in |_save_or_update_impl| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_update_impl*(*state*)*| Module sqlalchemy.orm.session:*1259* in |_update_impl| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id=e967e0737bb63910#self*.*_attach*(*state*)*| Module sqlalchemy.orm.session:*1286* in |_attach| | http://ec2-54-226-11-122.compute-1.amazonaws.com/workflow/run?id
[galaxy-dev] Cloudman - missing security rules in wiki page
Hi all, I have tried to fire up an Cloud instance of Galaxy following the wiki page: http://wiki.galaxyproject.org/CloudMan However, two attempts failed (Cloudman interface did not come up - I could SSH into the machine). Next, I have tried the CloudLaunch interface (added the link to the wiki page), and tried that way. It succeeded. It seems that the Security group settings differ between what CloudLaunch sets, and the instructions written on the wiki page. In wiki: Ports ProtocolSource GalaxyWeb 0-65535 tcp sg-1b495b72 20-21 tcp 0.0.0.0/0 22 tcp 0.0.0.0/0 80 tcp 0.0.0.0/0 3-30100 tcp 0.0.0.0/0 42284 tcp 0.0.0.0/0 Set by CloudLaunch: Ports ProtocolSource CloudMan 1-65535 tcp sg-61ba790a 1-65535 udp sg-61ba790a -1 icmpsg-61ba790a 20-21 tcp 0.0.0.0/0 22 tcp 0.0.0.0/0 80 tcp 0.0.0.0/0 3-30100 tcp 0.0.0.0/0 42284 tcp 0.0.0.0/0 Since I did not test manually launching the cloud instance with these settings, I post them here. If anybody has tested these settings, perhaps the wiki page can be updated? Cheers, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] selecting multiple inputs for workflows not possible anymore
Hi all, I just found out that I cannot select multiple inputs in workflows anymore in our local Galaxy. This option used to work on these workflows in the past. I only get now a dropdown box, without the little icon at the top of the input list to select multiple inputs. I am a little puzzled of what's wrong. Any light in the dark? I have repeated these steps in a freshly started cloud galaxy instance, updated to the latest release, and imported that workflow. In that instance, I can select multiple files: but it is not anymore with the multiple selection, but in a 'dropdown-checkbox-list' (hope this is clear). The problem is here that only one input gets processed, even the input list contains multiple items. Every item that is checked gets into the inputbox. A second issue is that this entry (already in the input list) is not removed from the checkbox-list: so you can select multiple times the same input. (although this might be wanted in some cases?) Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] selecting multiple inputs for workflows not possible anymore
OK. So I believe I can wait for the next stable release then. I assume a do not want to go the default branch. thanks, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/16/2013 05:01 PM, Dannon Baker wrote: Your instance isn't quite up-to-date enough for the fix -- it's around 9549:47ddf167c9f1 -central / 9452:94caae7433a7 grafted in -stable. On Thu, May 16, 2013 at 10:56 AM, Joachim Jacob | VIB | joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: Running the Cloud Galaxy: galaxy@ip-10-39-130-141:/mnt/galaxyTools/galaxy-central$ hg summary parent: 9232:75f09617abaa release_2013.04.01 Merge next-stable to stable for release. branch: stable commit: 2 modified, 9 unknown (new branch head) update: 30 new changesets (update) galaxy@ip-10-39-130-141:/mnt/galaxyTools/galaxy-central$ hg branch stable Our production Galaxy: [galaxy@galaxy galaxy-dist]$ hg summary parent: 9292:2cc8d10988e0 security_2013.04.08 controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 267 unknown (new branch head) update: 11 new changesets (update) [galaxy@galaxy galaxy-dist]$ hg branch stable Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 tel:%2B32%209%20244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/16/2013 04:35 PM, Dannon Baker wrote: I should have read the rest of my email before replying -- so you did update Galaxy for your cloud instance. What exact revision/branch are you running now? There was a brief period when select2 was enabled for the workflow page and it broke things. This should be fixed in -stable (and, of course, tip of -central). On Thu, May 16, 2013 at 10:12 AM, Joachim Jacob | VIB | joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: Hi all, I just found out that I cannot select multiple inputs in workflows anymore in our local Galaxy. This option used to work on these workflows in the past. I only get now a dropdown box, without the little icon at the top of the input list to select multiple inputs. I am a little puzzled of what's wrong. Any light in the dark? I have repeated these steps in a freshly started cloud galaxy instance, updated to the latest release, and imported that workflow. In that instance, I can select multiple files: but it is not anymore with the multiple selection, but in a 'dropdown-checkbox-list' (hope this is clear). The problem is here that only one input gets processed, even the input list contains multiple items. Every item that is checked gets into the inputbox. A second issue is that this entry (already in the input list) is not removed from the checkbox-list: so you can select multiple times the same input. (although this might be wanted in some cases?) Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 tel:%2B32%209%20244.66.34 tel:%2B32%209%20244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] starting run_reports.sh fails
run_reports.sh dies silently. Its last will in reports_webapp.log states: Traceback (most recent call last): File ./scripts/paster.py, line 33, in module serve.run() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1056, in run invoke(command, command_name, options, args[1:]) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1062, in invoke exit_code = runner.run(args) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 227, in run result = self.command() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 650, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 375, in loadobj return context.create() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 813, in create return self.object_type.invoke(self) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 249, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 97, in fix_call val = callable(*args, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/buildapp.py, line 61, in app_factory add_ui_controllers( webapp, app ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/buildapp.py, line 33, in add_ui_controllers from galaxy.web.base.controller import BaseUIController File /home/galaxy/galaxy-dist/lib/galaxy/web/base/controller.py, line 23, in module from galaxy.visualization.genome.visual_analytics import get_tool_def File /home/galaxy/galaxy-dist/lib/galaxy/visualization/genome/visual_analytics.py, line 3, in module from galaxy.tools.parameters.basic import IntegerToolParameter, FloatToolParameter, SelectToolParameter File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 57, in module from galaxy.visualization.genome.visual_analytics import TracksterConfig ImportError: cannot import name TracksterConfig Anybody knowing the recipe to reanimate the patient? Thanks. BTW, this is the first time I am starting up run_reports.sh after the 1 april/8 april update. Did not had this issue before. Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] starting run_reports.sh fails
Already reported in Trello https://trello.com/c/o0S3klAF Thanks for the fix! Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 05/15/2013 03:19 PM, Joachim Jacob | VIB | wrote: run_reports.sh dies silently. Its last will in reports_webapp.log states: Traceback (most recent call last): File ./scripts/paster.py, line 33, in module serve.run() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1056, in run invoke(command, command_name, options, args[1:]) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1062, in invoke exit_code = runner.run(args) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 227, in run result = self.command() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 650, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 375, in loadobj return context.create() File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 813, in create return self.object_type.invoke(self) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 249, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File /home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 97, in fix_call val = callable(*args, **kw) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/buildapp.py, line 61, in app_factory add_ui_controllers( webapp, app ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/buildapp.py, line 33, in add_ui_controllers from galaxy.web.base.controller import BaseUIController File /home/galaxy/galaxy-dist/lib/galaxy/web/base/controller.py, line 23, in module from galaxy.visualization.genome.visual_analytics import get_tool_def File /home/galaxy/galaxy-dist/lib/galaxy/visualization/genome/visual_analytics.py, line 3, in module from galaxy.tools.parameters.basic import IntegerToolParameter, FloatToolParameter, SelectToolParameter File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 57, in module from galaxy.visualization.genome.visual_analytics import TracksterConfig ImportError: cannot import name TracksterConfig Anybody knowing the recipe to reanimate the patient? Thanks. BTW, this is the first time I am starting up run_reports.sh after the 1 april/8 april update. Did not had this issue before. Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Protocol for changing the section of an installed tool shed repo
Hi all, Stupidly enough, I installed a tool shed repo under 'root' of the toolbox. Yes, it was EMBOSS5.0 (btw, installation went flawless!). 1. From the admin menu, I choose to unactivate the repo (manage tool shed repo's) 2. From the admin menu, I reinstalled the tool shed from Main Tool Shed, and chose to put it under a section 'EMBOSS'. This to no avail: all tools still under root. 3. I manually edited the shed_tool_conf.xml and added the section tags 4. The section is now displayed, containing the EMBOSS tools. BUT the tools under the root of the toolbox are still there. Any assistance here? Cheers, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
I can confirm that the proxy settings are the reason for the failing
export. When I go to localhost:8080 directly, I can export large files
from the Data Library.
When going via the proxy using the URL, download of large files does not
work. Here is a hint on what the solution might be
(http://serverfault.com/questions/185894/proxy-error-502-reason-error-reading-from-remote-server-with-apache-2-2-3-de)
*** The error in the browser:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request
/POST /library_common/act_on_multiple_datasets
http://galaxy.bits.vib.be/library_common/act_on_multiple_datasets/.
Reason: *Error reading from remote server*
*** The error in the http logs:
[Fri Mar 29 10:22:03 2013] [error] [client 157.193.10.20] (70007)The
timeout specified has expired: proxy: error reading status line from
remote server localhost, referer:
http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1
[Fri Mar 29 10:22:03 2013] [error] [client 157.193.10.20] proxy: Error
reading from remote server returned by
/library_common/act_on_multiple_datasets, referer:
http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1
*** Our proxy settings
I would really appreciate if somebody could have a look at our current
Apache proxy settings. Since I suspect the problem to be a time-out, I
have tried modifying related parameters, with no luck.
===
[root@galaxy conf.d]# cat galaxy_web.conf
NameVirtualHost 157.193.230.103:80
VirtualHost 157.193.230.103:80
ServerName galaxy.bits.vib.be
SetEnv force-proxy-request-1.0 1# tried this, does not help
SetEnv proxy-nokeepalive 1 # tried this, does not help
KeepAliveTimeout 600 # tried this, does not help
ProxyPass /library_common/act_on_multiple_datasets
http://galaxy.bits.vib.be/library_common /act_on_multiple_datasets max=6
keepalive=On timeout=600 retry=10 #tried this, does not help.
Proxy balancer://galaxy
BalancerMember http://localhost:8080
BalancerMember http://localhost:8081
BalancerMember http://localhost:8082
BalancerMember http://localhost:8083
BalancerMember http://localhost:8084
BalancerMember http://localhost:8085
BalancerMember http://localhost:8086
BalancerMember http://localhost:8087
BalancerMember http://localhost:8088
BalancerMember http://localhost:8089
BalancerMember http://localhost:8090
BalancerMember http://localhost:8091
BalancerMember http://localhost:8092
/Proxy
RewriteEngine on
RewriteLog /tmp/apacheGalaxy.log
# Location /
# AuthType Basic
# AuthBasicProvider ldap
# AuthLDAPURL ldap://smeagol.vib.be:389/DC=vib,DC=local?sAMAccountName
# AuthLDAPBindDN vib\administrator
# AuthLDAPBindPassword tofillin
# AuthzLDAPAuthoritative off
# Require valid-user
# # Set the REMOTE_USER header to the contents of the LDAP query
response's uid attribute
# RequestHeader set REMOTE_USER %{AUTHENTICATE_sAMAccountName}
# /Location
RewriteRule ^/static/style/(.*)
/home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L]
RewriteRule ^/static/scripts/(.*)
/home/galaxy/galaxy-dist/static/scripts/packed/$1 [L]
RewriteRule ^/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L]
RewriteRule ^/favicon.ico
/home/galaxy/galaxy-dist/static/favicon.ico [L]
RewriteRule ^/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L]
RewriteRule ^(.*) balancer://galaxy$1 [P]
/VirtualHost
==
Thanks,
Joachim
Joachim Jacob
Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS)
http://www.bits.vib.be
@bitsatvib
On 03/28/2013 03:21 PM, Joachim Jacob | VIB | wrote:
OK, it seems to be a proxy error.
When the proxy does not receive data from the server, it times out,
and closes the connection.
I think the process that packs the datasets takes too long, so the
connection is closed before the packaging is finished? Just a gues...
From the httpd logs:
=
[Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] (70007)The
timeout specified has expired: proxy: error reading status line from
remote server localhost, referer:
http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1
[Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] proxy: Error
reading from remote server returned by
/library_common/act_on_multiple_datasets, referer:
http://galaxy.bits.vib.be/library_common/browse_library?sort
Re: [galaxy-dev] FTP upload - symlink to uploaded data
Hi Rob, Indeed, I had/have some difficulties with setting temporary directories. The problem was that FTP uploaded data was first copied to TMPDIR prior to being put in the database directory. My solution: I extended the /tmp partition to several GB's, by mounting a bigger device over it. In addition, I have a large network share, which is mounted on /mnt/galaxytemp. The __new_file_path__ points to here. A bit messy indeed. I just had another discussion about the temporary directories with John Chilton and Jeremy Goecks, which you can read here: https://bitbucket.org/galaxy/galaxy-central/pull-request/139/letting-cuffdiff-use-__new_file_path__-as/diff From what I understood, __new_file_path__ is going to be phased out, in favour of __job_working_directory__. But apparently, the job_working_directory is not a temporary directory (in my case, it contains symlinks from the job_working_directory to database/files. In addition, job_working_directory is default part of the database directory of Galaxy.) The suggestion is to set TMPDIR env variable to a directory you specify. I have one file in /home/galaxy that contains the environment settings, and which gets sourced in the init script that launches Galaxy. Cheers, Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Wed 27 Mar 2013 08:44:08 PM CET, Rob Hooft wrote: Joachim, Nate, Leon Mei pointed me to a mailing list post of August 2012 where you two discussed a problem with uploads to Galaxy filling up /tmp. I think I have traced this down now after we suffered from this too several times. There are a number of places where temporary files are configurable in galaxy, but there is (at least) one place that uses the Python default directory (can be set with TMPDIR or some other envvars, but if you don't it is often /tmp). The unconfigurable place is tools/data_source/upload.py, where the code reads: if dataset.type == 'url': try: page = urllib.urlopen( dataset.path ) #page will be .close()ed by sniff methods temp_name, dataset.is_multi_byte = sniff.stream_to_file( page, prefix='url_paste', source_encoding=util.get_charset_from_http_headers( page.headers ) ) except Exception, e: file_err( 'Unable to fetch %s\n%s' % ( dataset.path, str( e ) ), dataset, json_file ) return dataset.path = temp_name sniff.stream_to_file uses the tempfile module, and since there is no dir= in the argument list to this call, the temporary file is made in /tmp. The central solution for the main galaxy code is in lib/galaxy/config.py: self.new_file_path = resolve_path( kwargs.get( new_file_path, database/tmp ), self.root ) tempfile.tempdir = self.new_file_path But this assignment to tempdir does not help in this case because upload.py is a tool? It would be nice to fix this, which we can obviously do ourselves for our andromeda deployment, but it would be better to do it centrally. Regards, Rob -- Rob W.W. Hooft Chief Technology Officer BioAssist, Netherlands Bioinformatics Centre http://www.nbic.nl/Skype: robhooftGSM: +31 6 27034319 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
Hi Assaf, After all, the problem appears not to be total size of the history, but the size of the individual datasets. Now, histories which contain big datasets (1GB) imported from Data Libraries causes the exporting process to crash. Can somebody confirm if this is a bug? I uploaded the datasets to a directory, which are then imported from that directory into a Data Library. Downloading data sets 1GB from a data library directly (as tar.gz) also crashes. Note: I have re-enabled abrt, but waiting for some jobs to be finished to restart. Cheers, Joachim. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Tue 26 Mar 2013 03:45:43 PM CET, Assaf Gordon wrote: Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. A crashing python process crosses the fine boundary between the Galaxy code and Python internals... perhaps the Galaxy developers can help with this problem. It would be helpful to find a reproducible case with a specific history or a specific sequence of events, then someone can help you with the debugging. Once you find a history that causes a crash (every time or sometimes, but in a reproducible way), try to pinpoint when exactly it happens: Is it when you start preparing the export (and export_history.py is running as a job), or when you start downloading the exported file. (I'm a bit behind on the export mechanism, so perhaps there are other steps involved?). Couple of things to try: 1. set cleanup_job=never in your universe_wsgi.ini - this will keep the temporary files, and will help you re-produce jobs later. 2. Enable abrt again - it is not the problem (just the symptom). You can cleanup the /var/spool/abrt/XXX directory from previous crash logs, then reproduce a new crash, and look at the collected files (assuming you have enough space to store at least one crash). In particular, look at the file called coredump - it will tell you which script has crashed. Try running: $ file /var/spool/abrt//coredump coredump ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'python XX.py' Instead of .py it would show the python script that crashed (hopefully with full command-line parameters). It won't show which python statement caused the crash, but it will point in the right direction. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). It could be a bad dataset (file on disk), or a problem in the database, or something completely different (a bug in the python archive module). No point guessing until there are more details. -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
OK, it seems to be a proxy error. When the proxy does not receive data from the server, it times out, and closes the connection. I think the process that packs the datasets takes too long, so the connection is closed before the packaging is finished? Just a gues... From the httpd logs: = [Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] (70007)The timeout specified has expired: proxy: error reading status line from remote server localhost, referer: http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1 [Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] proxy: Error reading from remote server returned by /library_common/act_on_multiple_datasets, referer: http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1 = See if changing time out settings fixes this issue. Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/28/2013 02:58 PM, Joachim Jacob | VIB | wrote: Hi Assaf, After all, the problem appears not to be total size of the history, but the size of the individual datasets. Now, histories which contain big datasets (1GB) imported from Data Libraries causes the exporting process to crash. Can somebody confirm if this is a bug? I uploaded the datasets to a directory, which are then imported from that directory into a Data Library. Downloading data sets 1GB from a data library directly (as tar.gz) also crashes. Note: I have re-enabled abrt, but waiting for some jobs to be finished to restart. Cheers, Joachim. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Tue 26 Mar 2013 03:45:43 PM CET, Assaf Gordon wrote: Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. A crashing python process crosses the fine boundary between the Galaxy code and Python internals... perhaps the Galaxy developers can help with this problem. It would be helpful to find a reproducible case with a specific history or a specific sequence of events, then someone can help you with the debugging. Once you find a history that causes a crash (every time or sometimes, but in a reproducible way), try to pinpoint when exactly it happens: Is it when you start preparing the export (and export_history.py is running as a job), or when you start downloading the exported file. (I'm a bit behind on the export mechanism, so perhaps there are other steps involved?). Couple of things to try: 1. set cleanup_job=never in your universe_wsgi.ini - this will keep the temporary files, and will help you re-produce jobs later. 2. Enable abrt again - it is not the problem (just the symptom). You can cleanup the /var/spool/abrt/XXX directory from previous crash logs, then reproduce a new crash, and look at the collected files (assuming you have enough space to store at least one crash). In particular, look at the file called coredump - it will tell you which script has crashed. Try running: $ file /var/spool/abrt//coredump coredump ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'python XX.py' Instead of .py it would show the python script that crashed (hopefully with full command-line parameters). It won't show which python statement caused the crash, but it will point in the right direction. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). It could be a bad dataset (file on disk), or a problem in the database, or something completely different (a bug in the python archive module). No point guessing until there are more details. -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use
Re: [galaxy-dev] Exporting histories fails: no space left on device
Hi Gordon, Thanks for your assistance and the recommendations. Freezing postgres sounds like hell to me :-) abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/25/2013 05:18 PM, Assaf Gordon wrote: Hello Joachim, Couple of things to check: On Mar 25, 2013, at 10:01 AM, Joachim Jacob | VIB | wrote: Hi, About the exporting of history, which fails: 1. the preparation seems to work fine: meaning: choosing 'Export this history' in the History menu leads to a URL that reports initially that the export is still in progress. 2. when the export is finished, and I click the download link, the root partition fills and the browser displays Error reading from remote server. A folder ccpp-2013-03-25-14:51:15-27045.new is created in the directory /var/spool/abrt, which fills the root partition. Something in your export is likely not finishing fine, but crashes instead (either the creation of the archive, or the download). The folder /var/spool/abrt/ccpp- (and especially a file named coredump) hints that the program crashed. abrt is a daemon (at least on Fedora) that monitors crashes and tries to keep all relevant information about the program which crashed (http://docs.fedoraproject.org/en-US/Fedora/13/html/Deployment_Guide/ch-abrt.html). So what might have happened, is that a program (galaxy's export_history.py or other) crashed during your export, and then abrt picked-up the pieces (storing a memory dump, for example), and then filled your disk. The handler reports in its log: galaxy.jobs DEBUG 2013-03-25 14:38:33,322 (8318) Working directory for job is: /mnt/galaxydb/job_working_directory/008/8318 galaxy.jobs.handler DEBUG 2013-03-25 14:38:33,322 dispatching job 8318 to local runner galaxy.jobs.handler INFO 2013-03-25 14:38:33,368 (8318) Job dispatched galaxy.jobs.runners.local DEBUG 2013-03-25 14:38:33,432 Local runner: starting job 8318 galaxy.jobs.runners.local DEBUG 2013-03-25 14:38:33,572 executing: python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat galaxy.jobs.runners.local DEBUG 2013-03-25 14:41:29,420 execution finished: python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat galaxy.jobs DEBUG 2013-03-25 14:41:29,476 Tool did not define exit code or stdio handling; checking stderr for success galaxy.tools DEBUG 2013-03-25 14:41:29,530 Error opening galaxy.json file: [Errno 2] No such file or directory: '/mnt/galaxydb/job_working_directory/008/8318/galaxy.json' galaxy.jobs DEBUG 2013-03-25 14:41:29,555 job 8318 ended The system reports: Mar 25 14:51:26 galaxy abrt[16805]: Write error: No space left on device Mar 25 14:51:27 galaxy abrt[16805]: Error writing '/var/spool/abrt/ccpp-2013-03-25-14:51:15-27045.new/coredump' One thing to try: if you have galaxy keeping temporary files, try running the export command manually: === python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat === Another thing to try: modify export_history.py, adding debug messages to track progress and whether it finishes or not. And: check the abrt program's GUI, perhaps you'll see previous crashes that were stored successfully, providing more information about which program crashed. As a general rule, it's best to keep the /var directory on a separate partition for production systems, exactly so that filling it up with junk wouldn't intervene with other programs. Even better, set each sub-directory of /var to a dedicated partition, so that filling up /var/log or /var/spool would not fill up /var/lib/pgsql and stop Postgres from working. -gordon
Re: [galaxy-dev] Issue with viz of mapped data in Trackster
OK (I see now I have posted twice the same error from the log - silly of me, sorry). First this: yes, the permissions in the data library are set so that only I and one other person can access the data library (set by one sharing role for both of us) The entries in the job table related to the summary tree converter: -- select * from job where tool_id like 'CONVERTER%summary_tree%' order by id desc; -- id |create_time |update_time | history_id | tool_id| tool_version | state | info | command_line | param_filename | runner_name | stdout | stderr | traceback | s ession_id | job_runner_name | job_runner_external_id | library_folder_id | user_id | imported | object_store_id | params | handler | exit_code --++++--+--+---+--- +-- --- ---++-+ +---+---+-- --+-++---+-+--+-++--+--- 8186 | 2013-03-19 09:22:29.165135 | 2013-03-19 09:27:44.335526 |622 | CONVERTER_bam_to_summary_tree_0 | 1.0.0 | ok| | python /home/galaxy/galaxy-dist/lib/galaxy/datatypes/converte rs/sam_or_bam_to_summary_tree_converter.py --bam /mnt/galaxydb/files/010/dataset_10477.dat /mnt/galaxydb/files/_metadata_files/001/metadata_1481.dat /m nt/galaxydb/files/013/dataset_13752.dat || | | | | 6188 | local:/// | 25218 | | 1 | f| || handler3 | 0 8185 | 2013-03-19 09:22:19.463602 | 2013-03-19 09:22:23.644461 |622 | CONVERTER_bam_to_summary_tree_0 | 1.0.0 | new | | || | | | | 6188 | local:/// | | | 1 | f | || handler2 | 8184 | 2013-03-19 09:20:45.225881 | 2013-03-19 09:26:54.337624 |622 | CONVERTER_bam_to_summary_tree_0 | 1.0.0 | ok| | python /home/galaxy/galaxy-dist/lib/galaxy/datatypes/converte rs/sam_or_bam_to_summary_tree_converter.py --bam /mnt/galaxydb/files/010/dataset_10461.dat /mnt/galaxydb/files/_metadata_files/001/metadata_1478.dat /m nt/galaxydb/files/013/dataset_13749.dat || | | | | 6188 | local:/// | 25112 | | 1 | f| || handler1 | 0 8183 | 2013-03-19 09:20:44.865602 | 2013-03-19 09:27:19.370773 |622 | CONVERTER_bam_to_summary_tree_0 | 1.0.0 | ok| | python /home/galaxy/galaxy-dist/lib/galaxy/datatypes/converte rs/sam_or_bam_to_summary_tree_converter.py --bam /mnt/galaxydb/files/010/dataset_10461.dat /mnt/galaxydb/files/_metadata_files/001/metadata_1477.dat /m nt/galaxydb/files/013/dataset_13750.dat || | | | | 6188 | local:/// | 25113 | | 1 | f| || handler1 | 0 8163 | 2013-03-19 08:29:59.965262 | 2013-03-19 08:34:30.257029 |622 | CONVERTER_bam_to_summary_tree_0 | 1.0.0 | ok| | python /home/galaxy/galaxy-dist/lib/galaxy/datatypes/converte rs/sam_or_bam_to_summary_tree_converter.py --bam /mnt/galaxydb/files/010/dataset_10457.dat /mnt/galaxydb/files/_metadata_files/001/metadata_1475.dat /m nt/galaxydb/files/013/dataset_13719.dat Thanks, Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/19/2013 10:54 PM, Jeremy Goecks wrote: Indeed, following error occurs: 157.193.10.150 - - [19/Mar/2013:08:30:06 +0200] GET /visualization/trackster HTTP/1.1 200 - http://galaxy.bits.vib.be/; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+ galaxy.webapps.galaxy.api.datasets ERROR 2013-03-19 08:30:13,722 Error in dataset API at listing contents: : Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 41, in show is_true( kwd.get( 'retry', False ) ) ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File
Re: [galaxy-dev] Issue with viz of mapped data in Trackster
Indeed, following error occurs: 157.193.10.150 - - [19/Mar/2013:08:30:06 +0200] GET /visualization/trackster HTTP/1.1 200 - http://galaxy.bits.vib.be/; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+ galaxy.webapps.galaxy.api.datasets ERROR 2013-03-19 08:30:13,722 Error in dataset API at listing contents: : Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 41, in show is_true( kwd.get( 'retry', False ) ) ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File /home/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py, line 814, in has_data st = summary_tree_from_file( self.converted_dataset.file_name ) File /home/galaxy/galaxy-dist/lib/galaxy/visualization/tracks/summary.py, line 114, in summary_tree_from_file st = cPickle.load( st_file ) EOFError I have found the root cause I believe: when I visualize datasets from a data library, the data is not displayed. Getting them from the source history gets displayed is OK Importing them from a data library in a history, and displaying in trackster works Thanks, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/18/2013 07:14 PM, Jeremy Goecks wrote: It's likely that the job creating the summary tree failed for some reason. Can you please look through your database for summary tree jobs and see if they have any standard out/err that could provide insight into the problem? Thanks, J. On Mar 15, 2013, at 9:47 AM, Joachim Jacob | VIB | wrote: Hi all, Since the last updata of Galaxy, I was not able anymore to view newly added tracks to existing or newly created visualisations. In the webhandler, I have traced down this error: 193.191.128.2 - - [14/Mar/2013:11:02:13 +0200] GET /api/datasets/407a09c0b5976b4e?hda_ldda=hdadata_type=converted_datasets_statechrom=chr4 HTTP/1.1 500 - http://galaxy.bits.vib.be/visualization/trackster?id=1ec2297821c7f2cb; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+ galaxy.webapps.galaxy.api.datasets ERROR 2013-03-14 11:03:02,409 Error in dataset API at listing contents: : Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 41, in show is_true( kwd.get( 'retry', False ) ) ) File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py, line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File /home/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py, line 814, in has_data st = summary_tree_from_file( self.converted_dataset.file_name ) File /home/galaxy/galaxy-dist/lib/galaxy/visualization/tracks/summary.py, line 114, in summary_tree_from_file st = cPickle.load( st_file ) EOFError Thanks for any assistance, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Exporting histories fails: no space left on device
Hi all, Exporting histories fails on our server: Reason: *Error reading from remote server. *When looking at the logs and the system: tail /var/log/messages Mar 19 15:52:47 galaxy abrt[25605]: Write error: No space left on device Mar 19 15:52:49 galaxy abrt[25605]: Error writing '/var/spool/abrt/ccpp-2013-03-19-15:52:37-13394.new/coredump' So I watched my system when I repeated the export, and saw that Galaxy fills up the root directory (/), instead of any temporary directory. Somebody has an idea where to adjust this setting, so the export function uses any temporary directory? Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Bam to sam error
Hello, I got an error on our local Galaxy using bam-to-sam. The BAM file is a result of a Tophat version 1 run. The error Error sorting alignments from (/mnt/galaxydb/files/006/dataset_6733.dat), error An error occurred with this dataset:/Samtools Version: 0.1.18 (r982:295) Error sorting alignments from (/mnt/galaxydb/files/006/dataset_6733.dat),/ Thanks for the assistance, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy Reports: 'unfinished jobs' seems not to do the job
Hi all, I make increasingly use of the Galaxy Reports tool. Until recently I started it and shut it down again. Now I run it continuously. I have noticed that a lot of the jobs listed in 'unfinished jobs' are actually finished. Any hint on how to clean up the database? Which table? Perhaps I can make a Trello card: 'Provide a button to remove the jobs from the unfinished jobs page' ? Thanks for taking me further, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Installing htseq-count via Toolshed: error installing dependency.
Hi all, Installing htseq-count with dependencies fails on my system: htseq-count installs four dependencies (numpy, picard, samtools, htseq) , of which one fails (htseq), strangely enough. The error is: error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 2] No such file or directory: '/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/test-easy-install-856.pth' The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was: /apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/ This directory does not currently exist. Please create it and try again, or choose a different installation directory (using the -d or --install-dir option). The /apps directory is completely owned by Galaxy. I tried to create that missing directory (reported above), but got a little stuck, since uninstalling the dependency removed that directory again. The tool created the folder /apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e with contents: INSTALLATION.log lib Next I looked to the tool_dependency.xml, but with my limited experience did not see something obvious. Thanks for any hint, (prefer to get the installation working from the toolshed) Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy Reports: 'unfinished jobs' seems not to do the job
Hi Hans, OK, thanks: diving into the job table. Your question: why you consider the job as being finished, and Galaxy thinks the job is still running is the core issue that I need to find out. None of those jobs have entries in the ps list. Currently, I think that those jobs are still listed because they were not nicely shut down by Galaxy, but because of a crash (e.g. of the virtual machine). Upon restarting Galaxy, Galaxy seems not to check whether those jobs are currently running, but seems to depend entirely on the job table to fill the 'unfinished jobs' table. Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/07/2013 11:21 AM, Hans-Rudolf Hotz wrote: Hi Joachim I make increasingly use of the Galaxy Reports tool. Until recently I started it and shut it down again. Now I run it continuously. I have noticed that a lot of the jobs listed in 'unfinished jobs' are actually finished. Any hint on how to clean up the database? Which table? Perhaps I can make a Trello card: 'Provide a button to remove the jobs from the unfinished jobs page' ? Well, you can modify the entries in the 'job' table.As always, be very careful when you directly access the MySQL or PostgreSQL database. In your case I would set the state to 'error', eg: update job set state='error' where id=123456789; Once again be careful. I would examine each job individually. And try to figure out, why you consider the job as being finished, and Galaxy thinks the job is still running Regards, Hans-Rudolf Thanks for taking me further, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy: Dependency error htseq_count
Hi Lance, [apologies for directly emailing to you and not through the galaxy-dev list. I will post it on the dev list later on] I am trying to install htseq-count, but it fails during the dependency installation of htseq. The error I get is: error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 2] No such file or directory: '/apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/test-easy-install-30482.pth' The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was: /apps/htseq/0.5.3p9/lparsons/htseq_count/f320093f1e8e/lib64/python/ This directory does not currently exist. Please create it and try again, or choose a different installation directory (using the -d or --install-dir option). Any idea whether it is related to my setup (and what I can do about it)? Note: only the htseq_count dependency is not being installed. Thanks for providing the htseq_count wrapper! Now getting it through the toolshed will make it again a little closer to a perfect world :-) Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 12/17/2012 05:28 PM, Lance Parsons wrote: Thanks for looking into this Greg. The environment where I'm having this problem is our testing and production systems. Both environments are CentOS release 5.8 (Final) with Mercurial version 2.4.1. Interestingly, when I tried the same thing with a development setup on my OSX box (10.6.8) with Mercurial 2.4 I did not have the same issue. One difference is that on my testing/production boxes I do have an older version of the tool installed, so perhaps that is the issue. I also tried uninstalling, but that led to the (expected) error that that a previous changeset is installed: The tool shed repository htseq_count with owner lparsons and changeset revision f320093f1e8e was previously installed using changeset revision 5d969cb56112. The repository has been uninstalled, however, so reinstall the original repository instead of installing it again. You can get the latest updates for the repository using the Get updates option from the repository's Repository Actions pop-up menu. Click here to manage the repository. I then attempted to reinstall and I got a javascript message the cloning failed, and the repo is now stuck in the cloning state. I did not see anything in the paster logs regarding this at all. Here are the steps I took on my local system: 1. Upload htseq-count v0.3 to local toolshed 2. Install on local galaxy 3. Upload htseq-count v0.3-release2 to toolshed [Errno 2] No such file or directory: '/Users/lparsons/Documents/projects/sequencing/galaxy/galaxy-test1/database/community_files/000/repo_3/./tool_data_table_conf.xml.sample' STDOUT from paster htseq-count.xml does not exist! sam_fa_indices.loc.sample does not exist! static/images/count_modes.png does not exist! test-data/htseq-test-paired.bam does not exist! test-data/htseq-test-paired_counts.tsv does not exist! test-data/htseq-test-paired_othercounts.tsv does not exist! test-data/htseq-test.bam does not exist! test-data/htseq-test.gff does not exist! test-data/htseq-test.sam does not exist! test-data/htseq-test_counts.tsv does not exist! test-data/htseq-test_othercounts.tsv does not exist! tool_data_table_conf.xml.sample does not exist! 4. Second attempt to upload The file 'htseq-count_0.3.release2.tar.gz' has been successfully uncompressed and uploaded to the repository. 1 undesirable items (.hg .svn .git directories, .DS_Store, hgrc files, etc) were removed from the archive. 13 files were removed from the repository root. 5. Get repository updates The installed repository named 'htseq_count' has been updated to change set revision '74a6a8221337'. Greg Von Kuster wrote: Hi Lance, I tried several scenarios today in an attempt to reproduce this but I was not able to. Have you tried uninstalling the repository (check the checkbox to completely uninstall it) and then reinstall it? I'm trying to reproduce this on a mac running mercurial version 2.2.3 and Python 2.5. What's your environment? Greg Von Kuster On Dec 13, 2012, at 2:30 PM, Lance Parsons wrote: Since I've had issues with workflow and the toolshed in the past, I have changed my workflow so that my only interaction with the toolshed is either via uploading a tarball via the web interface or installing a tool into my local Galaxy via the web interface (or occasionally just downloading tarballs). In this specific case, I uploaded a tar.gz file of the entire tool wrapper directory. I did NOT increment the tool version. In the past, incrementing the tool version has generally worked fine. However, when I leave the tool version the same, but change
Re: [galaxy-dev] Environment variables reset after manually restarting Galaxy
Hi Nate, I am sorry I missed your reply... Thanks for your input, will try this soon. Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/13/2013 06:31 PM, Nate Coraor wrote: On Feb 11, 2013, at 7:48 AM, Joachim Jacob |VIB| wrote: Hello all, After a *reboot* of our Galaxy server, the environment variables are set correctly. However, after *restarting* the Galaxy process on a running server, by logging in as Galaxy and running the init script on CentOS as service galaxyd restart, the environment variables seems to be messed up Hi Joachim, It seems odd you'd be logging in as Galaxy to restart the process. Since the system is going to run the init script as root, you should do this as well when you restart it. Presumably whatever init script you are using is properly switching to the galaxy user to start the server process(es). --nate After this manual Galaxy restart, some tools are not found, apparently caused by a modification of the environment variable PATH. Can somebody provide me perhaps with insight on what is causing this, and how to avoid this? My environment variables are set in /etc/profile.d/galaxy_environment_setup (which is a symbolic link to - /home/galaxy/environment_setup ) Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Bug in Galaxy Reports tool
Hi all, Running ~/galaxy-dist/run_reports.sh brings up the Galaxy Reports interface on poort 9001. When clicking on 'Jobs per user' - picking a user - picking a month - and clicking on 'State' in the bar at the top, clears the screen. Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Preffered way of running a tool on multiple input files
Hi Hagai, Actually, using a workflow, you are able to select multiple input files, and let the workflow run separately on all input files. I would proceed by creating a data library for all your fastq files, which you can upload via FTP, or via a system directory. You can use a sample of your fastq files to create the steps in a history you want to perform, and extract a workflow out of it. Next, copy all fastq files from a data library in a new history, and run your workflow on the all input files. I hope this helps you further, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/12/2013 04:02 PM, Hagai Cohen wrote: Hi, I'm looking for a preferred way of running Bowtie (or any other tool) on multiple input files and run statistics on the Bowtie output afterwards. The input is a directory of files fastq1..fastq100 The bowtie output should be bed1...bed100 The statistics tool should run on bed1...bed100 and return xls1..xls100 Then I will write a tool which will get xls1..xls100 and merge them to one final output. I searched for a smiliar cases, and I couldn't figure anyone which had this problem before. Can't use the parallelism tag, because what will be the input for each tool? it should be a fastq file not a directory of fastq files. Neither I would like to run each fastq file in a different workflow - creating a mess. I thought only on two solutions: 1. Implement new datatypes: bed_dir fastq_dir and implements new tool wrappers which will get a folder instead of a file. 2. merge the input files before sending to bowtie, and use parallelism tag to make them be splitted merged again on each tool. Does anyone has any better suggestion? Thanks, Hagai ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Preffered way of running a tool on multiple input files
You cannot directly couple different workflows. But you could indeed copy all outputs of the different workflows into one history, and create a separate workflow with your tool to work on all those input files. Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/12/2013 04:31 PM, Hagai Cohen wrote: Thanks for your answer. I figured that there is an option to run a workflow on multiple files, but I can't merge the outputs afterwardsl. I would like the workflow to return one final output. But you gave me another idea. Can I somehow tell one workflow to run on other workflow output? If this can be done, I can run 100 different workflows with bowtie statistics, each working on one fastq file, than run another workflow which gets 100 xls inputs and merge them to one. On Tue, Feb 12, 2013 at 5:20 PM, Joachim Jacob |VIB| joachim.ja...@vib.be mailto:joachim.ja...@vib.be wrote: Hi Hagai, Actually, using a workflow, you are able to select multiple input files, and let the workflow run separately on all input files. I would proceed by creating a data library for all your fastq files, which you can upload via FTP, or via a system directory. You can use a sample of your fastq files to create the steps in a history you want to perform, and extract a workflow out of it. Next, copy all fastq files from a data library in a new history, and run your workflow on the all input files. I hope this helps you further, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 tel:%2B32%209%20244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/12/2013 04:02 PM, Hagai Cohen wrote: Hi, I'm looking for a preferred way of running Bowtie (or any other tool) on multiple input files and run statistics on the Bowtie output afterwards. The input is a directory of files fastq1..fastq100 The bowtie output should be bed1...bed100 The statistics tool should run on bed1...bed100 and return xls1..xls100 Then I will write a tool which will get xls1..xls100 and merge them to one final output. I searched for a smiliar cases, and I couldn't figure anyone which had this problem before. Can't use the parallelism tag, because what will be the input for each tool? it should be a fastq file not a directory of fastq files. Neither I would like to run each fastq file in a different workflow - creating a mess. I thought only on two solutions: 1. Implement new datatypes: bed_dir fastq_dir and implements new tool wrappers which will get a folder instead of a file. 2. merge the input files before sending to bowtie, and use parallelism tag to make them be splitted merged again on each tool. Does anyone has any better suggestion? Thanks, Hagai ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Bug Re: Data library not properly showing up
Hi all, For the problem below, it appears the the permissions were not set correctly. I have found that, in the Admin section, selecting multiple folders in a data library and trying to change the permissions of those, does not work. I select the folder, and in the dropdown box below select 'Edit permissions'. Next, the folders get unchecked and a message on top appears: You must select at least one dataset. I can reproduce it every time. Galaxy changeset: 8525:a4113cc1cb5e This is related to enhancement request on Trello: Show roles associated with data libraries https://trello.com/card/show-roles-associated-with-data-libraries/506338ce32ae458f6d15e4b3/229 cheers Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/08/2013 04:29 PM, Joachim Jacob |VIB| wrote: Hi all, The data library I have created is only showing up partially. I have made a data libray, via het Admin menu, with one folder in it. I changed permissions, because I wanted it only to be visible to me. That worked fine. Next I created plenty more folders in that data library. Now, the folders created afterwards are not visible in the data library, when accessed via 'Shared data'. I have played a lot with changing permissions of those libraries, but to no avail. Cheers, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Environment variables reset after manually restarting Galaxy
Hello all, After a *reboot* of our Galaxy server, the environment variables are set correctly. However, after *restarting* the Galaxy process on a running server, by logging in as Galaxy and running the init script on CentOS as service galaxyd restart, the environment variables seems to be messed up After this manual Galaxy restart, some tools are not found, apparently caused by a modification of the environment variable PATH. Can somebody provide me perhaps with insight on what is causing this, and how to avoid this? My environment variables are set in /etc/profile.d/galaxy_environment_setup (which is a symbolic link to - /home/galaxy/environment_setup ) Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: Desire to contribute
The highlight enhancement request on Trello: https://trello.com/c/vUtbTQ7l Since it was imported from Bitbucket, the duplicated card I mentioned refers to the wrong Trello card. Please consider! Thanks, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/06/2013 12:08 PM, Peter Cock wrote: On Wed, Feb 6, 2013 at 11:01 AM, Joachim Jacob |VIB| joachim.ja...@vib.be wrote: In risk of getting a discussion here: a long standing enhancement request is to highlight the current history item one is currently viewing. Situation sketch: I often hide the history items panel to study results (displayed in middle panel) into detail, and when I bring the history item panel back, I often have to search which item I was viewing - no clue at all. It really annoys me, but I don't know whether this can be fixed easily, and how deep you need to dig. Anyway, you will make at least one person happy :-) Cheers, Joachim That sounds like a good usability enhancement - and would likely need some knowledge of the mako template system used in Galaxy, and HTML/CSS for the visual styling too. You said it was a long standing enhancement request - is it filed on Trello? http://galaxyproject.org/trello (I couldn't find it myself). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Data library not properly showing up
Hi all, The data library I have created is only showing up partially. I have made a data libray, via het Admin menu, with one folder in it. I changed permissions, because I wanted it only to be visible to me. That worked fine. Next I created plenty more folders in that data library. Now, the folders created afterwards are not visible in the data library, when accessed via 'Shared data'. I have played a lot with changing permissions of those libraries, but to no avail. Cheers, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: Desire to contribute
In risk of getting a discussion here: a long standing enhancement request is to highlight the current history item one is currently viewing. Situation sketch: I often hide the history items panel to study results (displayed in middle panel) into detail, and when I bring the history item panel back, I often have to search which item I was viewing - no clue at all. It really annoys me, but I don't know whether this can be fixed easily, and how deep you need to dig. Anyway, you will make at least one person happy :-) Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/06/2013 02:16 AM, Matthew Paul wrote: Dear Galaxy Project community, I am working with a group of students at College of Charleston of South Carolina. Being interested in bioinformatics and software engineering, we chose to work on Galaxy for our open source class project. We are subscribed to the appropriate mailing list, have been accessing Trello and are becoming familiar with the Galaxy architecture. Our first assignment is to identify and fix a bug, but unfortunately the bugs reported seem to be going right over our heads.Where would be a good place to start, so that we may be able to contribute to your system (documentation, etc)? We are looking forward to your response. Thank you, Matt Paul -- Forwarded message -- From: *Matthew Paul* mrp...@g.cofc.edu mailto:mrp...@g.cofc.edu Date: Tue, Feb 5, 2013 at 8:10 PM Subject: Desire to contribute To: galaxy-dev@lists.bx.psu.edu mailto:galaxy-dev@lists.bx.psu.edu Dear Galaxy Project community, ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Fwd: Desire to contribute
I thought it was on Trello already. Anyway, for the moment I cannot acces Trello... When I do I will search and perhaps add it! Cheers Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 02/06/2013 12:08 PM, Peter Cock wrote: On Wed, Feb 6, 2013 at 11:01 AM, Joachim Jacob |VIB| joachim.ja...@vib.be wrote: In risk of getting a discussion here: a long standing enhancement request is to highlight the current history item one is currently viewing. Situation sketch: I often hide the history items panel to study results (displayed in middle panel) into detail, and when I bring the history item panel back, I often have to search which item I was viewing - no clue at all. It really annoys me, but I don't know whether this can be fixed easily, and how deep you need to dig. Anyway, you will make at least one person happy :-) Cheers, Joachim That sounds like a good usability enhancement - and would likely need some knowledge of the mako template system used in Galaxy, and HTML/CSS for the visual styling too. You said it was a long standing enhancement request - is it filed on Trello? http://galaxyproject.org/trello (I couldn't find it myself). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Reloading tool's configuration not working
Hi, In the 'Admin' menu, reloading a tool's xml file does not update the tool's interface. Thanks for any assistance, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Reloading tool's configuration not working
Unfortunately, no. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 01/30/2013 10:56 AM, Peter Cock wrote: On Wed, Jan 30, 2013 at 9:54 AM, Joachim Jacob |VIB| joachim.ja...@vib.be wrote: Hi, In the 'Admin' menu, reloading a tool's xml file does not update the tool's interface. Thanks for any assistance, Joachim I had a 'funny' recently where I had to do the xml reload twice, then it worked. Same for you? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Reloading tool's configuration not working
Thanks. Yes, I am running multiple webworkers. OK, nice to know what causes this. The toolshed option will be considered. Joachim. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 01/30/2013 11:45 AM, Björn Grüning wrote: Hi Joachim, are you running more than one web-worker? In that case i think it is not possible currently. If you use the toolshed your tools will be reloaded automatically with each update. Maybe that is an workaround for you? Ciao! Bjoern Hi, In the 'Admin' menu, reloading a tool's xml file does not update the tool's interface. Thanks for any assistance, Joachim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] History not updating automatically
Hi,
I have a similar - but not identical issue. The history panel is not
being updated: I have to restart Galaxy to see the updated panel.
It appears only to happen after an error relating to the history API of
Galaxy (it's a popup error, and I have closed it).
My apache config (thanks for any hints and tweaks to improve my setup).
[galaxy@galaxy conf.d]$ cat galaxy_web.conf
NameVirtualHost 157.193.230.103:80
VirtualHost 157.193.230.103:80
ServerName galaxy.bits.vib.be
Proxy balancer://galaxy
BalancerMember http://localhost:8080
BalancerMember http://localhost:8081
BalancerMember http://localhost:8082
BalancerMember http://localhost:8083
BalancerMember http://localhost:8084
BalancerMember http://localhost:8085
BalancerMember http://localhost:8086
BalancerMember http://localhost:8087
BalancerMember http://localhost:8088
BalancerMember http://localhost:8089
BalancerMember http://localhost:8090
BalancerMember http://localhost:8091
BalancerMember http://localhost:8092
/Proxy
RewriteEngine on
RewriteLog /tmp/apacheGalaxy.log
### LDAP authentication will be implemented later
# Location /
# AuthType Basic
# AuthBasicProvider ldap
# AuthLDAPURL ldap://smeagol.vib.be:389/DC=vib,DC=local?sAMAccountName
# AuthLDAPBindDN vib\administrator
# AuthLDAPBindPassword tofillin
# AuthzLDAPAuthoritative off
# Require valid-user
# # Set the REMOTE_USER header to the contents of the LDAP query
response's uid attribute
# RequestHeader set REMOTE_USER %{AUTHENTICATE_sAMAccountName}
# /Location
RewriteRule ^/static/style/(.*)
/home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L]
RewriteRule ^/static/scripts/(.*)
/home/galaxy/galaxy-dist/static/scripts/packed/$1 [L]
RewriteRule ^/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L]
RewriteRule ^/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L]
RewriteRule ^/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L]
RewriteRule ^(.*) balancer://galaxy$1 [P]
/VirtualHost
Cheers,
Joachim
Joachim Jacob
Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS)
http://www.bits.vib.be
@bitsatvib
On 01/28/2013 09:02 PM, Langhorst, Brad wrote:
Nate:
I used the config from the wiki directly…
Location /
…
RequestHeader set REMOTE_USER %{AUTHENTICATE_sAMAccountName}e
...
/Location
but I also had a section like this below it...
# Location /api
# Satisfy Any
# Allow from all
# /Location
I thought that the /api location would inherit the auth and headers stuff of
it's container, but that's not true.
Commenting the api location fixes the problem form e.
Brad
On Jan 28, 2013, at 2:35 PM, Nate Coraor n...@bx.psu.edu
wrote:
Hi Brad and Sajoscha,
Is there any chance that your proxy configurations are not passing the username
in the REMOTE_USER header when the request is to an /api path? Could you
provide the relevant portions of your proxy server configs?
Thanks,
--nate
On Jan 28, 2013, at 7:12 AM, Langhorst, Brad wrote:
Hi Sajoscha:
I have exactly the same problem… it started about a month ago.
Also with external ldap auth.
I have not yet investigated in detail, since it's not crippling - just annoying.
Brad
On Jan 28, 2013, at 6:38 AM, Sajoscha Sauer sa...@embl.de
wrote:
Hi list,
I have a weird problem after updating to the latest version.
After doing the update as usual, the history panel is not updating
automatically anymore. And when I press refresh, I get an javascript popup
saying:
Error getting history updates from the server.
Forbidden
Also, in the log I see the following:
galaxy.web.framework WARNING 2013-01-28 12:30:01,673 User logged in as '(null)'
externally, but has a cookie as 'sa...@embl.de' invalidating session
We are using LDAP to connect to Galaxy, but I don't know if this has something
to do with the problem. As I said, it was working perfectly fine before the
update.
Does someone know the problem or has any ideas?
Thanks for your help,
Sajoscha
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
--
Brad Langhorst
langho...@neb.com
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
--
Brad Langhorst
langho...@neb.com
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail
[galaxy-dev] Jobs status not being updated
Hi all, After updating to the latest Galaxy dist-release (and moving our server to a new location - but this can not be the reason I think), the status of the jobs are not being updated. The box in the history stays grey. After restarting Galaxy or the server, the correct status of the job is displayed. Any advice on how to proceed is appreciated. Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Migration of tools
Hi, Has the devteam a schedule of the order of the tools they plan to migrate to the tool shed? Thanks, Joachim. -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
