Re: [galaxy-dev] sort list of ftp uploaded files
Dear John, dear Curtis, and all all that is required to fix my current issue is that the files show up in the 'Upload File' tool form in alphabetical order. I assume that at some point an 'ls' command is done on the users' ftp home folder. The result of that could possibly be stored in a hash of some sort, which could explain why the order is randomised when the files come back out to go on the form. But I might be wrong. Of course a better but more involved solution would be if the 'Files uploaded via FTP' table was sortable, e.g. by clicking on the table headers. Any idea where I might find the code that populates this table? Thanks a lot for your help. Cheers Ulf On 17/12/13 22:31, Curtis Hendrickson (Campus) wrote: John, I can't speak for Ulf, but a more general solution would be to allow sorting by a standard unix SORT set keys. That would allow things like - sort by the sample ID after the 3rd dash, then by the read direction. -t - -k1,1n -k2,2r # http://unixhelp.ed.ac.uk/CGI/man-cgi?sort Regards, Curtis PS - granted simple filename sort is a good place to start and a lot better than nothing. Date sort is often useful, too. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John Chilton Sent: Tuesday, December 17, 2013 4:03 PM To: Ulf Schaefer Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] sort list of ftp uploaded files If Galaxy just sorted these files alphabetically before display/import would that fix your problem? Or do your users need to be able to modify the order? -John On Mon, Dec 16, 2013 at 9:19 AM, Ulf Schaefer ulf.schae...@phe.gov.uk wrote: Dear all Is there a way to sort the list of files that a user has uploaded via FTP? Currently they appear on the form of the tool 'Upload File' in random order it seems. There is not much documentation on the param type=ftpfile / tag unfortunately. If the user just selects all files they end up in his/her history in the same random order, which causes a slight inconvenience when using these files as paired multiple input for a workflow. The obvious work-around is to put the files into the history one pair at a time, but this becomes a bit onerous for larger number of files. Thanks a lot for your help. Cheers Ulf ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked
Hi Peter, Thanks for reporting this. I am now facing the same error, but I don't get it For example: normally it appears when I try to change something in the DB (e.g. adding a new history), but then suddenly I managed to create one new history itemand now it is occurring again. It seems to be a transient locked state, so every now and then the lock is released somehow. Anyway, are there any ideas on what could be a real solution to this problem? This is the second Galaxy server I upgrade which now is stuck after the upgrade...any help is appreciated! Thanks and regards, Pieter. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Peter Cock Sent: donderdag 31 oktober 2013 11:54 To: Galaxy Dev Subject: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked Hello all, While attempting a fresh Galaxy install from both galaxy-central and galaxy-dist, I ran into a problem initialising the default SQLite database (database/universe.sqlite), $ hg clone https://bitbucket.org/galaxy/galaxy-dist ... $ cd galaxy-dist $ ./run.sh ... galaxy.model.migrate.check DEBUG 2013-10-31 10:28:26,143 pysqlite=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): ... OperationalError: (OperationalError) database is locked u'PRAGMA table_info(dataset)' () After some puzzlement, I realised this was down to the file system - I was trying this under my home directory mounted via a distributed file system (gluster I think). Repeating the experiment under /tmp on a local hard disk worked :) (I'm posting this message for future reference; hopefully Google and/or mailing list searches will help anyone else facing this error) Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Skip automated testing of tools in this revision setting?
Thanks Greg for the heads-up! These are great news, I can surely wait until mid-January (the wait will be mostly vacation time :). Just let us know when we can start checking again the test results on the main Tool Shed. Nicola Il 2013-12-17 21:15 Greg Von Kuster ha scritto: Hi Nicola, There's some bad news and good new regarding this issue. The bad news is that the old version of the tool shed's install and test framework required complete reengineering, and this resulted in it not being functional in the main tool shed's environment which runs the Galaxy stable branch. I've tried several approaches to committing and grafting changes to the stable branch in an attempt to piece together a functional environment for the main tool shed until the next Galaxy release, but the reengineering effort was so extensive that it made this extremely difficult. So the main tool shed cannot be tested until the next-stable branch is created in preparation for the next Galaxy release. The next-stable branch is currently scheduled for mid January, 2014, with the Galaxy release currently scheduled for the first week of February, 2014. The good news is that the reengineerig effort has resulted in a much more stable and functional install and test framework that will provide many more benefits than its previous incarnation. I am now working on enhancing the framework to include installing and testing repositories of type tool_dependency_definition. This enhancement will be included in the next-stable branch when it is created in mid-January. Sorry for the inconvenience this may cause, but after mid-January things should be in good shape on the main tool shed. Greg Von Kuster On Dec 12, 2013, at 5:00 PM, Nicola Soranzo sora...@crs4.it wrote: Hi Greg, thanks for looking at this! Unfortunately still no test results displayed for http://toolshed.g2.bx.psu.edu/view/crs4/blat Best, Nicola Il giorno mar, 10/12/2013 alle 10.43 -0500, Greg Von Kuster ha scritto: Hello Nicola, The main tool shed is running a bit behind the test tool shed since it is on the stable branch. I've commited a fix for the main tool shed in 11664:3bf805dfa14e that should correct the problem resulting in no test results being displayed. Results should begin being displayed with the test run tomorrow morning. I'll make sure to inspect the test runs for the main tool shed more closely from here on (I've been focusing more on the test tool shed) to make sure the different issues that arise on the stable branch are handled more quickly. Thanks for letting me know about this. Greg Von Kuster On Dec 10, 2013, at 10:09 AM, Nicola Soranzo sora...@crs4.it wrote: Hi Greg! I don't know if it's the same problem, but I've not been able to see test results for this repository for a while: http://toolshed.g2.bx.psu.edu/view/crs4/blat Best, Nicola Il giorno mar, 10/12/2013 alle 09.46 -0500, Greg Von Kuster ha scritto: There are several flags for each repository revision that determine if and how the revision is tested. If tested, flags also detemine which category the latest revision falls into when clicking on the links in the Tool Shed's menu. One of these flags is tools_functionally_correct, which was incorrectly set on http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80 due to a bug introduced during the reengineering of the Topll Shed's install and test framework that I recently did. This bug was corrected in the following change set. https://bitbucket.org/galaxy/galaxy-central/commits/ca25579a90e97157e32b527e7471cde90e9beed0 I've corrected the setting for the tools_functionally_correct flag for your repository so it should no longer be displayed in the Latest revision: failing tool tests category. Thanks for letting us know about this, and please let us know if you discover other reposiory revisions that require the same correction. Greg Von Kuster On Dec 9, 2013, at 5:06 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Dave Greg, Could you double check how the Skip automated testing of tools in this revision setting is being used on the Test Tool Shed? e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80 Test runs Automated test environment Not tested Licensing issues restrict automated tool dependency installation. 2013-11-29 17:19:34 Automated test environment Not tested Licensing issues restrict automated tool dependency installation. (Ticked) Skip automated testing of tools in this revision That all looks fine, and it is listed under Latest revision: skip tool tests. However, this tool is also listed under Latest revision: failing tool tests. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To
Re: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked
Hi Pieter, Using SQLite is NOT recommended for a production Galaxy, it is only really used for development and testing (having the whole database as one file on disk without worrying about creating a proper database, user accounts etc is nice). Peter On Wednesday, December 18, 2013, Lukasse, Pieter wrote: Hi Peter, Thanks for reporting this. I am now facing the same error, but I don't get it For example: normally it appears when I try to change something in the DB (e.g. adding a new history), but then suddenly I managed to create one new history itemand now it is occurring again. It seems to be a transient locked state, so every now and then the lock is released somehow. Anyway, are there any ideas on what could be a real solution to this problem? This is the second Galaxy server I upgrade which now is stuck after the upgrade...any help is appreciated! Thanks and regards, Pieter. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu javascript:; [mailto: galaxy-dev-boun...@lists.bx.psu.edu javascript:;] On Behalf Of Peter Cock Sent: donderdag 31 oktober 2013 11:54 To: Galaxy Dev Subject: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked Hello all, While attempting a fresh Galaxy install from both galaxy-central and galaxy-dist, I ran into a problem initialising the default SQLite database (database/universe.sqlite), $ hg clone https://bitbucket.org/galaxy/galaxy-dist ... $ cd galaxy-dist $ ./run.sh ... galaxy.model.migrate.check DEBUG 2013-10-31 10:28:26,143 pysqlite=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): ... OperationalError: (OperationalError) database is locked u'PRAGMA table_info(dataset)' () After some puzzlement, I realised this was down to the file system - I was trying this under my home directory mounted via a distributed file system (gluster I think). Repeating the experiment under /tmp on a local hard disk worked :) (I'm posting this message for future reference; hopefully Google and/or mailing list searches will help anyone else facing this error) Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked
Hi Peter, I know that ;) Still I am curious about the possible solution. Any ideas? Regards, Pieter. From: Peter Cock [mailto:p.j.a.c...@googlemail.com] Sent: woensdag 18 december 2013 13:31 To: Lukasse, Pieter Cc: Galaxy Dev Subject: Re: New Galaxy won't start, SQLite (OperationalError) database is locked Hi Pieter, Using SQLite is NOT recommended for a production Galaxy, it is only really used for development and testing (having the whole database as one file on disk without worrying about creating a proper database, user accounts etc is nice). Peter On Wednesday, December 18, 2013, Lukasse, Pieter wrote: Hi Peter, Thanks for reporting this. I am now facing the same error, but I don't get it For example: normally it appears when I try to change something in the DB (e.g. adding a new history), but then suddenly I managed to create one new history itemand now it is occurring again. It seems to be a transient locked state, so every now and then the lock is released somehow. Anyway, are there any ideas on what could be a real solution to this problem? This is the second Galaxy server I upgrade which now is stuck after the upgrade...any help is appreciated! Thanks and regards, Pieter. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edujavascript:; [mailto:galaxy-dev-boun...@lists.bx.psu.edujavascript:;] On Behalf Of Peter Cock Sent: donderdag 31 oktober 2013 11:54 To: Galaxy Dev Subject: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked Hello all, While attempting a fresh Galaxy install from both galaxy-central and galaxy-dist, I ran into a problem initialising the default SQLite database (database/universe.sqlite), $ hg clone https://bitbucket.org/galaxy/galaxy-dist ... $ cd galaxy-dist $ ./run.sh ... galaxy.model.migrate.check DEBUG 2013-10-31 10:28:26,143 pysqlite=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): ... OperationalError: (OperationalError) database is locked u'PRAGMA table_info(dataset)' () After some puzzlement, I realised this was down to the file system - I was trying this under my home directory mounted via a distributed file system (gluster I think). Repeating the experiment under /tmp on a local hard disk worked :) (I'm posting this message for future reference; hopefully Google and/or mailing list searches will help anyone else facing this error) Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] sort list of ftp uploaded files
Couple things... I just pushed a commit to galaxy-central to sort those files by default - that is a clear improvement: https://bitbucket.org/galaxy/galaxy-central/commits/ce186eb5fefcb7ff332c73bd0869b9e93d0b Clearly not enough though - it would be nice to have for more advanced sorting options though so I have created a Trello card - https://trello.com/c/0hgUrW4H If you really wanted to dig into this there are many files that may be relevant - templates/embed_base.mako lib/galaxy/web/form_builder.py lib/galaxy/tools/parameters/basic.py lib/galaxy/web/framework/helpers/grids.py Hopefully this helps. Also just as a heads up, coming from a different direction me and Martin are actively working building abstractions for pairs of data and that will have to include a nice UI for building these pairs - our plan is to start with library imports but we will try to architect the creation widget to be able to target history items as well. This would largely negate the need to import them in any particular order. This will likely not be in the next release, but hopefully the following one (I can build pairs via the API now, but I want to be able to do useful stuff with them before committing :) ). -John On Wed, Dec 18, 2013 at 2:37 AM, Ulf Schaefer ulf.schae...@phe.gov.uk wrote: Dear John, dear Curtis, and all all that is required to fix my current issue is that the files show up in the 'Upload File' tool form in alphabetical order. I assume that at some point an 'ls' command is done on the users' ftp home folder. The result of that could possibly be stored in a hash of some sort, which could explain why the order is randomised when the files come back out to go on the form. But I might be wrong. Of course a better but more involved solution would be if the 'Files uploaded via FTP' table was sortable, e.g. by clicking on the table headers. Any idea where I might find the code that populates this table? Thanks a lot for your help. Cheers Ulf On 17/12/13 22:31, Curtis Hendrickson (Campus) wrote: John, I can't speak for Ulf, but a more general solution would be to allow sorting by a standard unix SORT set keys. That would allow things like - sort by the sample ID after the 3rd dash, then by the read direction. -t - -k1,1n -k2,2r # http://unixhelp.ed.ac.uk/CGI/man-cgi?sort Regards, Curtis PS - granted simple filename sort is a good place to start and a lot better than nothing. Date sort is often useful, too. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John Chilton Sent: Tuesday, December 17, 2013 4:03 PM To: Ulf Schaefer Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] sort list of ftp uploaded files If Galaxy just sorted these files alphabetically before display/import would that fix your problem? Or do your users need to be able to modify the order? -John On Mon, Dec 16, 2013 at 9:19 AM, Ulf Schaefer ulf.schae...@phe.gov.uk wrote: Dear all Is there a way to sort the list of files that a user has uploaded via FTP? Currently they appear on the form of the tool 'Upload File' in random order it seems. There is not much documentation on the param type=ftpfile / tag unfortunately. If the user just selects all files they end up in his/her history in the same random order, which causes a slight inconvenience when using these files as paired multiple input for a workflow. The obvious work-around is to put the files into the history one pair at a time, but this becomes a bit onerous for larger number of files. Thanks a lot for your help. Cheers Ulf ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other
Re: [galaxy-dev] problems with exporting or importing data from file
Hi Jeremy, Anna, I'm having a similar problem here as well. I would like to get a history from one Galaxy server to another and I followed the export and import steps. I even did a wget at the destination Galaxy server to check whether it could download the tar.gz file and this worked. When using the web UI I also get this message on my log showing it is actually doing something when I trigger the import: 10.85.13.89 - - [18/Dec/2013:15:10:48 +0200] POST /history/import_archive HTTP/1.1 200 - http://dev1.ab.wurnet.nl:8088/history/import_archive; Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36 galaxy.jobs DEBUG 2013-12-18 15:10:48,391 (3201) Working directory for job is: /home/lukas007/galaxy-dist/database/job_working_directory/003/3201 galaxy.jobs.handler DEBUG 2013-12-18 15:10:48,404 (3201) Dispatching to local runner galaxy.jobs DEBUG 2013-12-18 15:10:48,478 (3201) Persisting job destination (destination id: local:///) galaxy.jobs.handler INFO 2013-12-18 15:10:48,514 (3201) Job dispatched galaxy.jobs.runners.local DEBUG 2013-12-18 15:10:48,742 (3201) executing: export GALAXY_SLOTS=1; python /home/lukas007/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.wur.nl/galaxy_production/history/export_archive?id=b1f249e032330dc4 /home/lukas007/galaxy-dist/database/tmp/tmpPGL6le --url galaxy.jobs DEBUG 2013-12-18 15:10:48,786 (3201) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-12-18 15:10:48,875 execution finished: export GALAXY_SLOTS=1; python /home/lukas007/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.wur.nl/galaxy_production/history/export_archive?id=b1f249e032330dc4 /home/lukas007/galaxy-dist/database/tmp/tmpPGL6le --url galaxy.jobs DEBUG 2013-12-18 15:10:49,040 job 3201 ended galaxy.datatypes.metadata DEBUG 2013-12-18 15:10:49,040 Cleaning up external metadata files However, nothing is visible in my histories list. What could be wrong? I also don't see any error messages in the log above. Thanks and regards, Pieter. From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Jeremy Goecks Sent: dinsdag 11 juni 2013 19:06 To: Edlund, Anna Cc: galaxy-...@bx.psu.edu; Nicola Segata Subject: Re: [galaxy-dev] problems with exporting or importing data Anna, Try following these steps: (1) Make your history accessible (Share or Publish -- Make History Accessible by Link). (2) Export your history again; make sure to wait until you can use the URL to download a .gz file of your history. (3) Try importing it via URL to the Huttenhower Lab Galaxy. Let us know if you have any problems. Best, J. On Jun 10, 2013, at 11:45 AM, Edlund, Anna wrote: Hi. I have uploaded fasta files on the main galaxy server through my FileZilla program. In want to transfer them to the Galaxy at the Huttenhower Lab (see http://huttenhower.org/galaxy/root) page and to do that go to history list when I am logged in at both locations and I select 'Import a History from an archive'. I paste the url address from the main galaxy server (where my archive is located) and select submit. Then I waited for 2 days and nothing was transferred? I am clearly doing something wrong and would really like you input asap. Thank you very much!! Best regards, Anna Edlund My user name is aedl...@jcvi.orgmailto:aedl...@jcvi.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug: Toolbox filters not applied in workflows
Ahh... missed that. Sorry (and thanks for putting together the page :) ). It seems like we are on the same page - I have created a Trello card for 'stronger' toolbox filters. https://trello.com/c/Sg8D2PBj It occurs to me there may another way to accomplish admin tool functionality - and in an even more locked down fashion. I think you may be able to setup another Galaxy process with a similar config but a different tool_config_file setting and it should act as its own handler+web process as well. Then you could configure your proxy so admin requests come through this process - or maybe it could have its own URL, etc Maybe this is crazy - I am sure Nate will respond is this is non-sense :). -John On Tue, Dec 17, 2013 at 6:26 PM, Eric Rasche rasche.e...@yandex.ru wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 John, I've been using the dynamic jobs and they've been working beautifully! (For context, I wrote the initial revision of the Access Control page based off of my readings and experiences setting it up in galaxy). I, for one, would be in favour of stronger filters in addition to the current versions of filtering, mostly due to my/IT's paranoid nature :) The current filters seem to accomplish exactly what they were designed to, and I see no reason to replace them. On 12/17/2013 06:15 PM, John Chilton wrote: Agreed with Dannon, it is a bug that they are showing up in the workflow editor but in general toolbox filters are not meant as a security mechanism. You can block access to running specific tools (filtered or not) using dynamic job destinations. (Under dynamic job destinations documentation ( https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping) there is an example of enforcing dev only tools for a specific set of e-mails as well as an example of how to get a list of admin users in a dynamic job destination. These could be combined to enforce admin only tools. I think there has been some other posts on this topic... ) It is true that one can still see the options of such tools, this is something we will have to think about... Perhaps a 'stronger' filter, though I think this variant of filtering should still be an option, there are many use cases for allowing users to run tools they cannot see. -John On Tue, Dec 17, 2013 at 5:39 PM, Dannon Baker dannon.ba...@gmail.comwrote: This sounds like a bug; the primary toolbox and workflow editor toolbox should reflect the same set of tools (exception being workflow-specific control steps, etc). I've created a trello card to track this issue here: https://trello.com/c/3TxFHkYR That said, do note the warning on the dynamic toolbox filters page: [image: !] Filters will only hide Tools from the User Interface, they are still available and can be made visible by means of HTML manipulation. That said these feature is not a security feature, it is intended to separate multiple groups of Tools and simplify the ToolBoxhttps://wiki.galaxyproject.org/ToolBox . On Tue, Dec 17, 2013 at 6:27 PM, Eric Rasche rasche.e...@yandex.ruwrote: I'm running the stable copy of galaxy and noticed that some custom, administrative tools (and otherwise tools which should be restricted in access due to licensing/etc.) were showing up in normal user's toolboxes inside the workflow editor. I feel that this is a bug, as the tool filters should be applied globally and not just in terms of what tools users are restricted from seeing in the normal toolbox. For me, this presents a problem as I strongly believe any administrative tools that exist should leak as little information as possible--not their entire set of options and associated documentation. Additionally, that sort of information leakage isn't acceptable by my organisation's policies. Do I have my instance misconfigured or is this an actual bug? I have my galaxy configured according to https://wiki.galaxyproject.org/Admin/Config/Access%20Control $ hg head changeset: 11242:9d4cbf2a1c13 branch: stable tag: tip user:Nate Coraor n...@bx.psu.edu n...@bx.psu.edu date:Fri Dec 06 16:28:31 2013 -0500 summary: Add missing destination long arg to cli runner's Torque plugin and fix an incorrectly used PBS option in the sample job conf. changeset: 11216:c458a0fe1ba8 parent: 11213:6d633418ecfa parent: 11215:f79149dd3d35 user:Nate Coraor n...@bx.psu.edu n...@bx.psu.edu date:Mon Nov 04 14:56:57 2013 -0500 summary: Merge security fix for filtering tools from stable/next-stable. $ hg summary parent: 11242:9d4cbf2a1c13 tip Add missing destination long arg to cli runner's Torque plugin and fix an incorrectly used PBS option in the sample job conf. branch: stable commit: 4 modified, 34 unknown update: (current) Thank you, Eric Rasche Programmer II Center for Phage Technology Texas AM University
Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster
File system performance varies wildly between storage architectures. There are storage server setups that can easily scale to orders of magnitude beyond the compute that backs usegalaxy.org - suffice to say we are currently bound by the number of cores we have available and not by IO/network performance. (Nate may elaborate on the specifics of the public servers setup, but I am not sure it is useful to you unless you have hundreds or thousands (or millions) of dollars to spend on new storage and network hardware :) ). Also my idea was not a second Galaxy instance - sorry I did not make that clearer. It was to restrict your current Galaxy instance to a smaller portion of your cluster. If your cluster is completely dedicated to Galaxy however this idea doesn't make sense, but if this is a shared condor cluster used for other things (besides Galaxy) it could make sense. Sorry have not been more helpful. -John On Tue, Dec 17, 2013 at 6:12 PM, Ben Gift corn8b...@gmail.com wrote: How do you have it set up on the main public galaxy install? I imagine that people run enough big jobs that there there is enormous use of your shared file system. How did you scale that to so many nodes without bogging down the file system with large dataset transfers? It seems that for now the solution of having a second Galaxy instance will work well, thank you very much John :) . But I'm still interested in a more permanent scaled solution. After reading up more on our shared file system it still seems like heavy traffic is bad, so could my initial idea still be good? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to use Galaxy API?
Hello, by API controller I meant the Python module that takes control over the endpoints of API thus providing logic. You don't need to 'use them' - you can create API requests by many different means by specifying URL (sometimes +params+payload). Look at the extension link I have sent you, that will help you call the API from the browser. M. On Wed, Dec 18, 2013 at 3:15 AM, xlwang xlwang_0...@163.com wrote: Hi, thank you very much for such a quick relpy. Now, I can connect to galaxy via API. In addition, please tell me what is API controller?How to use those modules and where are they in galaxy? At 2013-12-18 12:13:58,Martin Čech mar...@bx.psu.edu wrote: Hi, the best start for using the API, that you found already, is the documentation that, with a nice introduction, is nicely available here: https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html Regarding libraries your main controllers of interest are: *libraries*, *library_contents*, *folders, **folder_contents* - *please beware that folders API is considered work-in-progress and will probably be deprecated in future releases.* If you want to test how the API behaves you can use something like Postman extension ( https://chrome.google.com/webstore/detail/postman-rest-client/fdmmgilgnpjigdojojpjoooidkmcomcm?hl=en) for chrome or other browser that allows you to nicely simulate API calls. As of your error make sure that your local Galaxy, that you are trying to connect to via API, runs on the 4096 port that you specified for the script (by default local Galaxy runs at port 8080 - see universe_wsgi.ini for config changes). Thank you for using Galaxy and let us know how it goes! Martin, Galaxy Team On Tue, Dec 17, 2013 at 9:26 PM, xlwang xlwang_0...@163.com wrote: Hello, I want to know how to use Galaxy API(the path is /galaxy-dist/scripts/api). I follow the README but get errors. For instance, I have generated my_key and use this script: ./display.py my_key http://localhost:4096/api/libraries I get an error: urlopen error [Errno 111] Connection refused I do not know how it happened. Please tell me how to fix it. Thank you! In addition, my main purpose is to manage the data library in another system. Should I use the api such as library_create_library.py, library_upload_from_import_dir.py and so on? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] sort list of ftp uploaded files
Dear John Thanks very much. This is the perfect (minimal) solution that I need right now. I am looking forward to your work on pairs of data. It sounds like it will be exceptionally useful for us. Thanks again and best wishes Ulf On 18/12/13 14:29, John Chilton wrote: Couple things... I just pushed a commit to galaxy-central to sort those files by default - that is a clear improvement: https://bitbucket.org/galaxy/galaxy-central/commits/ce186eb5fefcb7ff332c73bd0869b9e93d0b Clearly not enough though - it would be nice to have for more advanced sorting options though so I have created a Trello card - https://trello.com/c/0hgUrW4H If you really wanted to dig into this there are many files that may be relevant - templates/embed_base.mako lib/galaxy/web/form_builder.py lib/galaxy/tools/parameters/basic.py lib/galaxy/web/framework/helpers/grids.py Hopefully this helps. Also just as a heads up, coming from a different direction me and Martin are actively working building abstractions for pairs of data and that will have to include a nice UI for building these pairs - our plan is to start with library imports but we will try to architect the creation widget to be able to target history items as well. This would largely negate the need to import them in any particular order. This will likely not be in the next release, but hopefully the following one (I can build pairs via the API now, but I want to be able to do useful stuff with them before committing :) ). -John On Wed, Dec 18, 2013 at 2:37 AM, Ulf Schaefer ulf.schae...@phe.gov.uk wrote: Dear John, dear Curtis, and all all that is required to fix my current issue is that the files show up in the 'Upload File' tool form in alphabetical order. I assume that at some point an 'ls' command is done on the users' ftp home folder. The result of that could possibly be stored in a hash of some sort, which could explain why the order is randomised when the files come back out to go on the form. But I might be wrong. Of course a better but more involved solution would be if the 'Files uploaded via FTP' table was sortable, e.g. by clicking on the table headers. Any idea where I might find the code that populates this table? Thanks a lot for your help. Cheers Ulf On 17/12/13 22:31, Curtis Hendrickson (Campus) wrote: John, I can't speak for Ulf, but a more general solution would be to allow sorting by a standard unix SORT set keys. That would allow things like - sort by the sample ID after the 3rd dash, then by the read direction. -t - -k1,1n -k2,2r # http://unixhelp.ed.ac.uk/CGI/man-cgi?sort Regards, Curtis PS - granted simple filename sort is a good place to start and a lot better than nothing. Date sort is often useful, too. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John Chilton Sent: Tuesday, December 17, 2013 4:03 PM To: Ulf Schaefer Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] sort list of ftp uploaded files If Galaxy just sorted these files alphabetically before display/import would that fix your problem? Or do your users need to be able to modify the order? -John On Mon, Dec 16, 2013 at 9:19 AM, Ulf Schaefer ulf.schae...@phe.gov.uk wrote: Dear all Is there a way to sort the list of files that a user has uploaded via FTP? Currently they appear on the form of the tool 'Upload File' in random order it seems. There is not much documentation on the param type=ftpfile / tag unfortunately. If the user just selects all files they end up in his/her history in the same random order, which causes a slight inconvenience when using these files as paired multiple input for a workflow. The obvious work-around is to put the files into the history one pair at a time, but this becomes a bit onerous for larger number of files. Thanks a lot for your help. Cheers Ulf ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Re: [galaxy-dev] New Galaxy won't start, SQLite (OperationalError) database is locked
On Wed, Dec 18, 2013 at 12:51 PM, Lukasse, Pieter pieter.luka...@wur.nl wrote: Hi Peter, I know that ;) Still I am curious about the possible solution. Any ideas? Regards, Pieter. Could you clarify the nature of the problem - is it failing to even start as in the situation I reported at the start of this thread, or an intermittent problem? Note that while generally stable I have had problems with my development machine with heavy SQLite DB access, e.g. a large multi-part job being split creating many child jobs in quick succession. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Error running CCAT test
Hi Nicola, This doesn't appear to be a known functional test failure (if I am reading the logs right), but I am also not the best resource to troubleshoot this particular type of problem. I am going to move this to the dev list for more feedback. If you want to add more details about your system (local? tracking galaxy-dist or central?) that might help. Best, Jen Galaxy team On 12/18/13 2:42 AM, Nicolas Lapalu wrote: Hi, I tried to run CCAT functionnal test, but I get an exception (same problem via web form) Data are well associated with the hg18 build, which is available in test-data/chrom/hg18.len Error running CCAT. Traceback (most recent call last): File /home/galaxy-dev/tools/peak_calling/ccat_wrapper.py, line 41, in module if __name__ == __main__: main() File /home/galaxy-dev/tools/peak_calling/ccat_wrapper.py, line 38, in main return stop_err( tmp_dir, e ) File /home/galaxy-dev/tools/peak_calling/ccat_wrapper.py, line 13, in stop_err raise exception AssertionError: The required chromosome length file does not exist. Any Idea ? Thanks, Nicolas -- Jennifer Hillman-Jackson http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] GCC2014 Training Day: Topic Nomination is now open
Hello all, Just a reminder that GCC2014 Training Day Topic Nominations close this Friday, December 20. We have 9 excellent nominations thus farhttp://wiki.galaxyproject.org/Events/GCC2014/TrainingDay, but we need more. If you have something you want to see covered, *then please nominate a topic http://bit.ly/gcc2014tdnom before the end of this Friday.* Thanks, Dave C, on behalf of the GCC2014 Organizing Committee On Mon, Dec 2, 2013 at 10:32 AM, Dave Clements cleme...@galaxyproject.orgwrote: Hello all, Training Day topics are nominated by you http://bit.ly/gcc2014tdnom, the Galaxy Community. Please take a minute to nominate a topic. Any topic of interest to the Galaxy Community can be nominated and you are encouraged to nominate more than one topic. If you are looking for ideas, see - what was offered at GCC2013http://wiki.galaxyproject.org/Events/GCC2013/TrainingDay , - what topics were nominated in 2013 http://bit.ly/1i2j1gN, and - the Events http://wiki.galaxyproject.org/Events and the Events Archive http://wiki.galaxyproject.org/Events/Archive pages. Nominated topics will be published on the Training Dayhttp://wiki.galaxyproject.org/Events/GCC2014/TrainingDay page as they come in. *Nominations close December 20*. Topics will be compiled into a uniform list by the GCC2014 Organizing Committee, and topics will be posted and voted on by the Galaxy Community, January 6-17. Nominate a topic now! http://bit.ly/gcc2014tdnom Topics will then be selected and scheduled based on topic interest, and the organizers' ability to confirm instructors for each session. Some very popular sessions may be scheduled more than once. The final schedule will be posted before registration opens. See you in Baltimore! GCC2014 Organizing Committeehttp://wiki.galaxyproject.org/Events/GCC2014/Organizers --- *About GCC2014:* The 2014 Galaxy Community Conference (GCC2014)http://galaxyproject.org/gcc2014will be held at the Homewood Campushttp://webapps.jhu.edu/jhuniverse/information_about_hopkins/campuses/homewood_campus/ of Johns Hopkins University http://jhu.edu/, inBaltimore, Marylandhttp://visitors.baltimorecity.gov/, United States, from June 30 through July 2, 2014. Galaxy Community Conferences are an opportunity to participate in presentations, discussions, poster sessions, keynotes, lightning talks and bird-of-a-feather gatherings, all about high-throughput biology and the tools that support it. -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Error with tool migration
Hello - I pulled down the latest changes today (it was long overdue), and am running into the following error when I run the tool migration: sh ./scripts/migrate_tools/0008_tools.sh install_dependencies No handlers could be found for logger galaxy.tools.data Repositories will be installed into configured tool_path location ../shed_tools [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdWuQjBx Skipping installation of tool dependency samtools version 0.1.18 since it is installed in /users/galaxy/galaxy_deps/bin/samtools/0.1.18/devteam/package_samtools_0_1_18/171cd8bc208d Traceback (most recent call last): File ./scripts/migrate_tools/migrate_tools.py, line 21, in module app = MigrateToolsApplication( sys.argv[ 1 ] ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py, line 83, in __init__ install_dependencies=install_dependencies ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 121, in __init__ is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 509, in install_repository is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 351, in handle_repository_contents guid = self.get_guid( repository_clone_url, relative_install_dir, tool_config ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 259, in get_guid full_path = str( os.path.abspath( os.path.join( root, name ) ) ) UnboundLocalError: local variable 'name' referenced before assignment Ideas? Thanks much! Kerry ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error with tool migration
Hello Kerry, I've committed a fix for this in 11800:a53f2f700699 to the galaxy-central branch. Is that the branch you're pulling? Thanks for reporting this. Greg Von Kuster On Dec 18, 2013, at 5:48 PM, Kerry Deutsch kerry.deut...@systemsbiology.org wrote: Hello - I pulled down the latest changes today (it was long overdue), and am running into the following error when I run the tool migration: sh ./scripts/migrate_tools/0008_tools.sh install_dependencies No handlers could be found for logger galaxy.tools.data Repositories will be installed into configured tool_path location ../shed_tools [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdWuQjBx Skipping installation of tool dependency samtools version 0.1.18 since it is installed in /users/galaxy/galaxy_deps/bin/samtools/0.1.18/devteam/package_samtools_0_1_18/171cd8bc208d Traceback (most recent call last): File ./scripts/migrate_tools/migrate_tools.py, line 21, in module app = MigrateToolsApplication( sys.argv[ 1 ] ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py, line 83, in __init__ install_dependencies=install_dependencies ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 121, in __init__ is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 509, in install_repository is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 351, in handle_repository_contents guid = self.get_guid( repository_clone_url, relative_install_dir, tool_config ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 259, in get_guid full_path = str( os.path.abspath( os.path.join( root, name ) ) ) UnboundLocalError: local variable 'name' referenced before assignment Ideas? Thanks much! Kerry ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error with tool migration
No, we're pulling galaxy-dist. Thanks for the quick reply, Kerry On Wed, Dec 18, 2013 at 4:07 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Kerry, I've committed a fix for this in 11800:a53f2f700699 to the galaxy-central branch. Is that the branch you're pulling? Thanks for reporting this. Greg Von Kuster On Dec 18, 2013, at 5:48 PM, Kerry Deutsch kerry.deut...@systemsbiology.org wrote: Hello - I pulled down the latest changes today (it was long overdue), and am running into the following error when I run the tool migration: sh ./scripts/migrate_tools/0008_tools.sh install_dependencies No handlers could be found for logger galaxy.tools.data Repositories will be installed into configured tool_path location ../shed_tools [localhost] local: rm -rf ./database/tmp/tmp-toolshed-mtdWuQjBx Skipping installation of tool dependency samtools version 0.1.18 since it is installed in /users/galaxy/galaxy_deps/bin/samtools/0.1.18/devteam/package_samtools_0_1_18/171cd8bc208d Traceback (most recent call last): File ./scripts/migrate_tools/migrate_tools.py, line 21, in module app = MigrateToolsApplication( sys.argv[ 1 ] ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py, line 83, in __init__ install_dependencies=install_dependencies ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 121, in __init__ is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 509, in install_repository is_repository_dependency=is_repository_dependency ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 351, in handle_repository_contents guid = self.get_guid( repository_clone_url, relative_install_dir, tool_config ) File /users/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py, line 259, in get_guid full_path = str( os.path.abspath( os.path.join( root, name ) ) ) UnboundLocalError: local variable 'name' referenced before assignment Ideas? Thanks much! Kerry ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
