[galaxy-dev] Multicore
Hi there, we have a 32 core server but I realised that a lot of programs are only using a fraction of that. I was able to adjust bowtie, bowtie2 and Tophat to use all of the cores but couldn't find an thread/core option for Macs14, Macs2 or PeakRanger. Does anyone know if these programs are capable of multi threading? If yes, where do I adjust it? Would be nice to have some global parameter set in the universewgsi.ini file. Thanks, Alex ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Manage Local data
Hi I am trying to load the mm10 genome and indexes using the manage local data admin feature. The genome downloads ok and so do the liftOver chain files, however the main controller and indexing jobs fail. They just display error, is there any way I can find out more about what is happening here and what is failing? Thanks Shaun Webb University of Edinburgh -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Multicore
Hi Alexander, There is a -t for PeakRanger to indicate the number of threads. I don't think macs14 or macs2 can run on mutltiple cores - I am cc'ing Tao Lui, the author of macs, who can confirm this. Thanks, Q On Tue, Mar 26, 2013 at 6:15 AM, Alexander Kurze alexander.ku...@gmail.com wrote: Hi there, we have a 32 core server but I realised that a lot of programs are only using a fraction of that. I was able to adjust bowtie, bowtie2 and Tophat to use all of the cores but couldn't find an thread/core option for Macs14, Macs2 or PeakRanger. Does anyone know if these programs are capable of multi threading? If yes, where do I adjust it? Would be nice to have some global parameter set in the universewgsi.ini file. Thanks, Alex ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
Hi Gordon, Thanks for your assistance and the recommendations. Freezing postgres sounds like hell to me :-) abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/25/2013 05:18 PM, Assaf Gordon wrote: Hello Joachim, Couple of things to check: On Mar 25, 2013, at 10:01 AM, Joachim Jacob | VIB | wrote: Hi, About the exporting of history, which fails: 1. the preparation seems to work fine: meaning: choosing 'Export this history' in the History menu leads to a URL that reports initially that the export is still in progress. 2. when the export is finished, and I click the download link, the root partition fills and the browser displays Error reading from remote server. A folder ccpp-2013-03-25-14:51:15-27045.new is created in the directory /var/spool/abrt, which fills the root partition. Something in your export is likely not finishing fine, but crashes instead (either the creation of the archive, or the download). The folder /var/spool/abrt/ccpp- (and especially a file named coredump) hints that the program crashed. abrt is a daemon (at least on Fedora) that monitors crashes and tries to keep all relevant information about the program which crashed (http://docs.fedoraproject.org/en-US/Fedora/13/html/Deployment_Guide/ch-abrt.html). So what might have happened, is that a program (galaxy's export_history.py or other) crashed during your export, and then abrt picked-up the pieces (storing a memory dump, for example), and then filled your disk. The handler reports in its log: galaxy.jobs DEBUG 2013-03-25 14:38:33,322 (8318) Working directory for job is: /mnt/galaxydb/job_working_directory/008/8318 galaxy.jobs.handler DEBUG 2013-03-25 14:38:33,322 dispatching job 8318 to local runner galaxy.jobs.handler INFO 2013-03-25 14:38:33,368 (8318) Job dispatched galaxy.jobs.runners.local DEBUG 2013-03-25 14:38:33,432 Local runner: starting job 8318 galaxy.jobs.runners.local DEBUG 2013-03-25 14:38:33,572 executing: python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat galaxy.jobs.runners.local DEBUG 2013-03-25 14:41:29,420 execution finished: python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat galaxy.jobs DEBUG 2013-03-25 14:41:29,476 Tool did not define exit code or stdio handling; checking stderr for success galaxy.tools DEBUG 2013-03-25 14:41:29,530 Error opening galaxy.json file: [Errno 2] No such file or directory: '/mnt/galaxydb/job_working_directory/008/8318/galaxy.json' galaxy.jobs DEBUG 2013-03-25 14:41:29,555 job 8318 ended The system reports: Mar 25 14:51:26 galaxy abrt[16805]: Write error: No space left on device Mar 25 14:51:27 galaxy abrt[16805]: Error writing '/var/spool/abrt/ccpp-2013-03-25-14:51:15-27045.new/coredump' One thing to try: if you have galaxy keeping temporary files, try running the export command manually: === python /home/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/export_history.py -G /mnt/galaxytemp/tmpHAEokb/tmpQM6g_R /mnt/galaxytemp/tmpHAEokb/tmpeg7bYF /mnt/galaxytemp/tmpHAEokb/tmpPXJ245 /mnt/galaxydb/files/013/dataset_13993.dat === Another thing to try: modify export_history.py, adding debug messages to track progress and whether it finishes or not. And: check the abrt program's GUI, perhaps you'll see previous crashes that were stored successfully, providing more information about which program crashed. As a general rule, it's best to keep the /var directory on a separate partition for production systems, exactly so that filling it up with junk wouldn't intervene with other programs. Even better, set each sub-directory of /var to a dedicated partition, so that filling up /var/log or /var/spool would not fill up /var/lib/pgsql and stop Postgres from working. -gordon
Re: [galaxy-dev] redirection vulnerability via URL injection
Thanks Dan! I am not sure about the dataset redirecting places other than ucsc and wormbase genome browser. By now, this can be done through Trackster right? Then I will probably disable the external redirecting. any comments/suggestions. thanks, --/Vipin Hi Vipin, Thank you for reporting this issue. This has to do with the way that the old-style (hard-coded) display applications were modified after introduction of roles to authorize access to an user's datasets that might be permission protected. Ideally, with these old-style applications, much of the work that is being done upfront on history load would be backended to happen after the user clicks to view a dataset -- e.g. the viewport is being generated for all datasets in a history, example link: http://localhost:8080/datasets/190/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dhg18%26position%3Dchr21%3A0-536870912%26hgt.customText%3D%25sdisplay_url=http%3A%2F%2Flocalhost%3A8080%2Froot%2Fdisplay_as%3Fid%3D190%26display_app%3Ducsc%26authz_method%3Ddisplay_at If redesigned so that everything happens after the user clicks the link, I see no reason why the redirect_url functionality could not be removed. As it stands now, the redirect url is %s substituted with the URL-encoded value that will contain the authorized URL to access the dataset (e.g. http://localhost:8080/root/display_as?id=190display_app=ucscauthz_method=display_at), and then the user is redirected there. I've added a Trello card (https://trello.com/c/uIctksud) for this issue. In the mean time, however, I have committed a patch to the stable branch that will allow administrators to disable the use of the old-style display applications. Thanks for using Galaxy, Dan On Mar 12, 2013, at 12:08 PM, Vipin TS wrote: Hello dev-members, We are trying to place our public Galaxy instancehttp://galaxy.raetschlab.org/in a more secured manner, Currently I am playing with few test cases about the redirection vulnerabilities. The following link uses a URL variable called “redirect_url” to redirect a user to a given page. While this variable is intended to only direct a user to a trusted page, it fails to validate the provided value and therefore can be used to redirect to any page. http://localhost:8080/datasets/332056/display_at/ucsc_test?redirect_url=http://www.google.comdisplay_url=http://localhost:8080/root This example redirects a user to Google, but it could just as easily be used to direct a user to a page that contains any malware. To resolve the issue, may be validate all user controlled input, including the GET request variables. If the input is intended to redirect a user, it must be validated to ensure it only presents them with a page on the trusted site. any comments or suggestions to work around this. thanks --/Vipin Rätschlab, Computational biology dept. Memorial Sloan-Kettering Cancer Center ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] hello Newbie questions about galaxy and clustering
Hello Jereme, I am going to post your question over to the galaxy-...@bx.psu.edu mailing list to give it better visibility to the development community. This is also where you will want to post future questions about your local install. http://wiki.galaxyproject.org/MailingLists For you question about cluster set-up: Starting at our basic install help: - http://wiki.galaxyproject.org/Admin/Get%20Galaxy At the bottom of the page is a link Running Galaxy in a production environment that has help for getting your server set up to use a cluster: - http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer The documentation for Using a compute cluster will then point to: - http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster Now, we there is a release pending. I might be worth waiting a week or so for this, unless you are already pulling from galaxy-central (and not the galaxy-dist) repository, to avoid having to configure changes twice. If you have follow-up, please leave the question on the mailing list - the developers will have the best advice about details. Best, Jen Galaxy team On 3/26/13 6:25 AM, leconte wrote: hello I have installed an galaxy instance for purpose testing on a 8 CPU server without problem all run perfectly. I have made some galaxy xml wrapper to understant how galaxy calls program and other little things. Now I want to install a Galaxy server with a cluster configuration. Before I have tested HTCondor but I have no Idea how I can mix Galaxy and HTCondor. The perfect thing was somebody have already tested this type of condor. But I can satisfy myself with any other cluster type. What I don't understant in fact ( maybe my worst problem ) where lie all parts of this puzzle. I explain. I know that I need Galaxy. I know too I need a cluster ( in my case HTCondor ) I understand I need something between these 2 parts ... I think something like PBS or SGE but it's not totaly clear for me I'm search on galaxy mailling list archive ... but I just found 2 mails in 2008 :( has anybody made this conf or have a configuration ? greetings PS : sorry for my poor english :) ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. A crashing python process crosses the fine boundary between the Galaxy code and Python internals... perhaps the Galaxy developers can help with this problem. It would be helpful to find a reproducible case with a specific history or a specific sequence of events, then someone can help you with the debugging. Once you find a history that causes a crash (every time or sometimes, but in a reproducible way), try to pinpoint when exactly it happens: Is it when you start preparing the export (and export_history.py is running as a job), or when you start downloading the exported file. (I'm a bit behind on the export mechanism, so perhaps there are other steps involved?). Couple of things to try: 1. set cleanup_job=never in your universe_wsgi.ini - this will keep the temporary files, and will help you re-produce jobs later. 2. Enable abrt again - it is not the problem (just the symptom). You can cleanup the /var/spool/abrt/XXX directory from previous crash logs, then reproduce a new crash, and look at the collected files (assuming you have enough space to store at least one crash). In particular, look at the file called coredump - it will tell you which script has crashed. Try running: $ file /var/spool/abrt//coredump coredump ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'python XX.py' Instead of .py it would show the python script that crashed (hopefully with full command-line parameters). It won't show which python statement caused the crash, but it will point in the right direction. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). It could be a bad dataset (file on disk), or a problem in the database, or something completely different (a bug in the python archive module). No point guessing until there are more details. -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy not picking up updates from test toolshed
Jorrit, Just to eliminate one possible cause of this behavior, could you confirm that your local Galaxy instance's universe_wsgi.ini file's enable_tool_shed_check setting is uncommented and set to True, and the hours_between_check is also uncommented and set to a value between 1 and 12? --Dave B. On 3/25/13 21:26:07.000, jorrit poelen wrote: Hey y'all, I noticed that my local galaxy instance doens't pickup latest updates from test toolshed. Steps to reproduce: 1. install obotools from test toolshed using hg clone http://jor...@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotools , revision 45:bb20766df19c 2. hg push new change of obotools of galaxy tool to testtoolshed , creating revision 46:532808477800 3. update obotools in galaxy admin expected result: galaxy admin detects new version and give an option to update. actual result: galaxy admin does not detect new version. I've attached some galaxy logging below. One thing I noticed that the reported commit time in testtoolshed seems 7 hours off - just minutes after I pushed the change, testtoolshed says the change was commit ~ 7 hours ago. Workaround I am using to do update is to deactivate and remove the tool all together, and re-install straight from test-toolshed. What should I do to make the tool version update work? I am probably missing something. Thanks! -jorrit Galaxy logging on looking for updates in test toolshed: --- 24.130.122.217 - - [26/Mar/2013:01:12:46 +] GET /admin_toolshed/browse_repositories HTTP/1.1 200 - http://galaxy:7474/admin; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/browse_repositories?operation=get+updatesid=a799d38679e985db HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/check_for_updates?id=a799d38679e985db HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:52 +] GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/name=obotoolsowner=jorritchangeset_revision=bb20766df19clatest_changeset_revision=bb20766df19clatest_ctx_rev=45 HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:52 +] GET /admin_toolshed/manage_repository?status=donemessage=The+installed+repository+named+%27obotools%27+is+current%2C+there+are+no+updates+available.++id=a799d38679e985db HTTP/1.1 200 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy not picking up updates from test toolshed
Hey Dave, Thanks against for your quick response! I checked my universe_wsgi.ini file and can confirm that the enabled_tool_shed_check was *commented*. I've uncommented the settings and changed them to: enable_tool_shed_check = True hours_between_check = 1 Then I followed the steps outlined in my initial email and the observed the same (unexpected) behavior. Perhaps I am describing a feature request rather than a bug: My expectation is that whenever I explicitly request to update a tool, a immediate check is performed against the relevant tool shed. To make sure that I correctly understand the existing tool update feature, please confirm that the following describes the currently expected tool update behavior: 0. admin of galaxy instance Z enables the feature to check for tool updates in toolsheds every hour 1. tool X revision 1.0 is installed from tool shed Y into galaxy instance Z by tool developer using hg push 2. tool X is current revision is updated to revision 1.1 in tool shed Y 3. immediately after, admin of galaxy instance Z attempts to (manually) upgrade tool X to version 1.1 using the admin web interface 4. galaxy instance Z reports that no new version of tool X are available, even though a new version exists in the toolshed 5. galaxy instance Z detects version 1.1 of tool X at most one hour after the update was made 6. admin of galaxy instance Z tries again Thanks for your patience and efforts, -jorrit On Mar 26, 2013, at 8:19 AM, Dave Bouvier wrote: Jorrit, Just to eliminate one possible cause of this behavior, could you confirm that your local Galaxy instance's universe_wsgi.ini file's enable_tool_shed_check setting is uncommented and set to True, and the hours_between_check is also uncommented and set to a value between 1 and 12? --Dave B. On 3/25/13 21:26:07.000, jorrit poelen wrote: Hey y'all, I noticed that my local galaxy instance doens't pickup latest updates from test toolshed. Steps to reproduce: 1. install obotools from test toolshed using hg clone http://jor...@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotools , revision 45:bb20766df19c 2. hg push new change of obotools of galaxy tool to testtoolshed , creating revision 46:532808477800 3. update obotools in galaxy admin expected result: galaxy admin detects new version and give an option to update. actual result: galaxy admin does not detect new version. I've attached some galaxy logging below. One thing I noticed that the reported commit time in testtoolshed seems 7 hours off - just minutes after I pushed the change, testtoolshed says the change was commit ~ 7 hours ago. Workaround I am using to do update is to deactivate and remove the tool all together, and re-install straight from test-toolshed. What should I do to make the tool version update work? I am probably missing something. Thanks! -jorrit Galaxy logging on looking for updates in test toolshed: --- 24.130.122.217 - - [26/Mar/2013:01:12:46 +] GET /admin_toolshed/browse_repositories HTTP/1.1 200 - http://galaxy:7474/admin; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/browse_repositories?operation=get+updatesid=a799d38679e985db HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/check_for_updates?id=a799d38679e985db HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:52 +] GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/name=obotoolsowner=jorritchangeset_revision=bb20766df19clatest_changeset_revision=bb20766df19clatest_ctx_rev=45 HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:52 +] GET /admin_toolshed/manage_repository?status=donemessage=The+installed+repository+named+%27obotools%27+is+current%2C+there+are+no+updates+available.++id=a799d38679e985db HTTP/1.1 200 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
[galaxy-dev] best practice to replace the hashing for galaxy-user passwords
Dear dev-team, I am interested to implement the secure hashed password specifically SHA-256 or greater to the galaxy users using publicly available python module PBKDF2. I have a piece of python code which will do the job but not quite sure how I can integrate this to my local galaxy code repository. I saw the current implementation in galaxy/util/hash_util.py any suggestions, thanks, --/Vipin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy not picking up updates from test toolshed
Jorrit, First, allow me to clarify a point of naming: It's not quite right to say tools in this situation, as tools are only one of several utilities that a tool shed repository can contain, along with datatypes, workflows, repository dependencies, and so on. This is an important distinction, because some repositories can (and do) contain no tools, and only datatypes and/or other Galaxy utilities. In your process description for the current behavior, the tool shed will not report updates to a repository if the tool version has changed. For more details on how and why this happens, see the following wiki page: http://wiki.galaxyproject.org/RepositoryRevisions#Adding_additional_change_sets_to_the_initial_change_set_in_a_repository Where the behavior is detailed in these sections: Adding additional change sets to the initial change set in a repository • What is repository metadata? • What does it mean when repository metadata is associated with a change set in the repository change log? • Adding a change set that does not generate a new repository metadata record • Adding a change set that generates a new repository metadata record • More examples of installable repository change set revisions --Dave B. On 3/26/13 12:45:41.000, jorrit poelen wrote: Hey Dave, Thanks against for your quick response! I checked my universe_wsgi.ini file and can confirm that the enabled_tool_shed_check was *commented*. I've uncommented the settings and changed them to: enable_tool_shed_check = True hours_between_check = 1 Then I followed the steps outlined in my initial email and the observed the same (unexpected) behavior. Perhaps I am describing a feature request rather than a bug: My expectation is that whenever I explicitly request to update a tool, a immediate check is performed against the relevant tool shed. To make sure that I correctly understand the existing tool update feature, please confirm that the following describes the currently expected tool update behavior: 0. admin of galaxy instance Z enables the feature to check for tool updates in toolsheds every hour 1. tool X revision 1.0 is installed from tool shed Y into galaxy instance Z by tool developer using hg push 2. tool X is current revision is updated to revision 1.1 in tool shed Y 3. immediately after, admin of galaxy instance Z attempts to (manually) upgrade tool X to version 1.1 using the admin web interface 4. galaxy instance Z reports that no new version of tool X are available, even though a new version exists in the toolshed 5. galaxy instance Z detects version 1.1 of tool X at most one hour after the update was made 6. admin of galaxy instance Z tries again Thanks for your patience and efforts, -jorrit On Mar 26, 2013, at 8:19 AM, Dave Bouvier wrote: Jorrit, Just to eliminate one possible cause of this behavior, could you confirm that your local Galaxy instance's universe_wsgi.ini file's enable_tool_shed_check setting is uncommented and set to True, and the hours_between_check is also uncommented and set to a value between 1 and 12? --Dave B. On 3/25/13 21:26:07.000, jorrit poelen wrote: Hey y'all, I noticed that my local galaxy instance doens't pickup latest updates from test toolshed. Steps to reproduce: 1. install obotools from test toolshed using hg clone http://jor...@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotools , revision 45:bb20766df19c 2. hg push new change of obotools of galaxy tool to testtoolshed , creating revision 46:532808477800 3. update obotools in galaxy admin expected result: galaxy admin detects new version and give an option to update. actual result: galaxy admin does not detect new version. I've attached some galaxy logging below. One thing I noticed that the reported commit time in testtoolshed seems 7 hours off - just minutes after I pushed the change, testtoolshed says the change was commit ~ 7 hours ago. Workaround I am using to do update is to deactivate and remove the tool all together, and re-install straight from test-toolshed. What should I do to make the tool version update work? I am probably missing something. Thanks! -jorrit Galaxy logging on looking for updates in test toolshed: --- 24.130.122.217 - - [26/Mar/2013:01:12:46 +] GET /admin_toolshed/browse_repositories HTTP/1.1 200 - http://galaxy:7474/admin; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/browse_repositories?operation=get+updatesid=a799d38679e985db HTTP/1.1 302 - http://galaxy:7474/admin_toolshed/browse_repositories; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10 24.130.122.217 - - [26/Mar/2013:01:12:51 +] GET /admin_toolshed/check_for_updates?id=a799d38679e985db HTTP/1.1 302 -
Re: [galaxy-dev] Multicore
Thanks a lot for your answers. I tried to set the -t value to 32 in peakranger but nothing really happens...it still runs on 1 thread even so it says -t 32 in the execution line I guess I need to contact the author. Thanks for your help! Alex On Tue, Mar 26, 2013 at 2:01 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Tue, Mar 26, 2013 at 10:15 AM, Alexander Kurze alexander.ku...@gmail.com wrote: Hi there, we have a 32 core server but I realised that a lot of programs are only using a fraction of that. I was able to adjust bowtie, bowtie2 and Tophat to use all of the cores but couldn't find an thread/core option for Macs14, Macs2 or PeakRanger. Does anyone know if these programs are capable of multi threading? If yes, where do I adjust it? Would be nice to have some global parameter set in the universewgsi.ini file. Thanks, Alex A more configurable thread setting has come up several times - something that could integrate nicely with cluster back ends for scheduling would be ideal. Some of my wrappers use $NSLOTS if set, and if not default to four threads. This works very nicely under SGE where you can set the thread count via the runner settings in universewgsi.ini - but this is not really general enough. For now however, you do generally have to tweak the wrappers as you have done to edit any hard coded thread setting :( Peter -- Alexander Kurze 70 Sunningwell Road Oxford, OX1 4SX Tel: +44 1865-243-609 Mobile: +44 7962-425-865 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
