Re: [galaxy-dev] FTP upload - symlink to uploaded data
Hi Rob, Indeed, I had/have some difficulties with setting temporary directories. The problem was that FTP uploaded data was first copied to TMPDIR prior to being put in the database directory. My solution: I extended the /tmp partition to several GB's, by mounting a bigger device over it. In addition, I have a large network share, which is mounted on /mnt/galaxytemp. The __new_file_path__ points to here. A bit messy indeed. I just had another discussion about the temporary directories with John Chilton and Jeremy Goecks, which you can read here: https://bitbucket.org/galaxy/galaxy-central/pull-request/139/letting-cuffdiff-use-__new_file_path__-as/diff From what I understood, __new_file_path__ is going to be phased out, in favour of __job_working_directory__. But apparently, the job_working_directory is not a temporary directory (in my case, it contains symlinks from the job_working_directory to database/files. In addition, job_working_directory is default part of the database directory of Galaxy.) The suggestion is to set TMPDIR env variable to a directory you specify. I have one file in /home/galaxy that contains the environment settings, and which gets sourced in the init script that launches Galaxy. Cheers, Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Wed 27 Mar 2013 08:44:08 PM CET, Rob Hooft wrote: Joachim, Nate, Leon Mei pointed me to a mailing list post of August 2012 where you two discussed a problem with uploads to Galaxy filling up /tmp. I think I have traced this down now after we suffered from this too several times. There are a number of places where temporary files are configurable in galaxy, but there is (at least) one place that uses the Python default directory (can be set with TMPDIR or some other envvars, but if you don't it is often /tmp). The unconfigurable place is tools/data_source/upload.py, where the code reads: if dataset.type == 'url': try: page = urllib.urlopen( dataset.path ) #page will be .close()ed by sniff methods temp_name, dataset.is_multi_byte = sniff.stream_to_file( page, prefix='url_paste', source_encoding=util.get_charset_from_http_headers( page.headers ) ) except Exception, e: file_err( 'Unable to fetch %s\n%s' % ( dataset.path, str( e ) ), dataset, json_file ) return dataset.path = temp_name sniff.stream_to_file uses the tempfile module, and since there is no dir= in the argument list to this call, the temporary file is made in /tmp. The central solution for the main galaxy code is in lib/galaxy/config.py: self.new_file_path = resolve_path( kwargs.get( new_file_path, database/tmp ), self.root ) tempfile.tempdir = self.new_file_path But this assignment to tempdir does not help in this case because upload.py is a tool? It would be nice to fix this, which we can obviously do ourselves for our andromeda deployment, but it would be better to do it centrally. Regards, Rob -- Rob W.W. Hooft Chief Technology Officer BioAssist, Netherlands Bioinformatics Centre http://www.nbic.nl/Skype: robhooftGSM: +31 6 27034319 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Knowing who is currently logged into your system
Hi Akshay, I am going to post your question over to the galaxy-...@bx.psu.edu mailing list, to give it better viability. You could probably run an sql query against a database table to find out this information, but there may be a better way. Let's see if someone has a method worked out they want to share. Another alternative is to look in our documentation, ReadTheDocs. I did a search on login and a few potentials popped up: http://galaxy-dist.readthedocs.org/en/latest/ Going forward, the galaxy-dev list is the list you will want to post to - and consider subscribing to - if you are running a local instance and want to join/discuss issues with the community of other users doing the same. http://wiki.galaxyproject.org/MailingLists http://wiki.galaxyproject.org/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/12/13 5:46 PM, Akshay Vivek Choche wrote: Hello All, I was wondering if there is a way to know all the users logged into your local galaxy server? Thanks, -Akshay Choche ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Knowing who is currently logged into your system
Akshay If you have galaxy_reports running on your server you can see the most likely logged in users under: Users/Date of last login when you query it for 0 days. You at least have a login per day (not on the hour though). For restarting purposes we ignore viewers and monitor the running jobs in galaxy from the admin interface. Additionally we check with htop on the server side of things if galaxy is very active. Or where you looking for some different functionality? Alex -Oorspronkelijk bericht- Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens Jennifer Jackson Verzonden: donderdag 28 maart 2013 10:29 Aan: Akshay Vivek Choche; Galaxy Dev Onderwerp: [galaxy-dev] Knowing who is currently logged into your system Hi Akshay, I am going to post your question over to the galaxy-...@bx.psu.edu mailing list, to give it better viability. You could probably run an sql query against a database table to find out this information, but there may be a better way. Let's see if someone has a method worked out they want to share. Another alternative is to look in our documentation, ReadTheDocs. I did a search on login and a few potentials popped up: http://galaxy-dist.readthedocs.org/en/latest/ Going forward, the galaxy-dev list is the list you will want to post to - and consider subscribing to - if you are running a local instance and want to join/discuss issues with the community of other users doing the same. http://wiki.galaxyproject.org/MailingLists http://wiki.galaxyproject.org/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/12/13 5:46 PM, Akshay Vivek Choche wrote: Hello All, I was wondering if there is a way to know all the users logged into your local galaxy server? Thanks, -Akshay Choche ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Circster Visualization
It works!! I didn't know I have to save before to use circster!! Great!! Thanks a lot!! :D Stefania On 27/mar/2013, at 17:57, James Taylor ja...@jamestaylor.org wrote: I think if you save your trackster visualization first then switching over to circster should work fine. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Wed, Mar 27, 2013 at 10:04 AM, Merella Stefania merella.stefa...@hsr.it wrote: Hi Jeremy, thanks for replying. When I click on Circster visualization the browser ask me Are you sure you want to leave this page? and when I click on Leave this page I receive this error: Visualization not found And this is from paster.log 172.25.46.193 - - [27/Mar/2013:15:02:57 +0200] GET /visualization/circster?id=undefined HTTP/1.1 200 - http://172.21.156.12/visualization/trackster; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31 Thanks! On 27/mar/2013, at 14:53, Jeremy Goecks jeremy.goe...@emory.edu wrote: But if I click on it I receive an error. Obviously I get this error because I didn't change any file to activate it. Unfortunately, this is not the case. If Trackster works, Circster should work. Can you share the error that you're getting when you click on the Circster icon? Thanks, J. -- LA TUA CURA E' SCRITTA NEL TUO DNA. AL SAN RAFFAELE LA STIAMO REALIZZANDO. AIUTA LA RICERCA, DAI IL TUO 5XMILLE - CF: 07636600962 info:www.5xmi...@hsr.it - www.5xmille.org Disclaimer added by CodeTwo Exchange Rules 2007 http://www.codetwo.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trinity wrapper not working on local installation
On Thu, Mar 28, 2013 at 4:36 AM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to get the trinity wrapper to work on our local galaxy installation using the latest trinity version. The main issue is that the Trinity.pl call has --SS_lib_type None in the script file. The data I am using is unstranded paired end reads and I am selecting None in the tool parameters. I looked at the trinity_all.xml file and it seems like the following code is not working: #if $inputs.library_type != 'None': --SS_lib_type $inputs.library_type #end if I am new to cheeta and python and I am not sure why this code is not working. Any suggestion on how to go about debugging it? This is almost certainly a type comparison error, obscured by the cheetah template language and the parameter proxy classes. In Python there is a special object None, which is probably what the library type is using. I would try making this an explicit comparison of strings (a pattern used in many other wrappers, e.g. tools/gatk/*.xml): #if str($inputs.library_type) != 'None': --SS_lib_type $inputs.library_type #end if Or, this might work too: #if $inputs.library_type != None: --SS_lib_type $inputs.library_type #end if (This does seem to be a bug in the trinity wrapper) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Cancel running jobs/workflows through API
Richard, You can find the code for admin users stopping jobs in lib/galaxy/web/base/controllers/admin.py, in the jobs method of the Admin class. Hopefully that will provide the base structure for what you want to accomplish. --Dave B. On 3/27/13 12:02:20.000, Richard Park wrote: Hey guys, I know there's no current way of canceling jobs or workflows through the API, but does anyone have any idea on how to implement this in Galaxy? i.e. where in the code base to see how galaxy is currently killing jobs. My ideal solution would be to get the list of qeued jobs for a giving history and call something to kill all running jobs in a history or a way of iterating through all jobs. Any help would greatly appreciated, thanks, Richard ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
Hi Assaf, After all, the problem appears not to be total size of the history, but the size of the individual datasets. Now, histories which contain big datasets (1GB) imported from Data Libraries causes the exporting process to crash. Can somebody confirm if this is a bug? I uploaded the datasets to a directory, which are then imported from that directory into a Data Library. Downloading data sets 1GB from a data library directly (as tar.gz) also crashes. Note: I have re-enabled abrt, but waiting for some jobs to be finished to restart. Cheers, Joachim. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Tue 26 Mar 2013 03:45:43 PM CET, Assaf Gordon wrote: Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. A crashing python process crosses the fine boundary between the Galaxy code and Python internals... perhaps the Galaxy developers can help with this problem. It would be helpful to find a reproducible case with a specific history or a specific sequence of events, then someone can help you with the debugging. Once you find a history that causes a crash (every time or sometimes, but in a reproducible way), try to pinpoint when exactly it happens: Is it when you start preparing the export (and export_history.py is running as a job), or when you start downloading the exported file. (I'm a bit behind on the export mechanism, so perhaps there are other steps involved?). Couple of things to try: 1. set cleanup_job=never in your universe_wsgi.ini - this will keep the temporary files, and will help you re-produce jobs later. 2. Enable abrt again - it is not the problem (just the symptom). You can cleanup the /var/spool/abrt/XXX directory from previous crash logs, then reproduce a new crash, and look at the collected files (assuming you have enough space to store at least one crash). In particular, look at the file called coredump - it will tell you which script has crashed. Try running: $ file /var/spool/abrt//coredump coredump ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'python XX.py' Instead of .py it would show the python script that crashed (hopefully with full command-line parameters). It won't show which python statement caused the crash, but it will point in the right direction. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). It could be a bad dataset (file on disk), or a problem in the database, or something completely different (a bug in the python archive module). No point guessing until there are more details. -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Exporting histories fails: no space left on device
OK, it seems to be a proxy error. When the proxy does not receive data from the server, it times out, and closes the connection. I think the process that packs the datasets takes too long, so the connection is closed before the packaging is finished? Just a gues... From the httpd logs: = [Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] (70007)The timeout specified has expired: proxy: error reading status line from remote server localhost, referer: http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1 [Thu Mar 28 15:14:46 2013] [error] [client 157.193.10.52] proxy: Error reading from remote server returned by /library_common/act_on_multiple_datasets, referer: http://galaxy.bits.vib.be/library_common/browse_library?sort=namef-description=Allf-name=Allid=142184b92db50a63cntrller=libraryasync=falseshow_item_checkboxes=falseoperation=browsepage=1 = See if changing time out settings fixes this issue. Cheers, Joachim Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 03/28/2013 02:58 PM, Joachim Jacob | VIB | wrote: Hi Assaf, After all, the problem appears not to be total size of the history, but the size of the individual datasets. Now, histories which contain big datasets (1GB) imported from Data Libraries causes the exporting process to crash. Can somebody confirm if this is a bug? I uploaded the datasets to a directory, which are then imported from that directory into a Data Library. Downloading data sets 1GB from a data library directly (as tar.gz) also crashes. Note: I have re-enabled abrt, but waiting for some jobs to be finished to restart. Cheers, Joachim. Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Tue 26 Mar 2013 03:45:43 PM CET, Assaf Gordon wrote: Hello Joachim, Joachim Jacob | VIB | wrote, On 03/26/2013 10:01 AM: abrt was filling the root directory indeed. So disabled it. I have done some exporting tests, and the behaviour is not consistent. 1. *size*: in general, it worked out for smaller datasets, and usually crashed on bigger ones (starting from 3 GB). So size is key? 2. But now I have found several histories of 4.5GB that I was able to export... So far for the size hypothesis. Another observation: when the export crashes, the corresponding webhandler process dies. A crashing python process crosses the fine boundary between the Galaxy code and Python internals... perhaps the Galaxy developers can help with this problem. It would be helpful to find a reproducible case with a specific history or a specific sequence of events, then someone can help you with the debugging. Once you find a history that causes a crash (every time or sometimes, but in a reproducible way), try to pinpoint when exactly it happens: Is it when you start preparing the export (and export_history.py is running as a job), or when you start downloading the exported file. (I'm a bit behind on the export mechanism, so perhaps there are other steps involved?). Couple of things to try: 1. set cleanup_job=never in your universe_wsgi.ini - this will keep the temporary files, and will help you re-produce jobs later. 2. Enable abrt again - it is not the problem (just the symptom). You can cleanup the /var/spool/abrt/XXX directory from previous crash logs, then reproduce a new crash, and look at the collected files (assuming you have enough space to store at least one crash). In particular, look at the file called coredump - it will tell you which script has crashed. Try running: $ file /var/spool/abrt//coredump coredump ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'python XX.py' Instead of .py it would show the python script that crashed (hopefully with full command-line parameters). It won't show which python statement caused the crash, but it will point in the right direction. So now I suspect something to be wrong with the datasets, but I am not able to trace something meaningful in the logs. I am not confident in turning on logging in Python yet, but apparently this happens with the module logging initiated like logging.getLogger( __name__ ). It could be a bad dataset (file on disk), or a problem in the database, or something completely different (a bug in the python archive module). No point guessing until there are more details. -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the
Re: [galaxy-dev] Improving Administrative Data Clean Up (pgcleanup.py vs cleanup_datasets.py)
On Mar 22, 2013, at 4:57 PM, Lance Parsons wrote: Nate Coraor wrote: On Mar 22, 2013, at 11:56 AM, Lance Parsons wrote: I have been running a Galaxy server for our sequencing researchers for a while now and it's become increasingly successful. The biggest resource challenge for us has been, and continues to be disk space. As such, I'd like to implement some additional cleanup scripts. I thought I run a few questions by this list before I got too far into things. In general, I'm wondering how to implement updates/additions to the cleanup system that will be in line with the direction that the Galaxy project is headed. The pgcleanup.py script is the newest piece of code in this area (and even adds cleanup of exported histories, which are absent from the older cleanup scripts). Also, the pgcleanup.py script uses a cleanup_event table that I don't believe is used by the older cleanup_datasets.py script. However, the new pgcleanup.py script only works for Postgres, and worse, only for version 9.1+. I run my system on RedHat (CentOS) and thus we use version 8.4 of Postgres. Are there plans to support other databases or older versions of Postgres? Hi Lance, pgcleanup.py makes extensive use of Writable CTEs, so there is not really a way to port it to older versions. For 8.4 or MySQL, you can still use the older cleanup_datasets.py. After looking at it a bit more, I see what you mean. Are there plans to implement and additional cleanup scripts for non-postgres 9.1 users? Just curious so I don't reinvent the wheel, I'd be happy to help with existing efforts. No, there aren't any plans as long as the alternative (cleanup_datasets.py) still works for other versions. I'd like to implement a script to delete (set the deleted flag) for certain datasets (e.g. raw data imported from our archive, for old, inactive users, etc.). I'm wondering if it would make sense to try and extend pgcleanup.py or cleanup_datasets.py. Or perhaps it would be best to just implement a separate script, though that seems like I'd have to re-implement a lot of boilerplate code for configuration reading, connections, logging, etc. Any tips on generally acceptable (supported) procedures for marking a dataset as deleted? You could probably reuse a lot of the code from either of the cleanup scripts for this. Right. It seems to make sense to me to focus on the cleanup_datasets.py since that will work for everyone. I would like to essentially mimic the user deleting a dataset. I'd then email them to let them know that some old data had been marked for deletion and let the rest of the scripts proceed as normal, cleaning that up if they don't undelete it. It looks like I would want to mark the HistoryDatasetAssociations as deleted? Is that correct? Would I need to do anything else to simulate the user deleting the dataset? That's correct. --nate Thanks for the help, Lance Thanks, --nate -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Missing from tool shed - lost in migration or never uploaded?
Hi Greg, Is there anything left of the old (pre-hg) Tool Shed? I'm wondering if I ever uploaded this tool back in early 2011, in which case could it have got lost in the migration to the current hg-based Tool Shed?: https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.txt https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.xml https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.py tool id=seq_primer_clip name=Primer clip sequences version=0.0.7 descriptionTrim off 5' or 3' primers/description command interpreter=python seq_primer_clip.py $input_file $input_file.ext $primer_fasta $primer_type $mm $min_len $keep_negatives $output_file /command ... If searching the archives is a hassle, don't worry - I'm just curious. Another likely explanation is simply that I never upload the tool - it was written for a Roche 454 dataset (to remove SMART primers) and could work directly on the SFF files. Unfortunately it is probably too slow to use on large Illumina FASTQ files. I might as well (re)post it to the Tool Shed now, perhaps with a warning that it is intended for smaller Roche/IonTorrent/Sanger sized datasets. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Missing from tool shed - lost in migration or never uploaded?
Hi Peter, Everything that was in the pre-hg tool shed was moved to the new tool shed - I don't remember any issues during that transition that would have eliminated anything. I don't think there is anything retrievable from the original tool shed at this point without some effort. You most likely had not uploaded this tool to the original tool shed. In any case, it's probably best to just upload it to a new repository in the tool shed. Thanks! Greg Von Kuster On Mar 28, 2013, at 10:39 AM, Peter Cock wrote: Hi Greg, Is there anything left of the old (pre-hg) Tool Shed? I'm wondering if I ever uploaded this tool back in early 2011, in which case could it have got lost in the migration to the current hg-based Tool Shed?: https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.txt https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.xml https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/primers/seq_primer_clip.py tool id=seq_primer_clip name=Primer clip sequences version=0.0.7 descriptionTrim off 5' or 3' primers/description command interpreter=python seq_primer_clip.py $input_file $input_file.ext $primer_fasta $primer_type $mm $min_len $keep_negatives $output_file /command ... If searching the archives is a hassle, don't worry - I'm just curious. Another likely explanation is simply that I never upload the tool - it was written for a Roche 454 dataset (to remove SMART primers) and could work directly on the SFF files. Unfortunately it is probably too slow to use on large Illumina FASTQ files. I might as well (re)post it to the Tool Shed now, perhaps with a warning that it is intended for smaller Roche/IonTorrent/Sanger sized datasets. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Knowing who is currently logged into your system
Hey Alex, Thanks for the quick response. The reason why I asked this question is because we reimplemented our plugin WS-Extension tool, which allows users to import web services as tools in Galaxy. This process involves making changes to tool_conf.xml, hence the restarting process is required. I also had an extension to this question, is it possible to customize Galaxy server for each user?, I think its not possible since it implies having separate tool_conf.xml for each user. I would appreciate if you could enlighten me on this. Thanks, -Akshay Choche On Mar 28, 2013, at 6:02 AM, Bossers, Alex alex.boss...@wur.nl wrote: Akshay If you have galaxy_reports running on your server you can see the most likely logged in users under: Users/Date of last login when you query it for 0 days. You at least have a login per day (not on the hour though). For restarting purposes we ignore viewers and monitor the running jobs in galaxy from the admin interface. Additionally we check with htop on the server side of things if galaxy is very active. Or where you looking for some different functionality? Alex -Oorspronkelijk bericht- Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens Jennifer Jackson Verzonden: donderdag 28 maart 2013 10:29 Aan: Akshay Vivek Choche; Galaxy Dev Onderwerp: [galaxy-dev] Knowing who is currently logged into your system Hi Akshay, I am going to post your question over to the galaxy-...@bx.psu.edu mailing list, to give it better viability. You could probably run an sql query against a database table to find out this information, but there may be a better way. Let's see if someone has a method worked out they want to share. Another alternative is to look in our documentation, ReadTheDocs. I did a search on login and a few potentials popped up: http://galaxy-dist.readthedocs.org/en/latest/ Going forward, the galaxy-dev list is the list you will want to post to - and consider subscribing to - if you are running a local instance and want to join/discuss issues with the community of other users doing the same. http://wiki.galaxyproject.org/MailingLists http://wiki.galaxyproject.org/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/12/13 5:46 PM, Akshay Vivek Choche wrote: Hello All, I was wondering if there is a way to know all the users logged into your local galaxy server? Thanks, -Akshay Choche ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] negative user data usage
Hi, Today, I found out that one user in our local galaxy installation (the administrator user) has a negative disk usage. - Reports shows : -72780720701 bytes - Galaxy history shows: -1% Does anybody have suggestions on what might be causing this and how to solve it? There is about 660 Gb of data in the histories of that user, but it was more before. I believe it happened after some histories were deleted and there was a message of one of them being shared. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trinity wrapper not working on local installation
I haven't looked at this directly, but it should be using None, in which case the correct comparison is library_type is None and library_type is not None. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Mar 28, 2013 at 6:35 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Mar 28, 2013 at 4:36 AM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to get the trinity wrapper to work on our local galaxy installation using the latest trinity version. The main issue is that the Trinity.pl call has --SS_lib_type None in the script file. The data I am using is unstranded paired end reads and I am selecting None in the tool parameters. I looked at the trinity_all.xml file and it seems like the following code is not working: #if $inputs.library_type != 'None': --SS_lib_type $inputs.library_type #end if I am new to cheeta and python and I am not sure why this code is not working. Any suggestion on how to go about debugging it? This is almost certainly a type comparison error, obscured by the cheetah template language and the parameter proxy classes. In Python there is a special object None, which is probably what the library type is using. I would try making this an explicit comparison of strings (a pattern used in many other wrappers, e.g. tools/gatk/*.xml): #if str($inputs.library_type) != 'None': --SS_lib_type $inputs.library_type #end if Or, this might work too: #if $inputs.library_type != None: --SS_lib_type $inputs.library_type #end if (This does seem to be a bug in the trinity wrapper) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trinity wrapper not working on local installation
Hi, None of the solutions that were mentioned fixed the issue. I will do some more debugging later this evening. Thanks, Raj On Thursday, March 28, 2013 12:26:56 PM, James Taylor wrote: I haven't looked at this directly, but it should be using None, in which case the correct comparison is library_type is None and library_type is not None. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Thu, Mar 28, 2013 at 6:35 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Mar 28, 2013 at 4:36 AM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am trying to get the trinity wrapper to work on our local galaxy installation using the latest trinity version. The main issue is that the Trinity.pl call has --SS_lib_type None in the script file. The data I am using is unstranded paired end reads and I am selecting None in the tool parameters. I looked at the trinity_all.xml file and it seems like the following code is not working: #if $inputs.library_type != 'None': --SS_lib_type $inputs.library_type #end if I am new to cheeta and python and I am not sure why this code is not working. Any suggestion on how to go about debugging it? This is almost certainly a type comparison error, obscured by the cheetah template language and the parameter proxy classes. In Python there is a special object None, which is probably what the library type is using. I would try making this an explicit comparison of strings (a pattern used in many other wrappers, e.g. tools/gatk/*.xml): #if str($inputs.library_type) != 'None': --SS_lib_type $inputs.library_type #end if Or, this might work too: #if $inputs.library_type != None: --SS_lib_type $inputs.library_type #end if (This does seem to be a bug in the trinity wrapper) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Bio-informatics consultant QBCG (http://qbcg.uga.edu) 706-542-6092 706-583-0442 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Running with Multiple DRMs
Hi Galaxy-Dev, I have a new local instance that submits to cluster1 via pbs-drmaa-1.0.14. Galaxy is on a VM that is a submit node on cluster1, and there is a shared filesystem amongst all the clusters where the datasets are stored. The Galaxy VM is also a submit node on cluster2 and we have installed another Torque client on the VM in a different location to interface with that one. We have confirmed that using /usr/local/bin/qsub submits to cluster1 and using /N/softs/galaxy/torque/qsub submits to cluster2. The two clusters are using different versions of Torque, but our end goal is to find a way to make it possible to submit to condor and sge as well (as I understand, setting up a distributed Galaxy is a hot topic for multiple institutions). My question is then, how to go about submitting certain jobs (say, all blast jobs) to cluster2 from Galaxy? Here are my ideas so far: (1.) If I can change the path from the default Torque client to our new Torque client while a tool is being submitted, that should do the trick -- that way Galaxy and Drmaa don't need to know the details. I thought about doing this in 200_runners.py but I don't think it would work to just export the path. (2.) If I can get drmaa configured to submit using the correct qsub command given drmaa://-q queuename@host, that would work. Currently just trying the queue and hostname hasn't worked. (3.) If tool handlers can be configured to use different settings for running jobs, this might be workable. I looked into: http://wiki.galaxyproject.org/Admin/Config/Jobs That looks very promising but I don't see job_conf* anywhere in my new clone. (4.) Change the wrapper to include some other parameter returned by 200_runners.py as is suggested here: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010080.html Not sure if this would do what I want. (5.) I may need to make changes to execute() and DefaultToolAction as alluded to here: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/009964.html This will require more time and energy to understand and modify than I have available at the moment. Are there any other insights into this sort of setup? Thanks! -Carrie Ganote ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Setting up SSL on CloudMan EC2
Has anyone configured SSL for nginx on CloudMan (AWS)? If so, can you share
your nginx.conf file as I'm having some issues.
*Details*
I tried adopting the nginx.conf for CloudMan+CloudBioLiniux as described
herehttps://production-galaxy-instances-with-cloudman-and-cloudbiolinux.readthedocs.org/en/latest/
but
ran into a problem with the line (in red):
server {
* listen 443 default_server ssl;*
ssl_certificate /usr/nginx/conf/cert;
ssl_certificate_key /usr/nginx/conf/key;
}
According to *nginx.conf.default* the configuration for HTTPS looks like:
*server {*
*listen 443;*
*server_name localhost;*
*...*
*}*
This suggested that the documentation was incorrect, and that I needed to
insert a semicolon after '*listen 443*'.
*[emerg]: unknown directive default_server in
/opt/galaxy/pkg/nginx/conf/nginx.conf:123*
*
*
*
*Returning again to nginx.conf.default I 'replaced default_server ssl' with:
*server {*
*listen 443;*
*server_name localhost;*
*...*
*}
*
After restarting the galaxy server, I hit out server and then directed to *
http://localhost*
On the one hand, I'm not surprised that I am directed to localhost, however
I see the same usage for describing the service for port 80.
Nevertheless, in the CloudMan+CloudBioLinux docs following section was to
be added to the port 80 section.
server {
listen 80;
server_namemygalaxy.example.com;
rewrite^ https://$server_name$request_uri? permanent;
}
so I changed $server_name - $host. Redirection worked, but there was no
response on the other end. There are several avenues I could take from
here. Any suggestions?
-Rob
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Toolshed - Installation of static files
Hello Lance, I've committed a fix for this issue into the next-stable branch on the galaxy-central repository in change set revision 9212:92c8a55a6679, so the fix will be included in the upcoming Galaxy release. This fix is currently running on both the test and main Galaxy tool sheds as well. You'll need to change your tool config file to use a new reserved word as follows: .. image:: $PATH_TO_IMAGES/count_modes.png This reserved word will route correctly in the tool shed when you display the tool inside of it's containing repository, and it will also route correctly when the tool is loaded from it's repository after it has been installed into Galaxy. All future tool configs that display images in the tool's help section will need to use this approach for tools included in repositories in the tool shed. Thanks very much for reporting this issue, and please let me know if you run into any issues. Greg Von Kuster On Mar 22, 2013, at 1:50 PM, Lance Parsons wrote: Thanks for the quick response Greg. I tried this and uploaded an update to the testtoolshed and installed that in a test instance of Galaxy. Unfortunately, it didn't seem to resolve the issue. Galaxy still renders the html with the image source pointing to the same location (e.g. http://127.0.0.1:8080/static/images/count_modes.png). This doesn't resolve. You can try installing the htseq_count package from the testtoolshed to see the issue for yourself. Lance Greg Von Kuster wrote: Hi Lance, I answered this in your previous email - here's the answer: To make this work in the tool shed, make the static directory that contains the image file a relative path in the tool config. For example, your repository has the following structure: htseq_count htseq-count.xml sam_fa_indices.loc.sample static/ images/ count_modes.png test-data/ tool_data_table_conf.xml.sample tool_dependencies.xml The help section of your tool config should simply use a relative path to the image file ( in this case, just a a . in front of the path to the image file ) .. image:: ./static/images/count_modes.png :width: 500 I believe that's all that is necessary. Greg Von Kuster On Mar 22, 2013, at 12:44 PM, Lance Parsons wrote: For one of my tool wrappers (htseq-count) in the toolshed, I point to an image file in the Help section. At the moment, I'm including the image in the repository as static/imagescount_modes.png and pointing to it in the wrapper.xml file with .. image:: /static/images/count_modes.png. However, the toolshed installation doesn't copy this file to the static directory of the galaxy install (which doesn't seem like a good idea anyway). Is there any way for me to include a static file and then point to that file in the help when the tool is installed automatically via the toolshed.. Thanks. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] upgrading cufflinks version to 2.0.2
I am trying to upgrade cufflinks version from 1.3.0 to 2.0.2. I changed the directory in the PATH to version 2.0.2 but when I run cuffdiff on the galaxy it still uses the version 1.3.0. Do I need to change the code of the wrapper? This is the right approach, but there must be a glitch somewhere. If you're running Galaxy on a cluster and/or via a dedicated user, you'll want to set the PATH accordingly. For debugging, you could try echoing the path in the tool wrapper. Another question is how to I enable both version of cufflinks to appear and run on the galaxy? This isn't possible now but will be in the future when Cufflinks is moved to the tool shed. Best, J.___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] inable to login to testtoolshed login because of Server Error: AttributeError: 'GalaxySession' object has no attribute 'current_history'
Hey y'all galaxiers, Today, I tried to login using the testtoolshed web ui. To make sure that it wasn't just isolated to an existing account, I created a new account, and still I encountered the Server Error attached below. This error is not shown when entering an invalid password. I was able to login and push hg changes a couple of days ago. Was the test toolshed recently upgraded? Please let me know what I can do to login to testtoolshed at http://testtoolshed.g2.bx.psu.edu thx, -jorrit PS Please find attached stack trace on login with valid password through toolshed web ui at around 2pm PST. URL: http://testtoolshed.g2.bx.psu.edu/user/login?use_panels=True File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 128 in __call__ return self.handle_request( environ, start_response ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 184 in handle_request body = method( trans, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/webapps/galaxy/controllers/user.py', line 466 in login active_view=user ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 863 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 874 in fill_template_mako return template.render( **data ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/base/base_panels.mako.py', line 85 in render_body __M_writer(unicode(self.masthead())) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/webapps/galaxy/base_panels.mako.py', line 337 in render_masthead usage = trans.app.quota_agent.get_usage( trans=trans ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/quota/__init__.py', line 26 in get_usage history = trans.history File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 741 in get_history history = self.galaxy_session.current_history AttributeError: 'GalaxySession' object has no attribute 'current_history' ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy not picking up updates from test toolshed
Jorrit, I may have been a bit unclear in my previous response. Allow me to illustrate two scenarios: First scenario: 1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, tool version is *not* changed. Second scenario: 1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, and updated BWA tool is uploaded with version 0.6 In the first scenario, Galaxy will find that the repository has been updated, and apply the update. In the second scenario, Galaxy will not find that the repository has been updated, but will allow you to install both repositories, the one containing version 0.5.9 and the one containing version 0.6 of the BWA tool. The reason for this behavior is that if a tool has been updated to produce different output, the tool version should be updated, which leads to a separate installable changeset revision being generated. This way, any analyses run on the old tool version can be re-run and produce the same output, but any new analyses can run on the new version of the tool, with whatever new features have been added. This ensures reproducibility throughout the version lineage of the tool, while still allowing Galaxy users to use the most recent version of an available tool for future analyses. --Dave B. On 3/28/13 17:03:10.000, jorrit poelen wrote: Hey Dave, Thanks for pointing me to the wiki that helped demystify the hg changeset -- repository metadata connection for me. I'll make sure to change the version numbers in the tool xml definitions whenever a change needs to be make available to galaxy instances through the toolshed. So, even for an update in the README file, I'll make sure to change some version to make the change available. This feels a little clunky for me now, but I am sure there's a good reason to make it work as it does now. I very much appreciate your prompt response and I'll make sure to dig around the wiki a little longer before posting a question next time ; ) -jorrit On Mar 26, 2013, at 10:41 AM, Dave Bouvier wrote: Jorrit, First, allow me to clarify a point of naming: It's not quite right to say tools in this situation, as tools are only one of several utilities that a tool shed repository can contain, along with datatypes, workflows, repository dependencies, and so on. This is an important distinction, because some repositories can (and do) contain no tools, and only datatypes and/or other Galaxy utilities. In your process description for the current behavior, the tool shed will not report updates to a repository if the tool version has changed. For more details on how and why this happens, see the following wiki page: http://wiki.galaxyproject.org/RepositoryRevisions#Adding_additional_change_sets_to_the_initial_change_set_in_a_repository Where the behavior is detailed in these sections: Adding additional change sets to the initial change set in a repository • What is repository metadata? • What does it mean when repository metadata is associated with a change set in the repository change log? • Adding a change set that does not generate a new repository metadata record • Adding a change set that generates a new repository metadata record • More examples of installable repository change set revisions --Dave B. On 3/26/13 12:45:41.000, jorrit poelen wrote: Hey Dave, Thanks against for your quick response! I checked my universe_wsgi.ini file and can confirm that the enabled_tool_shed_check was *commented*. I've uncommented the settings and changed them to: enable_tool_shed_check = True hours_between_check = 1 Then I followed the steps outlined in my initial email and the observed the same (unexpected) behavior. Perhaps I am describing a feature request rather than a bug: My expectation is that whenever I explicitly request to update a tool, a immediate check is performed against the relevant tool shed. To make sure that I correctly understand the existing tool update feature, please confirm that the following describes the currently expected tool update behavior: 0. admin of galaxy instance Z enables the feature to check for tool updates in toolsheds every hour 1. tool X revision 1.0 is installed from tool shed Y into galaxy instance Z by tool developer using hg push 2. tool X is current revision is updated to revision 1.1 in tool shed Y 3. immediately after, admin of galaxy instance Z attempts to (manually) upgrade tool X to version 1.1 using the admin web interface 4. galaxy instance Z reports that no new version of tool X are available, even though a new version exists in the toolshed 5. galaxy instance Z detects version 1.1 of tool X at most one hour after the update was made 6. admin of galaxy instance Z tries again Thanks for your patience and efforts, -jorrit On
Re: [galaxy-dev] inable to login to testtoolshed login because of Server Error: AttributeError: 'GalaxySession' object has no attribute 'current_history'
Jorrit, While I investigate the root cause of this, have you tried clearing all cookies for testtoolshed.g2.bx.psu.edu? --Dave B. On 3/28/13 17:14:55.000, jorrit poelen wrote: Hey y'all galaxiers, Today, I tried to login using the testtoolshed web ui. To make sure that it wasn't just isolated to an existing account, I created a new account, and still I encountered the Server Error attached below. This error is not shown when entering an invalid password. I was able to login and push hg changes a couple of days ago. Was the test toolshed recently upgraded? Please let me know what I can do to login to testtoolshed at http://testtoolshed.g2.bx.psu.edu thx, -jorrit PS Please find attached stack trace on login with valid password through toolshed web ui at around 2pm PST. URL: http://testtoolshed.g2.bx.psu.edu/user/login?use_panels=True File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 128 in __call__ return self.handle_request( environ, start_response ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 184 in handle_request body = method( trans, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/webapps/galaxy/controllers/user.py', line 466 in login active_view=user ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 863 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 874 in fill_template_mako return template.render( **data ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/base/base_panels.mako.py', line 85 in render_body __M_writer(unicode(self.masthead())) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/webapps/galaxy/base_panels.mako.py', line 337 in render_masthead usage = trans.app.quota_agent.get_usage( trans=trans ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/quota/__init__.py', line 26 in get_usage history = trans.history File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 741 in get_history history = self.galaxy_session.current_history AttributeError: 'GalaxySession' object has no attribute 'current_history' ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] list of tool versions and galaxy ami?
Hi all, Is there a canonical list of current tool versions that are running in the public galaxy? Is there a way to get that list? Also, I was hoping that there might be a galaxy ami that I could use in the amazon cloud and by that I mean NOT a cloudman instance. I am looking for an instance that runs galaxy directly. Does that exist? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy not picking up updates from test toolshed
Jorrit, I've not been able to reproduce the first scenario using a local Galaxy instance against the test tool shed. Could you tell me which revision of Galaxy you're running when you encounter this behavior? I do see that the revision bb20766df19c that you were initially trying to update is immediately followed by another downloadable revision, in which case Galaxy will (correctly) not find any updates for it, due to the reproducibility concerns I mentioned in my previous email. --Dave B. On 3/28/13 17:58:10.000, jorrit poelen wrote: Hey Dave, I took some time to try to reproduce the behavior you described using a local test tool shed. As I understand your previous email, the first scenario (below) outcome of using my local test toolshed is expected, but the second scenario is not. Also, the first scenario outcome using the psu test toolshed seems different compared to running it against my local test tool shed. I was unable to test the second scenario, because of a apparent (recently introduced) bug in the psu test tool shed login. What am I missing? Thanks much for you patience, -jorrit --- First scenario - Change README / update Steps: 1. install repository A from local test tool shed B into local galaxy instance C 2. clone respository A from local test tool into local dev environment **3. apply change to README** 4. hg commit / push changes to local test tool shed B 5. verify that test tool shed received changes by checking content of README and change set hash. 6. in galaxy instance C admin manage installed tool shed repositories repository A get updates 7. galaxy instance C web page says that updates have been detected and applied **8. verify that README of repo A in galaxy instance C has been changed.** Outcome: local tool shed instance test - update detected, changes were applied to local galaxy instance C psu tool shed test - update *not* detected, changes were *not* applied to local galaxy instance C (see initial post) -- Second scenario - Bump tool version / update -- Steps: same as first scenario, except for **3. apply change to tool version 0.1 -- 0.11 ** **8. verify that tool version 0.1 has changed to 0.11 of repo A in galaxy instance C has been changed.** Outcome: local tool shed instance test - update detected, changes were applied: a new version of tool is installed in Galaxy instance. psu tool shed test - unable to test because I could not login to psu test toolshed On Mar 28, 2013, at 2:19 PM, Dave Bouvier wrote: Jorrit, I may have been a bit unclear in my previous response. Allow me to illustrate two scenarios: First scenario: 1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, tool version is *not* changed. Second scenario: 1. A repository is created containing a BWA tool with version 0.5.9 2. The repository is installed to Galaxy. 3. New README file is uploaded, and updated BWA tool is uploaded with version 0.6 In the first scenario, Galaxy will find that the repository has been updated, and apply the update. In the second scenario, Galaxy will not find that the repository has been updated, but will allow you to install both repositories, the one containing version 0.5.9 and the one containing version 0.6 of the BWA tool. The reason for this behavior is that if a tool has been updated to produce different output, the tool version should be updated, which leads to a separate installable changeset revision being generated. This way, any analyses run on the old tool version can be re-run and produce the same output, but any new analyses can run on the new version of the tool, with whatever new features have been added. This ensures reproducibility throughout the version lineage of the tool, while still allowing Galaxy users to use the most recent version of an available tool for future analyses. --Dave B. On 3/28/13 17:03:10.000, jorrit poelen wrote: Hey Dave, Thanks for pointing me to the wiki that helped demystify the hg changeset -- repository metadata connection for me. I'll make sure to change the version numbers in the tool xml definitions whenever a change needs to be make available to galaxy instances through the toolshed. So, even for an update in the README file, I'll make sure to change some version to make the change available. This feels a little clunky for me now, but I am sure there's a good reason to make it work as it does now. I very much appreciate your prompt response and I'll make sure to dig around the wiki a little longer before posting a question next time ; ) -jorrit On Mar 26, 2013, at 10:41 AM, Dave Bouvier wrote: Jorrit, First, allow me to clarify a point of naming: It's not quite right to say tools in this situation, as tools are only one of several utilities that a tool shed repository can contain, along with datatypes, workflows, repository
Re: [galaxy-dev] inable to login to testtoolshed login because of Server Error: AttributeError: 'GalaxySession' object has no attribute 'current_history'
Jorrit, Thank you for the additional information, following those steps I was able to reproduce the error and will be looking into the issue. --Dave B. On 3/28/13 18:04:48.000, jorrit poelen wrote: Hey Dave, I just reproduced this issue by: 1. clear all cookies (reset browser) 2. login to testtoolshed.g2.bx.psu.edu 3. login succeeds 4. logout 5. logout succeeds 6. (re-)login to testtoolshed.g2.bx.psu.edu 7. login fails with Server Error (for stacktrace, see below). thx, -jorrit URL: http://testtoolshed.g2.bx.psu.edu/user/login?use_panels=True File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 128 in __call__ return self.handle_request( environ, start_response ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 184 in handle_request body = method( trans, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/webapps/galaxy/controllers/user.py', line 466 in login active_view=user ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 863 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 874 in fill_template_mako return template.render( **data ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/base/base_panels.mako.py', line 85 in render_body __M_writer(unicode(self.masthead())) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/database/compiled_templates/community/webapps/galaxy/base_panels.mako.py', line 209 in render_masthead tab( lab, Lab, None, menu_options=menu_options, visible=( trans.user and ( trans.user.requests or trans.app.security_agent.get_accessible_request_types( trans, trans.user ) ) ) ) AttributeError: 'User' object has no attribute 'requests' On Mar 28, 2013, at 2:22 PM, Dave Bouvier wrote: Jorrit, While I investigate the root cause of this, have you tried clearing all cookies for testtoolshed.g2.bx.psu.edu? --Dave B. On 3/28/13 17:14:55.000, jorrit poelen wrote: Hey y'all galaxiers, Today, I tried to login using the testtoolshed web ui. To make sure that it wasn't just isolated to an existing account, I created a new account, and still I encountered the Server Error attached below. This error is not shown when entering an invalid password. I was able to login and push hg changes a couple of days ago. Was the test toolshed recently upgraded? Please let me know what I can do to login to testtoolshed at http://testtoolshed.g2.bx.psu.edu thx, -jorrit PS Please find attached stack trace on login with valid password through toolshed web ui at around 2pm PST. URL: http://testtoolshed.g2.bx.psu.edu/user/login?use_panels=True File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File
[galaxy-dev] directory copies as zero-length file from job_working_directory?
Hi all, So I have a tool that I am writing a wrapper script for that produces a directory of images as one of its outputs. This directory gets created just fine in the job_working_directory, but then when the job finishes and the contents of the job_working_directory are copied, the copy of the images directory is a zero-length *file*. I seem to recall having this problem before and finding a solution to it, but now I can't seem to find it. Any ideas? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
