[galaxy-user] Importing archived Histories
Hi all, I have some previously exported Histories saved as files. I want to import them to Galaxy but I can't work out how. I suspect I'm missing something basic! When I go to the History panel drop-down menu and select 'Export to File', I get a downloaded tar.gz file. When I select 'Import from File', I get a text box that asks me for the Archived History URL. But, I don't have a URL - I have a file on my local machine which I got by exporting as above. I have been able to import Histories directly from running Galaxy instances, by the way, I just can't work out how to do it from a file. Since it's possible to export them to a file in the first place, I am guessing that there's a way to import them too? I noticed that the forms for importing Histories and importing Workflows (.../workflow/import_workflow) are different in that the Workflows version explicitly asks for either a URL or for a local file. So, I have been able to successfully import Workflows from json files, but am confused about Histories. Thanks for any help! Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Sort sam or bam
Hi Carlos, Thanks! I didn't realise the conversion was doing that. In fact I want bam files (I'm also using GATK) so this is really helpful. Cheers, Clare On Thu, Jan 19, 2012 at 2:32 AM, Carlos Borroto carlos.borr...@gmail.com wrote: Hi Clare, I ran into a similar question testing out GATK pipeline on Galaxy. My solution was to always convert my SAM files to BAM. The wrapper also sorts the final BAM file output and it does this using the known 'samtools sort', which sorts chromosomes in the same order of the reference genome. The reference genome can be automatically selected by the build associated with the dataset or choose from the history. I haven't confirmed if a conversion from BAM to SAM would do the same thing. Hope it helps, Carlos On Wed, Jan 18, 2012 at 12:00 AM, Clare Sloggett s...@unimelb.edu.au wrote: Hi Jen, Thanks for this, belatedly! And happy new year! I think this will work for some of my cases but possibly not all, since it looks like chromosomes are being sorted alphabetically. It depends what the reference genome used was and what tools you're planning to use after sorting as to whether this is ok. I was initially just wondering why 'samtools sort' hadn't been wrapped - not as a complaint, but since the various other samtools options mostly seem to be already wrapped, I wondered if there was some particular reason not to have this one. If I were to try to wrap 'samtools sort' myself is there some difficulty I should know about? Thanks again, Clare On Wed, Dec 7, 2011 at 4:24 AM, Jennifer Jackson j...@bx.psu.edu wrote: Hello Clare, An example of how to sort a SAM file is included in the workflow from #2 on this FAQ (it can be imported and the sort modified as needed): http://usegalaxy.org/u/jeremy/p/transcriptome-analysis-faq If you are starting with a BAM file, convert BAM-SAM, then after sorting, back with SAM-BAM, using tools in the group NGS: SAM Tools. Best regards, Jen Galaxy team On 12/4/11 9:45 PM, Clare Sloggett wrote: Hi galaxy users, Am I right in thinking there is no tool for sorting a sam/bam file in Galaxy? I think this has probably been discussed before, sorry. I just want to check I haven't missed anything, since sibling tools from e.g. the samtools and picard suites are wrapped. Thanks, Clare -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Sort sam or bam
Hi Jen, Thanks for this, belatedly! And happy new year! I think this will work for some of my cases but possibly not all, since it looks like chromosomes are being sorted alphabetically. It depends what the reference genome used was and what tools you're planning to use after sorting as to whether this is ok. I was initially just wondering why 'samtools sort' hadn't been wrapped - not as a complaint, but since the various other samtools options mostly seem to be already wrapped, I wondered if there was some particular reason not to have this one. If I were to try to wrap 'samtools sort' myself is there some difficulty I should know about? Thanks again, Clare On Wed, Dec 7, 2011 at 4:24 AM, Jennifer Jackson j...@bx.psu.edu wrote: Hello Clare, An example of how to sort a SAM file is included in the workflow from #2 on this FAQ (it can be imported and the sort modified as needed): http://usegalaxy.org/u/jeremy/p/transcriptome-analysis-faq If you are starting with a BAM file, convert BAM-SAM, then after sorting, back with SAM-BAM, using tools in the group NGS: SAM Tools. Best regards, Jen Galaxy team On 12/4/11 9:45 PM, Clare Sloggett wrote: Hi galaxy users, Am I right in thinking there is no tool for sorting a sam/bam file in Galaxy? I think this has probably been discussed before, sorry. I just want to check I haven't missed anything, since sibling tools from e.g. the samtools and picard suites are wrapped. Thanks, Clare -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] Sort sam or bam
Hi galaxy users, Am I right in thinking there is no tool for sorting a sam/bam file in Galaxy? I think this has probably been discussed before, sorry. I just want to check I haven't missed anything, since sibling tools from e.g. the samtools and picard suites are wrapped. Thanks, Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Using Galaxy Cloudman for a workshop
Hi Ravi, Yes of course.. it should all be finished by 8th December so if I forget please feel free to ask me about the details after that! Cheers, Clare On Nov 17, 2011 3:27 PM, Clare Sloggett s...@unimelb.edu.au wrote: Hi all (especially Enis :) ), We are planning to use Amazon (Galaxy CloudMan) to run a workshop for about 50 people. We won't need to transfer any data during the workshop, but need the virtual cluster to be reasonably responsive and cope with: a) the load on the front end b) the workshop participants each trying to run a bwa alignment - at the moment each alignment would be of about 2.8M reads, but we could cut it down c) any other scalability issues I may not have thought of? I wanted to ask if anyone has used CloudMan for a similar purpose, or has an understanding, based on running a Galaxy cluster, of any problems we might encounter? I can add enough nodes to the cluster on the day to cope with the computational load (I assume) but I'm not sure if I should be expecting any other problems. Is the size of the node (e.g. Amazon's 4-core vs 8-core nodes) very important? I can scale out by adding more nodes, but should I be concerned about the capacity of the master node which handles the traffic? Also, is there any sensible way for me to test it in advance (in terms of the user load)? Many thanks for any advice! Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Using Galaxy Cloudman for a workshop
On Fri, Nov 18, 2011 at 12:56 AM, Jeremy Goecks jeremy.goe...@emory.edu wrote: Scalability issues are more likely to arise on the back end than the front end, so you'll want to ensure that you have enough compute nodes. BWA uses four nodes by default--Enis, does the cloud config change this parameter?--so you'll want 4x50 or 200 total nodes if you want everyone to be able to run a BWA job simultaneously. Actually, one other question - this paragraph makes me realise that I don't really understand how Galaxy is distributing jobs. I had thought that each job would only use one node, and in some cases take advantage of multiple cores within that node. I'm taking a node to be a set of cores with their own shared memory, so in this case a VM instance, is this right? If some types of jobs can be distributed over multiple nodes, can I configure, in Galaxy, how many nodes they should use? Thanks again, Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] Using Galaxy Cloudman for a workshop
Hi all (especially Enis :) ), We are planning to use Amazon (Galaxy CloudMan) to run a workshop for about 50 people. We won't need to transfer any data during the workshop, but need the virtual cluster to be reasonably responsive and cope with: a) the load on the front end b) the workshop participants each trying to run a bwa alignment - at the moment each alignment would be of about 2.8M reads, but we could cut it down c) any other scalability issues I may not have thought of? I wanted to ask if anyone has used CloudMan for a similar purpose, or has an understanding, based on running a Galaxy cluster, of any problems we might encounter? I can add enough nodes to the cluster on the day to cope with the computational load (I assume) but I'm not sure if I should be expecting any other problems. Is the size of the node (e.g. Amazon's 4-core vs 8-core nodes) very important? I can scale out by adding more nodes, but should I be concerned about the capacity of the master node which handles the traffic? Also, is there any sensible way for me to test it in advance (in terms of the user load)? Many thanks for any advice! Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] Problem launching cloud instance
Hi all, I wasn't sure whether to send this to galaxy-dev or galaxy-user. I'm having a problem launching a Galaxy instance on Amazon. I have done this in the past with no problems. I suspect I'm doing something trivial wrong but can't work out what! The problem is that after starting the instance, when I go to http://public-dns-url/cloud and the Initial Cluster Configuration window pops up, the initial storage size text box is disabled. That is, the text box after the first radio box (Galaxy Cluster) is greyed out. So, I can't enter a size for the storage and if I just proceed without doing so, it (I think) causes CloudMan to fail. I thought I was probably just having browser issues but I get the same problem repeatedly in Chrome and Firefox. When I inspect the element, I see input disabled=true ... name=g_pss... so I think it really is disabled. This seems like a weird problem so I suspect it's me, but I can't work it out. Any help would be appreciated! Thanks, Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Problem launching cloud instance
Update - I think I can see the problem, and I think it's a bug in the image on Amazon. I might have misunderstood, but thought I would point this out as a possible cause, assuming others experience the same issue I do. Looking at the repository at https://bitbucket.org/galaxy/cloudman/src/65f0ff532a29/templates/index.mako I can see input disabled='true' style=margin-left:20px type=text name=g_pss class=LV_field id=g_pss value= size=3GBspan id=g_pss_vtag... It looks like the input is being set to disabled in all cases instead of just when (as the comments suggest) the underlying machine image doesn't support it? It does look like this is a recent change - I think the culprit is commit https://bitbucket.org/galaxy/cloudman/changeset/65f0ff532a29#chg-templates/index.mako The thing I don't understand though is that I thought the image I was launching was older than this change. I launched 861460482541/galaxy-cloudman-2011-03-22 , and another older image seems to have the same problem . Do the instances draw on a configuration script which is newer than the image? Or maybe the cause is not what I've suggested! Any suggestions for quick workarounds to get an instance running successfully would also very much appreciated as I was planning to use a CloudMan instance to demo both Galaxy and CloudMan tomorrow to some people interested in using them (in about 11 hours from now in fact). Could I maybe log in to the CloudMan back end and configure it from there? Thanks, Clare On Sun, Oct 30, 2011 at 11:32 PM, Clare Sloggett s...@unimelb.edu.au wrote: Hi all, I wasn't sure whether to send this to galaxy-dev or galaxy-user. I'm having a problem launching a Galaxy instance on Amazon. I have done this in the past with no problems. I suspect I'm doing something trivial wrong but can't work out what! The problem is that after starting the instance, when I go to http://public-dns-url/cloud and the Initial Cluster Configuration window pops up, the initial storage size text box is disabled. That is, the text box after the first radio box (Galaxy Cluster) is greyed out. So, I can't enter a size for the storage and if I just proceed without doing so, it (I think) causes CloudMan to fail. I thought I was probably just having browser issues but I get the same problem repeatedly in Chrome and Firefox. When I inspect the element, I see input disabled=true ... name=g_pss... so I think it really is disabled. This seems like a weird problem so I suspect it's me, but I can't work it out. Any help would be appreciated! Thanks, Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Problem launching cloud instance
Wow, thanks for the fast response Brad Enis! On 31/10/2011 3:04 AM, Enis Afgan eaf...@emory.edu wrote: Hi Clare, Sorry about the trouble. I just disabled that if statement in the template so the option should no longer be disabled for any new cluster. My goal was to simplify usage but I'll obviously have to revisit the implementation... Sorry for the trouble and good luck with the demo tomorrow. Let me know if you run in to any other issues as an update to CloudMan was released just a couple of days ago. Enis On Sun, Oct 30, 2011 at 3:19 PM, Brad Chapman chapm...@50mail.com wrote: Clare; Enis will be able to better answer the questions about fixing this long term, but here is a quick workaround to get it running immediately: ssh to the image and edit '/mnt/cm/templates/index.mako' to remove disabled='true' from this line: https://bitbucket.org/galaxy/cloudman/src/b017694e2cfa/templates/index.mako#cl-384 Then if you refresh the http://your-dns/cloud page you'll be able to enter into this box. From there everything was working fine for me. Sorry about the problem and thanks for bringing this up, Brad On Sun, 30 Oct 2011 23:49:34 +1100, Clare Sloggett s...@unimelb.edu.au wrote: Update - I think I can see the problem, and I think it's a bug in the image on Amazon. I might have misunderstood, but thought I would point this out as a possible cause, assuming others experience the same issue I do. Looking at the repository at https://bitbucket.org/galaxy/cloudman/src/65f0ff532a29/templates/index.mako I can see input disabled='true' style=margin-left:20px type=text name=g_pss class=LV_field id=g_pss value= size=3GBspan id=g_pss_vtag... It looks like the input is being set to disabled in all cases instead of just when (as the comments suggest) the underlying machine image doesn't support it? It does look like this is a recent change - I think the culprit is commit https://bitbucket.org/galaxy/cloudman/changeset/65f0ff532a29#chg-templates/index.mako The thing I don't understand though is that I thought the image I was launching was older than this change. I launched 861460482541/galaxy-cloudman-2011-03-22 , and another older image seems to have the same problem . Do the instances draw on a configuration script which is newer than the image? Or maybe the cause is not what I've suggested! Any suggestions for quick workarounds to get an instance running successfully would also very much appreciated as I was planning to use a CloudMan instance to demo both Galaxy and CloudMan tomorrow to some people interested in using them (in about 11 hours from now in fact). Could I maybe log in to the CloudMan back end and configure it from there? Thanks, Clare On Sun, Oct 30, 2011 at 11:32 PM, Clare Sloggett s...@unimelb.edu.au wrote: Hi all, I wasn't sure whether to send this to galaxy-dev or galaxy-user. I'm having a problem launching a Galaxy instance on Amazon. I have done this in the past with no problems. I suspect I'm doing something trivial wrong but can't work out what! The problem is that after starting the instance, when I go to http://public-dns-url/cloud and the Initial Cluster Configuration window pops up, the initial storage size text box is disabled. That is, the text box after the first radio box (Galaxy Cluster) is greyed out. So, I can't enter a size for the storage and if I just proceed without doing so, it (I think) causes CloudMan to fail. I thought I was probably just having browser issues but I get the same problem repeatedly in Chrome and Firefox. When I inspect the element, I see input disabled=true ... name=g_pss... so I think it really is disabled. This seems like a weird problem so I suspect it's me, but I can't work it out. Any help would be appreciated! Thanks, Clare -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 -- E: s...@unimelb.edu.au P: 03 903 53357 M: 0414 854 759 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use
