Re: [galaxy-dev] Trouble maintaining Galaxy config on cloud instance

Hi David, I'm afraid some of that is embedded in CloudMan's code. Given you're restarting your cluster, you can unpack cm.tar.gz from your cluster's bucket (you can find this on CloudMan's Admin page) and edit *cm/util/galaxy_conf.py* to change the value for desired variables (under *populate_galax

Re: [galaxy-dev] SSL/LDAP configuration for CloudMan/Galaxy

Hi David, The nginx change will require a new AMI that includes the appropriately compiled nginx. We've been working on automating the process of building the image and it's captured in this Ansible playbook: https://github.com/galaxyproject/galaxy-cloudman-playbook (the current README is a bit of

Re: [galaxy-dev] job queue problem with cloud man

Hi Alexander, I'm glad to hear you got the jobs running. I don't think there's an informative message that is being delivered when a job is not able to run but looking at Galaxy (and/or CloudMan) logs might be necessary. FYI, the version of CloudMan you're running (i.e., the currentlt released ver

Re: [galaxy-dev] Data Manager - BWA-MEM index

To see the data in a Data Manager, you first need to fetch the data via a Data Manager vs. a history. So, you'll first need to install the fetch DM ( https://toolshed.g2.bx.psu.edu/repository?repository_id=cdf172d633d603c2&changeset_revision=bca4c608408c) and then click on 'Reference Genome - fetch

Re: [galaxy-dev] config files in cloudman

The variables that get 'reset' are stored in this file https://github.com/galaxyproject/cloudman/blob/master/cm/util/galaxy_conf.py#L133 You’d have to change the values in that file, upload the new cm.tar.gz to your bucket and restart/relaunch the cluster. It's on the agenda to fix this so it can

Re: [galaxy-dev] cloudman error somewhere

Hi Alexander, When you build the image, did you run the image cleanup step? RabbitMQ's database needs to be cleaned up before building the image otherwise the application won't start properly and the AMQP connection will never be established. You can also post the entire CloudMan's log somewhere a

Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

Hi Nick, Sorry to hear you're having trouble. I just tried a couple of scenarios and they all worked as expected (e.g., with and without elastic IPs, different instance types). The main CloudMan log is located in /mnt/cm/paster.log, on both master and worker instances (if you didn't download the s

Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

gt; G12 8TA > > Tel: +44 141 330 8282 > > http://fb.me/WTCMPbix > @WTCMPbix > http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/ > > http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/ > > From: Enis Afgan

Re: [galaxy-dev] Cloudman cluster not starting workers

://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/ > > http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/ > > From: Nick Dickens > Date: Monday, 22 June 2015 17:02 > To: Enis Afgan > Cc: "galaxy-dev@lists.galaxyproject.

Re: [galaxy-dev] cloudman error somewhere

t; and stdout: 'Activating virtualenv at /mnt/galaxy/galaxy-app/.venv >> Error: Unable to write to log file: [Errno 13] Permission denied: 'main.log' >> ' >> >> The directory /mnt/galaxy/galaxy-app is owned by galaxy. > Where is cloudman trying to wr

Re: [galaxy-dev] cloudman launch

Hi Alexander, Let me investigate this a bit more and get back to you because I was under the impression I had fixed this. In the mean time, I updated your flavor to specify snap-6f0b211d. On Wed, Jul 1, 2015 at 4:44 PM, Alexander Vowinkel < vowinkel.alexan...@gmail.com> wrote: > Hi Enis, > > I

Re: [galaxy-dev] cloudman launch

anything about it until after GCC). Hope this helps and let me know if you have more questions, Enis On Thu, Jul 2, 2015 at 9:29 AM, Enis Afgan wrote: > Hi Alexander, > Let me investigate this a bit more and get back to you because I was under > the impression I had fixed this. In

Re: [galaxy-dev] cloudman persist filesystem error

Hi Alexander, I tested the process of persisting file system changes but it all worked as expected. Looking at the log snippet you sent, the only thing that seems off is the very first line: Completed creation of a snapshot for the volume 'None', snap id: 'snap-6f0b211d' - volume should not be None

Re: [galaxy-dev] cloudman launch

> > > > A) I wanted to tell you that I did the update of the galaxyFS > with the playbook, by using the role cm.filesystem only. > It worked - thanks! > Excellent! > > B) How could I mount another volume to the upstarting cluster? > > - name: stuff >snap_id: snap-xyz > > Would that work? >

Re: [galaxy-dev] cloudman galaxy - too large volume

I just fixed this here https://github.com/galaxyproject/cloudman/commit/0f6ab132958ad57a338c66387112407edbc7632d#diff-9eba488c9175de87945a13d7b7228a6dR621 This required switching to SSD-based volumes on AWS and, besides actually being able to create volumes up to 16TB now, it should give us better

Re: [galaxy-dev] cloudman galaxy - too large volume

olume type manually by putting 'volume_type' > to the user data? > So I can also have smaller volumes with SSD? > Best, > Alexander > > 2015-07-14 13:57 GMT-05:00 Enis Afgan : > >> I just fixed this here >> https://github.com/ga

Re: [galaxy-dev] CloudMan auto-scaling - Use spot instance for worker node?

I've created a trello card for this, hence making it adding it to the actual todo list: https://trello.com/c/cqEwunvl Not sure when I'll get to this but it'll get done. Thanks. On Fri, Jul 17, 2015 at 11:27 AM, Alexander Vowinkel < vowinkel.alexan...@gmail.com> wrote: > Hi, > > I am referring to

Re: [galaxy-dev] Galaxy in Amazon

Hi Ryan, What you're suggesting to do is still somewhat experimental but we're continuing to work on it to make it more integrated into the Galaxy ecosystem and more robust. There are really three general approaches: 1. Run Galaxy via CloudMan 100% on AWS. This option is most robust and basically r

Re: [galaxy-dev] ProFTPd in CloudMan

Thanks for reporting this (and figuring out the solution). It's been fixed with this commit so any new cluster will work fine. Cheers. On Tue, Sep 1, 2015 at 7:42 PM, Alexander Vowinkel < vowinkel.alexan..

Re: [galaxy-dev] CloudMan launch error

Hi Aaron, Does the "AMQP Connection Failure:" error continue indefinitely? It would be helpful to see the log files from the time a worker is being added as well as the CloudMan logs from a worker node. It appears you're launching GVL v3.04 however, the default is now 4.0.0. Have you tried keeping

Re: [galaxy-dev] Cloudman vs StarCluster

Hi Ryan, Sorry you're having trouble porting it over. Let us know if you have specific questions that we may be able to help with. In the mean time, here are some of the benefits of CloudMan: technically, CloudMan provides a management interface tailored for Galaxy so you can control the Galaxy pr

Re: [galaxy-dev] Current state of cloudman etc. within private AWS VPCs?

Hi Chris, I believe you're the first person that's tried to run this in a completely private VPC so I don't have experience with it but can suggest a change that might get over this initial hump at least. My guess is that get_galaxy_dns method is why you're seeing this log message ( https://github

Re: [galaxy-dev] Galaxy on Azure

Hi Alex, Assuming you'd want to set it up as a dedicated server on a VM running on Azure, I'd suggest taking a look at the production instance setup notes: https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer Other admin topics are also available on the wiki: https://wiki.galax

Re: [galaxy-dev] Cloudman & AWS IAM_PROFILE support?

Hi Ryan, I'm not sure I really understand your question but CloudMan can run under an IAM account - it is just necessary to give the IAM user permissions to create EC2 and S3 resources. Let us know if that's not what you had in mind or if you have any more questions. Cheers, Enis On Mon, Mar 14,

Re: [galaxy-dev] Cloudman & AWS IAM_PROFILE support?

I'd be > interested in learning your methods. Thanks! > > Regards, > Chris > > > Enis Afgan wrote: > >> Hi Ryan, >> I'm not sure I really understand your question but CloudMan can run under >> an IAM account - it is just necessary to give the IAM u

Re: [galaxy-dev] Cloudman & AWS IAM_PROFILE support?

Dagdigian wrote: > > > > > > Hi Ryan, > > > > I've never been able to get cloudman to function within a private VPC so > if this is your architecture as well and you manage to make progress I'd be > interested in learning your methods. Thanks! >

Re: [galaxy-dev] Cloudman & AWS IAM_PROFILE support?

Ryan; Chris: I just added the ability to launch instances into non-default VPCs to CloudLaunch. This is live on https://launch.usegalaxy.org/, under Advanced startup options. Feel free to give it a shot and let me know if you encounter any issues. On Thu, Mar 17, 2016 at 10:09 AM, Enis Afgan

[galaxy-dev] Galaxy CloudMan with Galaxy 16.01

Hello! *We have released a new version of Galaxy CloudMan on AWS.* CloudMan offers an easy way to get a personal and completely functional instance of Galaxy in the cloud in just a few minutes, without any manual configuration or imposed quotas. Once runn

Re: [galaxy-dev] ERROR: PIP is not a legl parament of an Ansible Play

Hi Bill, Do you have the value set in Galaxy's *$GALAXY_ROOT/config/galaxy.ini* file for setting *tool_config_file *option? I believe this variable should be set to a file other than the default **.sample*. On Wed, Apr 6, 2016 at 9:21 PM, William Ge wrote: > Hi, Marius, > > > > I got ansible run

Re: [galaxy-dev] ERROR: PIP is not a legl parament of an Ansible Play

evel above the Galaxy subdirectories: i.e., all > subdirectories (config, database, tool-data, tools, static, etc) are in > /home/galaxy/galaxy, while “shed_tools” is in /home/galaxy. Do not know why. > > > > Thanks, > > Bill > > > > *From:* Enis Afgan [mailto:enis.af..

Re: [galaxy-dev] Parallelism (job splitting) for ncbi_blast_plus running through CloudMan

> > > Now we have what may be a CloudMan question: our working cluster has a > head node and 4 workers. The head node is loaded up but the workers are > idle. I would have thought jobs should be pushing out to the workers but we > don't see any load on these machines. > > So are the jobs queued jus

Re: [galaxy-dev] Amazon Instance Types for Galaxy

Hi David, Yes there are: t2 class it typically a bit small, particularly t2.small and smaller. m4 and c4 classes work fine but require use of VPC (meaning you need to select a subnet to launch into from the advanced startup options on https://launch.usegalaxy.org/). I haven't tried all the instance

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

Hi David, I totally missed this email from you - apologies! Are you still trying to do this or did you figure it out? Following the general guidelines on customizing the instance is a good place to start. There is an option in CloudMan to turn on non-signed SSL (from the Admin page) but the steps p

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

s the optimal way to move to > 16.07. Wait for the AMI or update? > -- > David Lapointe Ph.D. > Sr. Bioinformatics Specialist > Research Technology (RT) > Tufts Technology Services (TTS) > 16 Dearborn Road > Somerville MA 02144 > > Phone: 617-627-5319 > Fax: 617-6

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

TS) > 16 Dearborn Road > Somerville MA 02144 > > Phone: 617-627-5319 > Fax: 617-627-3667 > http://it.tufts.edu > > From: Enis Afgan > Date: Thursday, September 15, 2016 at 5:30 PM > > To: David Lapointe > Cc: "galaxy-dev@lists.galaxyproject.org" gala

Re: [galaxy-dev] how to make an exact clone of the usegalaxy.org

Hi Rui, You can also take a look at this repo https://github.com/ ARTbio/GalaxyKickStart, and in particular this bit https://github.com/ARTbio/GalaxyKickStart/tree/master/ extra-files/cloud_setup. We used this to build standalone Galaxy images on the Jetstream and Google clouds with a toolset that

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

; 16 Dearborn Road > Somerville MA 02144 > > Phone: 617-627-5319 > Fax: 617-627-3667 > http://it.tufts.edu > > From: galaxy-dev on behalf > of David Lapointe > Date: Friday, September 16, 2016 at 9:24 AM > To: Enis Afgan > > Cc: "galaxy-dev@lists.galaxyproj

Re: [galaxy-dev] Galaxy+Slurm (with elastic cluster) error: "Job output not returned from cluster"

Hi Marco, You could do something similar to what we're doing with CloudMan: add a placeholder node to Slurm. This will make Slurm accept a job but not run it until a node is added that is actually capable of executing it. The placeholder node is just a definition in slurm.conf that has state=future

Re: [galaxy-dev] Galaxy on the Cloud with Openstack

Hi Katherine, A good starting point is the official documentation available here https://wiki.galaxyproject.org/CloudMan/Building. It links to a few more technical README's from there. Did you go through this? Were there issues you encountered? Overall, the complete deployment is a reasonably invol

Re: [galaxy-dev] Galaxy on the Cloud with Openstack

--limit @/home/ubuntu/galaxy-cloudman-playbook/image.retry > > > Do you have an idea of what could be the cause of this error? > > Thanks for the help! > > On Mon, Nov 14, 2016 at 11:58 AM, Enis Afgan wrote: > >> Hi Katherine, >> A good starting point is the offic

[galaxy-dev] GalaxyAdmins Feb online meetup

Hello everyone, We are going to have a GalaxyAdmins call this month! This is a wonderful venue to get an update on what's going on in the Galaxy community as mostly relevant to people managing Galaxy instance(s). The talk will be held at the 'standard' GalaxyAdmins time: on Feb 16th at 11AM EST.

[galaxy-dev] GalaxyAdmins online meetup: this Thursday

This is a reminder that we are going to have a GalaxyAdmins call really soon now! The talk will be held on Thursday Feb 16th at 11AM EST. See your local time here . with the following topic being presented: CloudLaunch is an applicatio

[galaxy-dev] GalaxyAdmins online meetup: call details

So you don't miss a single email in your inbox about the GalaxyAdmins call, I'm sending the second one. This one has the necessary call details info: The meetup's online conference room will open 15 minutes before the meetup. The call will use Johns Hopkins' Adobe Connect server

[galaxy-dev] GalaxyAdmins meetup starting soon

Hello again, This month's GalaxyAdmins call with start in just a bit over an hour. We will be taking a look at the all-new CloudLaunch application ( beta.launch.usegalaxy.org) with a live demo and an overview of the backend architecture. The call will be held using an online conference room

Re: [galaxy-dev] ansible-galaxy-tools and venv

Hi Matthias, I'm not sure I understand what you're suggesting here, if anything? Granted, I haven't used software collections but this problem feels like it has more to do with it, where the libpython is not installed as part of the Python install there. Perhaps a manual install of the library with

Re: [galaxy-dev] Deploy on cloud

Hello Stavros, Galaxy shells jobs to a job variety of job managers for which interfaces have been implemented. Traditionally, this has been the DRMAA-compatible job schedulers but as of about a year ago support for Kubernetes has been added. This is the only container cluster manager that Galaxy na

Re: [galaxy-dev] Disk organization for Jetstream Galaxy instance

Hi David, Have you seen this bit of documentation about setting up Galaxy to use an external volume on Jetstream: https://galaxyproject.org/cloud/jetstream/storage/ That's the most straightforward way at the moment. Changing all the configs to use the volume would me more involved, particularly bec

Re: [galaxy-dev] Galaxy cloud azur

Hi Mohamed, We are working on making Galaxy CloudMan available on Azure but it will still take us a few months. If you are interested in helping out, let me know. Otherwise, setting up Galaxy on a dedicated instance running on Azure will be no different than setting it up on a VM. You can take a l

Re: [galaxy-dev] AWS launch error

Hi Sijung, Sorry to hear about the trouble you're having. However, I'm not able to replicate this issue (across 2 computers and 3 browsers) so can you please share a bit more info from your side. To start, what operating system and browser are you using? Also, can you open the Inspect console in th

Re: [galaxy-dev] AWS launch error

om "Galaxy-dev Digest Vol ..." >> to the original subject for the thread. >> >> 2. Strip out everything else in the digest that is not part of the thread >> you are responding to. >> >> >> Why? >> &

Re: [galaxy-dev] TR: Close cluster

Hi Arnaud, How long after you shut it down did you check? It will take several minutes for the cluster to complete the shutdown procedure. One thing that may lead to taking it much longer is if there is a logged in user that has an open shell under /mnt/galaxy directory. Either way, clicking on th

[galaxy-dev] 2022 Galaxy Community Conference (GCC2022): July 17-23 in Minneapolis, MN, USA

2022 Galaxy Community Conference (GCC2022) July 17-23, Minneapolis, Minnesota, USA https://galaxyproject.org/events/gcc2022/ Deadlines: April 12: Oral presentation and lightning talk abstracts due