Yeah, also that is is pretty much how CloudMan handles
the infrastructure management now - it has it's own descriptor (i.e.,
template) and from there it handles all the provisioning and coordination of
the underlying resources. Granted, having Amazon do it would add to the
robustness of the
Hi Curtis,
setup.sh has been removed from the installation process, as discussed in
this thread:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-March/004588.html
On Wed, Mar 9, 2011 at 2:12 PM, Curt Palm cp...@stanford.edu wrote:
I just cloned the current release version of galaxy:
hg
Hi Clare,
Once you have a VM setup and accessible via ssh, you can also use our
scripts for automated configuration and deployment of dependencies and
tools. These scripts are used to setup Galaxy Cloud and they're targeted for
Ubuntu 10.04 but should be applicable to other distributions as well.
A wiki page is available here: http://wiki.g2.bx.psu.edu/GCC2011
On Mon, May 30, 2011 at 4:20 AM, Martin Senger martin.sen...@gmail.comwrote:
I really enjoyed the recent conference. Many thanks for the splendid
logistics and, of course, the content. I wonder if the slides presented
there will
Hi Ravi,
Just a quick check - did you restart Galaxy after saving the changes? What
you did should direct any new files to the specified location.
Enis
On Wed, Jun 29, 2011 at 10:43 AM, Sanka, Ravi rsa...@jcvi.org wrote:
Greetings,
** **
My name is Ravi Sanka. I have a question
---
** **
*From:* Enis Afgan [mailto:eaf...@emory.edu]
*Sent:* Wednesday, June 29, 2011 11:32 AM
*To:* Sanka, Ravi
*Cc:* galaxy-dev@lists.bx.psu.edu
*Subject:* Re: [galaxy-dev] changing file storage location
** **
Hi Ravi,
Just
?
** **
---
Ravi Sanka
ICS - Bioinformatics Engineer
J. Craig Venter Institute
301-795-7743
---
** **
*From:* Enis Afgan [mailto:eaf...@emory.edu]
*Sent:* Wednesday, June 29
Hi Mattias,
PyCogent is not currently being installed as part of mi-deployment, so the
method for installing it should be added (if you do so, please consider
issuing a pull request on bitbucket so it can be added to the project).
When it comes to composing $PYTHONPATH to include the reference to
to set the PYTHONPATH for Fabric' run() command?
Thanks,
Mattias
On Thu, 2011-07-14 at 09:05 +, Enis Afgan wrote:
Hi Mattias,
PyCogent is not currently being installed as part of mi-deployment, so
the method for installing it should be added (if you do so, please
consider issuing
Hi Joe,
mi-deployment scripts are currently being used to create cloud images so
this is the correct code your looking at. However, when it comes to R and
rpy, those tools are (as you discovered) not being installed via those
scripts; they are installed from packages but, in the case of Galaxy
Hargitai
joseph.hargi...@einstein.yu.edu wrote:
Hi,
so for local install i can uncomment the R and rpy from your script as far
as at the _install_R.. part?
j
--
*From:* Enis Afgan [eaf...@emory.edu]
*Sent:* Monday, August 01, 2011 12:37 PM
*To:* Joseph Hargitai
dependencies
b - installs tools
c - creates indexes
d - points tools to indexes
j
--
*From:* Enis Afgan [eaf...@emory.edu]
*Sent:* Monday, August 01, 2011 12:54 PM
*To:* Joseph Hargitai
*Cc:* Galaxy Dev
*Subject:* Re: [galaxy-user] mi-tools
Hi Mattias,
We use a combination of mi-deployment and Brad's cloudbiolinux script (
https://github.com/chapmanb/cloudbiolinux) to install tools (including the
biolinux ones). It requires some manual effort but has worked for us. The
approach you describe sounds pretty good for packaged tools, I
I forgot to mention earlier that you'll still need to do the database GRANT
etc. as described on the wiki page you pointed to. The mentioned method does
not do that.
Enis
On Tue, Sep 13, 2011 at 8:10 AM, Enis Afgan eaf...@emory.edu wrote:
You can try extracting the ProFTP install method from
: unexpected operator
[: 359: 11: unexpected operator
[: 359: 11: unexpected operator
error resolving local host: can't resolve host name (h_errno =
HOST_NOT_FOUND)
j
--
*From:* Enis Afgan [afg...@gmail.com]
*Sent:* Tuesday, September 13, 2011 4:20 AM
*To:* Joseph
defaults
0 0
/dev/sdb/mntautodefaults,nobootwait,comment=cloudconfig0
0
--
*From:* Enis Afgan [eaf...@emory.edu]
*Sent:* Thursday, September 22, 2011 8:48 AM
*To:* Joseph Hargitai
*Cc:* galaxy-dev@lists.bx.psu.edu
*Subject:* Re
Hi Andrey,
CloudMan does not currently support eucalyptus but I am looking into it now
so the support is coming (sorry but I don't have a timeline yet, still
working on setting up the infrastructure).
Enis
On Thu, Oct 6, 2011 at 5:13 PM, Andrey Tovchigrechko atovt...@jcvi.orgwrote:
We want to
Hi Rodrigo,
I'm currently working on support for Eucalyptus clouds, so if that's the
cloud infrastructure you have setup locally, support will be available soon.
However, it is not working yet.
Enis
2011/10/10 Rodrigo García Herrera rgar...@inmegen.gob.mx
Hello!
If I setup a private cloud
On Fri, Oct 28, 2011 at 5:50 PM, Cittaro Davide cittaro.dav...@hsr.itwrote:
On Oct 28, 2011, at 5:35 PM, James Taylor wrote:
It currently uses a tiny amount of S3 storage just to save configuration
information about your instance.
Ok.. never used AWS, actually, I didn't know S3 holds the
Hi Clare,
As a technical add-on - you didn't mention what type OS you're looking to
install this on so I just wanted to mention that mi-deployment scripts are
tested primarily on Ubuntu 10.04 LTS and thus most likely to work properly
there. If you're also looking to setup the data, unfortunately
Hi Marcel,
It would be best to use 'galaxy' user to add any tools. To do so, after
you've logged in as ubuntu user, simply execute:
sudo su galaxy
and you will become galaxy user. You can then make the desired
modifications.
Good luck,
Enis
On Wed, Nov 30, 2011 at 3:42 PM, Greg Von Kuster
PM, Enis Afgan eaf...@emory.edu wrote:
Hi Clare,
I don't recall what instance type we used earlier, but I think an Extra
Large Instance is going to be fine. Do note that the master node is also
being used to run jobs. However, if it's loaded by just the web server,
SGE
will typically just
Unfortunately not. When removing nodes, CloudMan chooses from the nodes
that are currently not being used but among those the choice is random.
You can manually terminate a particular worker instance from the AWS
console and thus remove it from your cluster by force. CloudMan will then
reconfigure
Regards,
Marcel
On 11/30/11 3:57 PM, Enis Afgan wrote:
Hi Marcel,
It would be best to use 'galaxy' user to add any tools. To do so, after
you've logged in as ubuntu user, simply execute:
sudo su galaxy
and you will become galaxy user. You can then make the desired
modifications
from there.
Enis
Cheers,
Marcel
On 12/1/11 10:45 AM, Enis Afgan wrote:
Hi Marcel,
However, when I create an AMI, terminate the cluster and create a new
cluster using the new AMI, both /mnt/galaxyData and /mnt/galaxyTools do
not
exist anymore, i.e. /dev/sdg3 and /dev/sdg4
Hi Marcel,
On Thu, Dec 1, 2011 at 10:35 PM, Marcel Schumann
schum...@informatik.uni-tuebingen.de wrote:
Hi Brad, hi Enis,
ok, so one possible way should thus be to use the _cleanup_ec2() function
before creating an AMI via the amazon console and the second option perhaps
is this
are correct. I have an existing cluster (that had the old 2
GB volume size) that I'm trying to expand. With additional tools and log
files, that volume keeps getting full.
Can you help guide me through this process?
Thanks again,
Dave
On Thu, Feb 16, 2012 at 2:36 PM, Enis Afgan eaf...@emory.edu
Hi Yves,
When you create the LVM file system - are you composing it from the volume
that already contains the data (i.e., directory structure++ created by
CloudMan) and then adding another volume into the LVM or starting with 2
new, clean volumes?
Maybe trying again and not messing with SGE at all
Hi Greg,
On Mon, Mar 19, 2012 at 11:01 AM, Greg Edwards gedwar...@gmail.com wrote:
Hi,
I've got an implementation of some proteomics tools going well in Galaxy
on AWS EC2 under Cloudman. Thanks for the help along the way.
I need to drive the costs down a bit. I'm using an m1.large AMI and
.
Greg E
On Mon, Mar 19, 2012 at 4:49 PM, Enis Afgan eaf...@emory.edu wrote:
Hi Greg,
On Mon, Mar 19, 2012 at 11:01 AM, Greg Edwards gedwar...@gmail.comwrote:
Hi,
I've got an implementation of some proteomics tools going well in Galaxy
on AWS EC2 under Cloudman. Thanks for the help along
Awesome! Also glad to hear your rate of success can be quantified as a 700%
improvement. Amazing! :)
Best,
Enis
On Wed, Mar 28, 2012 at 2:57 PM, Greg Edwards gedwar...@gmail.com wrote:
Enis,
Thanks, your instructions below worked ok and I have reduced my 700GB to 1
GB.
No doubt a pile of
Hi Gordon,
On Sat, Apr 14, 2012 at 8:06 AM, Assaf Gordon gor...@cshl.edu wrote:
Hello,
Great work with the Galaxy on the Cloud - it's the first time I've used it
and it looks very impressive (and I'm not a big cloud fan :) ).
Thank you very much for the comment!
Once I got it working
Hi Mo,
On Wed, Apr 18, 2012 at 4:53 AM, Mohammad Heydarian mheyd...@jhmi.eduwrote:
Hello,
I am new to cloud computing and am trying to use the Galaxy cloudman
service through Amazon to analyse NGS data. I have a couple of questions
regarding data transfer:
If I am running an EC2 instance,
Hi Madeleine,
By design (ie, for your own benefit), there is no way for anyone but
yourself to check on the status of an AWS instance once started. In order
to check what happened and if an instance got started, you should log into
https://console.aws.amazon.com/ec2/home with the same AWS
Jose,
Did you see the 'Share-an-instance' option within CloudMan? It's the green
icon next to the cluster name in the console. It should do exactly what
you're describing.
Cheers,
Enis
On Thu, Jun 14, 2012 at 11:46 AM, Jose Navas josenavasmol...@hotmail.comwrote:
Ok, I will change my XML
Hi Greg,
On my laptop, I run Python 2.7.1 and as much as one can test things
locally, things are working fine. The version on the AMI you mention is
obviously working fine so I'd say that there shouldn't be any surprises
arising from the different Python versions.
Cheers,
Enis
On Fri, Jun 15,
Hi Quang,
We have no immediate plans to extend the toolset of the tools available on
the default image so I'd suggest one of two options to get this tool out:
1. Create a Galaxy Tool wrapper for it and upload it to the Tool Shed at
http://toolshed.g2.bx.psu.edu/
2. Install it on an Galaxy CloudMan
Hi Mark,
You can always confirm the zone you started an instance in by logging in to
the AWS console and, under Instances, find the instance you're wanting to
check. After selecting it, under the instance properties, the zone will be
displayed for the given instance.
The security group and the
Yes there is, the setup is just not documented very well at this point...
Also, are you familiar with the cloud concepts or just looking to use
Galaxy?
The basic approach is to setup a CloudMan (usecloudman.org) machine image
on the local cloud (using the image building process from
Hi Niel,
I'm assuming the data uploaded to a library on a galaxy cloudman instance.
In that case, all the data that's uploaded to a data library (or a history)
goes on an nfs-exported file system and is thus available to all the worker
nodes.
Cheers,
Enis
On Wed, Oct 24, 2012 at 3:51 PM,
Hi Shea,
This is a reasonable feature so we'll add it on the todo list. To make the
search for all untagged resources easier, did you have a list of the ones
that are currently not getting tagged?
Thanks,
Enis
On Tue, Dec 4, 2012 at 7:34 AM, Shea Lovan shea.lo...@lscg.ucsb.edu wrote:
We are
We also have a python library for dealing with the API, the documentation
is available at http://bioblend.readthedocs.org/
The library source code also contains an example very similar to what
you're trying to achieve: imports a workflow, imports data, and runs the
workflow. Here's that script:
as well ? or
worker images can be vanilla OS ?
BR
Zeeshan
On W6-Feb 7, 2013, at 11:50 PM, Enis Afgan wrote:
Hi Zeeshan,
In order to gain from the scalability of the cloud, SGE does need to
run. However, CloudMan sets all that up and manages it going forward.
Enis
) and workers , am i right
?
actually i built one image via https://bitbucket.org/galaxy/cloudman . DO
you suggest that I should built image from CBL tree you mentioned below ?
which wd have both cloudman and galaxy together.
BR
Zeeshan
On W6-Feb 8, 2013, at 10:21 PM, Enis Afgan wrote:
As far
Hi Chun-Yuan,
Sorry to see you're running running into so much trouble. The reality of
the situation is that the Update Galaxy button in CloudMan is currently
broken due to the changes in Galaxy that make the updates a rather manual
process. For the past several weeks, the team has been working on
Hi Rafa,
The .loc files in 'tool-data' directory link the index files with the
actual paths so you need to configure each of the .loc files to match where
the index files are placed. With that, you can put the index files anywhere
you'd like on the file system (preferably outside Galaxy's home
Correct; certain NSG tools require additional configuration for their
indices (http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup)
On Tue, Mar 5, 2013 at 1:07 PM, shenwiyn shenw...@gmail.com wrote:
**
Hi,
I've installed Galaxy in local as what Rafael does.I want to make sure
that the
Yes, this is good suggestion. The good news is that the functionality is
already implemented in the soon-to-be-released version of CloudMan.
On Fri, Jun 14, 2013 at 6:52 PM, Ravpreet Setia
ravpreet.se...@oicr.on.cawrote:
Initially only one volume is shown on Amazon's EC2 management console.
Hi Teresa,
It's not available on the curent release but we are finalizing a new
release that will make addition of tools very straightforward so, if not
included by default, it will be easy to do.
Thanks for using Galaxy.
On Sun, Jun 16, 2013 at 4:25 PM, Murphy, Theresa
*Last night, we released an update to Galaxy CloudMan.* CloudMan offers an
easy way to get a personal and completely functional instance of Galaxy in
the cloud in just a few minutes, without any manual configuration.
*IMPORTANT - please read*
Any new cluster will automatically start using this
Is this a migrated cluster or a brand new one? I'm guessing you're using
the new AMI?
The migration process supposed to have updated those env.sh paths.
More to the point, is your tophat job failing due to bowtie not being
found? Try adding the path to bowtie executables to tophat's env.sh file
Hi Greg,
Sorry for replying really late.
So, I'm guessing this was an old cluster that was shared and is now being
derived on a new cluster? There was a large number of paths we explored
while getting ready for the upgrade and I was of the opinion we covered
that path but it seems things are not
...@jhmi.edu
Date: Fri, 12 Jul 2013 14:21:23 -0400
To: Microsoft Office User ravpreet.se...@oicr.on.ca
Cc: Enis Afgan afg...@gmail.com, Galaxy-dev galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy CloudMan release - Tophat does not work
Cufflinks, Cuffcompare, and Cuffdiff are also failing
FTP issues are fixed now so things will be functional out of the box
without any of these workarounds.
On Tue, Jul 16, 2013 at 4:04 PM, Daniel Blankenberg d...@bx.psu.edu wrote:
Hey Mo,
You can use ssh to connect to the Galaxy machine. If you used cloudlaunch
to create your instance, it
Not quite next week, but an update to the tools volume and CloudMan itself
was just released so things should be working now. Sorry for the trouble
and the delay and let us know if how things are going for you now.
Cheers,
Enis
On Sat, Jul 13, 2013 at 9:27 AM, Enis Afgan afg...@gmail.com wrote
Hi Mo,
Just like in the email I just sent, the new release should fix this. Let us
know if you have any more issues.
On Tue, Jul 16, 2013 at 6:29 PM, Dannon Baker dannon.ba...@gmail.comwrote:
Hey Mo,
The new volume we pushed out for the conference has several known issues.
Enis and I are
Hi Vipin,
It seems there are too many volumes (or too much volume storage) in your
AWS account and AWS is preventing any more from being created. You can
either delete some volumes if you don't need them or send a request to AWS
to increase your volume limit:
Hello,
Galaxy workflow and history structure is stored in Galaxy's database vs.
flat files so you can't just find the representation of those on the file
system.
I actually don't know if there is any documentation about the JSON
structure for those artifacts - I've CC'd the -dev mailing list so
Someone else might have but we have not (the same argument still holds).
Please let us know if you go down that path,
Enis
On Fri, Apr 25, 2014 at 12:35 PM, Luca Toldo lucato...@gmail.com wrote:
dear Galaxians,
AWS CloudFormation is now mature, and I have interest in exploiting it for
*We just released an update to Galaxy CloudMan.* CloudMan offers an easy
way to get a personal and completely functional instance of Galaxy in the
cloud in just a few minutes, without any manual configuration.
This is mostly an incremental bug fix release with the following summary of
changes:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/
I've just tried it and worked fine.
Sorry for a delay in getting back to you,
Enis
On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker jmccus...@5amsolutions.com
wrote:
I have a galaxy/cloudman cluster where
, Aug 14, 2014 at 2:09 PM, Enis Afgan afg...@gmail.com wrote:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/
I've just tried it and worked fine.
Sorry for a delay in getting back to you,
Enis
On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker jmccus
to the default arguments?
On Thu, Aug 14, 2014 at 2:52 PM, Enis Afgan afg...@gmail.com wrote:
If I'm understanding your question correctly (ie, have the ability to
specify SGE requirements on a per-tool basis), then yes. You'd do something
like this in universe_wsgi.ini:
[galaxy:tool_runners
Hi Inge,
There is an implementation for using the AWS S3 object store as the data
store for a given Galaxy instance. The implementation is located here
https://bitbucket.org/galaxy/galaxy-central/src/3a51eaf209f2502bf32dbb421ecabb7fe46243ea/lib/galaxy/objectstore/s3.py?at=default
and it offers
Hello Anne,
Overall, what we typically recommend is to launch a new instance and
configure it as desired (eg, with tools, histories, and datasets of
interest for the workshop). Then share that instance (using the CloudMan
share-a-cluster feature
http://www.biomedcentral.com/1471-2105/13/315 -
Hi Iry,
Try adding the following to your
*/mnt/galaxy/galaxy-app/tool_data_table_conf.xml*, populating the
referenced files (tool-data/gatk2_picard_index.loc and
tool-data/gatk2_annotations.txt) as desired and restarting Galaxy:
!-- Location of Picard dict files valid for GATK --
table
.
From: Enis Afgan afg...@gmail.com
Date: Saturday, October 4, 2014 6:10 AM
To: Iry Witham iry.wit...@jax.org
Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Cloudman indices installation/configuration
Hi Iry,
Try adding the following to your
*/mnt
Hi Robert,
Can you check on CloudMan's Admin page if the following is shown (towards
the bottom of the page): Switch master to not run jobs
If it says, Switch master to run jobs, just click on the text and the
master will become an execution host on the cluster, which should then give
Galaxy some
there has to be
at least one execution host (either master or always on worker) auto
scaling to work?
Thanks for the help,
Robert
--
*From:* Enis Afgan [afg...@gmail.com]
*Sent:* Thursday, October 09, 2014 11:30 AM
*To:* Petit III, Robert A.
*Cc:* galaxy-dev
Changing the default job manager is not interchangeable in an automated
fashion. SGE is currently the main job manager that Galaxy has been
configured to be used with. Slurm will replace SGE in the next Galaxy
CloudMan release. HTCondor has been added as an experimental module but due
to the lack
No it is not. The thinking was that autoscaling should be responsive and
spot instance are not necessarily predictable so we decided against it. We
can revisit that idea though if you're keen?
Cheers,
Enis
On Wed, Oct 15, 2014 at 11:05 AM, Petit III, Robert A.
robert.pe...@emory.edu wrote:
As Hans said, this might reguire a change in the proxy server configuration
as discussed here:
http://gmod.827538.n3.nabble.com/Data-uploaded-with-new-upload-tool-doesn-t-get-added-to-history-td4046375.html
On Fri, Oct 17, 2014 at 2:43 AM, Hans-Rudolf Hotz h...@fmi.ch wrote:
Hi Bongsoo
we had
Hi Jim,
How did you launch your instance?
If you still have the instance up, could you please also forward the first
~200 lines of /mnt/cm/paster.log (deleting any identifiable info from the
log).
Sorry abou the trouble and thanks for reporting it,
Enis
On Sat, Oct 25, 2014 at 8:58 PM, Jim
Hi Denise,
Although the instructions you found can still be used, a couple of steps
can be updated now so I'll to that now and hopefully answer your questions
in the process.
Updated instructions:
1. Start a brand new Galaxy CloudMan cluster/instance (this will be only a
temporary cluster needed
74 matches
Mail list logo
