[galaxy-dev] 2022 Galaxy Community Conference (GCC2022): July 17-23 in Minneapolis, MN, USA

[Enis Afgan]

2022 Galaxy Community Conference (GCC2022)

July 17-23, Minneapolis, Minnesota, USA

https://galaxyproject.org/events/gcc2022/

Deadlines:

April 12: Oral presentation and lightning talk abstracts due


May 12: Early registration

ends

June 3: Demo and poster abstracts due


The 2022 Galaxy Community Conference (GCC2022)
 will be held July 17-23 in
Minneapolis,
Minnesota. GCC brings together hundreds of faculty, clinicians,
researchers, and students, all working in and supporting data intensive
science that is accessible, shareable, and reproducible.

GCC2022 features oral presentations, lightning talks, posters, demos,
birds-of-a-feather gatherings (BoFs), training, a CollaborationFest, and
plenty of opportunities for networking.

Presentations will cover the full spectrum of Galaxy applications,
enhancements, and deployments. If you are working in data intensive science
then GCC2022 is an ideal conference for sharing your work, learning from
others, and finding new collaborators.

Present your work!

Abstract submission

for talks, lightning talks, demos, and posters is now open. If you work in
data-intensive science then please consider presenting your work at
GCC2022. This is a chance to present to 200+ researchers, all addressing
challenges in data intensive science. Abstract submissions for oral
presentations are due on April 12. Submit an abstract (or two) now!



Topics of Interest

If you’re currently working on any of these topics, this conference is for
you:

   -

   Omics data analysis: genomic sequencing, microbiome, proteomics,
   imaging, you name it!
   -

   Tools and visualizations for working with omics data
   -

   Compute infrastructure for enabling data intensive science
   -

   Galaxy applications in any domain or topic: ML, climate, NPL, …
   -

   Outreach and training related to data analysis


Registration

Registration information is available on the conference website
.  Early
registration ends May 12. So, register
 early!

About Galaxy

Galaxy  is a platform for data integration and
analysis in the life sciences. It enables researchers to build, run, share,
and repeat their own complex computational analyses using only a web
browser and without having to first learn system administration and command
line interfaces.

The Galaxy Project is driven by a vibrant community who publish workflows
and analyses, wrap new tools, maintain and enhance the source code, provide
support, and write documentation and training materials. Galaxy is
open-source and freely available, and is deployed in hundreds of
organizations, running on everything from laptops to supercomputers to
public and private clouds. Over 150 of these platforms are publicly
available and can be used with little or no setup. Thousands of tools have
been ported to Galaxy and are deployable from the Galaxy Tool Shed. Galaxy
was developed to support life science research, but the platform is domain
agnostic and is now used in domains as diverse as natural language
processing, computational solid geometry, and social science.

Hope to see you in Minneapolis!

GCC2022 Organizers 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] TR: Close cluster

[Enis Afgan]

Hi Arnaud,
How long after you shut it down did you check? It will take several minutes
for the cluster to complete the shutdown procedure. One thing that may lead
to taking it much longer is if there is a logged in user that has an open
shell under /mnt/galaxy directory.

Either way, clicking on the link you sent leads to a timeout so it looks
like you deleted it by now.

Ultimately, you can log into the AWS console and delete the VM instance
from there.

On Wed, Nov 29, 2017 at 8:40 AM, Arnaud Cheuk 
wrote:

>
>
> Hi,
>
> I would like to close my cluster, but when click on delete permanently, I
> see the cluster is still active. Could you help me to close the cluster?
>
>
> *cduanra-galaxy-cloud-17-11-28t22-14*
>
> Appliance: Galaxy CloudMan Version: 17.05
> Cloud: amazon-us-east-n-virginia
>
> RUNNING
> http://52.202.138.61/cloud
>
> Best,
>
> Arnaud
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] AWS launch error

[Enis Afgan]

Hi David,
Do you perhaps have credentials stored on the beta server? If not, and
don't mind storing them, you can try that. I'm not having an issue at the
moment but I know Nuwan has done some changes recently that I did not catch
up with yet. CC'ing him in case he has additional input.

On Tue, Nov 7, 2017 at 12:53 PM, David Kovalic <kova...@analome.com> wrote:

> Enis,
>
> I am having issues launching previously existing clusters today, here is
> some more info:
>
>- I go to https://launch.usegalaxy.org/launch
>- I fill in the required AWS credentials
>- I click on "Choose a saved cluster"
>- the page goes into a loop trying to retrieve the 'Saved Cluster'
>information to populate  the drop down list
>- I can terminate this loop by clicking on the "x"
>- I can enter the cluster name manually
>- The cluster bucket entered does exist on my S3
>- After entering the cluster name manually I click "Create a cluster"
>- The next page is shown "Please wait, your scalable cluster is
>starting on the Amazon - Virginia cloud."
>- I see no instance stared on the AWS console
>
> I have similar issues on the new page "https://beta.launch.usegalaxy.org/;,
> on this page my previous clusters are not shown.
>
> Nothing has changed on my end, to my knoledge… Any ideas? Need any more
> information from my end?
>
> lmk and thanks,
>
> David
>
>
>
> David Kovalic Ph.D.   President, Analome
> phone: +314.496.8808
> web: www.analome.com
> email: kova...@analome.com
> <https://www.linkedin.com/in/davidkovalic>
>
>
>
> On Fri, Nov 3, 2017 11:00 AM, galaxy-dev-requ...@lists.galaxyproject.org
> wrote:
>
>> Send galaxy-dev mailing list submissions to
>>
>> galaxy-dev@lists.galaxyproject.org
>>
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>
>> https://lists.galaxyproject.org/listinfo/galaxy-dev
>>
>> or, via email, send a message with subject or body 'help' to
>>
>> galaxy-dev-requ...@lists.galaxyproject.org
>>
>>
>> You can reach the person managing the list at
>>
>> galaxy-dev-ow...@lists.galaxyproject.org
>>
>>
>> When replying, please edit your Subject line so it is more specific
>>
>> than "Re: Contents of galaxy-dev digest..."
>>
>> HEY! This is important! If you reply to a thread in a digest, please
>>
>> 1. Change the subject of your response from "Galaxy-dev Digest Vol ..."
>> to the original subject for the thread.
>>
>> 2. Strip out everything else in the digest that is not part of the thread
>> you are responding to.
>>
>>
>> Why?
>>
>> 1. This will keep the subject meaningful. People will have some idea from
>> the subject line if they should read it or not.
>>
>> 2. Not doing this greatly increases the number of emails that match
>> search queries, but that aren't actually informative.Today's Topics:
>>
>>
>> 1. Re: AWS launch error (Enis Afgan)
>>
>> 2. Enable viewing of large HTML files (Peter Briggs)
>>
>> 3. Re: Enable viewing of large HTML files (Martin Čech)
>>
>> ___
>>
>> galaxy-dev mailing list
>>
>> galaxy-dev@lists.galaxyproject.org
>>
>> https://lists.galaxyproject.org/listinfo/galaxy-dev
>>
>>
>> To search Galaxy mailing lists use the unified search at:
>>
>> http://galaxyproject.org/search/mailinglists/
>>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Disk organization for Jetstream Galaxy instance

[Enis Afgan]

Hi David,
Have you seen this bit of documentation about setting up Galaxy to use an
external volume on Jetstream:
https://galaxyproject.org/cloud/jetstream/storage/
That's the most straightforward way at the moment. Changing all the configs
to use the volume would me more involved, particularly because of the
database and how it gets installed...

On Thu, Mar 9, 2017 at 4:13 PM, David Kovalic  wrote:

> Hi,
>
> When  we launch a Galaxy instance on Jetstream, the galaxy user's home dir
> is set up on the root volume, this has inherent limitations. Is there a
> good way to change the home dir location to an attached disk?
>
> Thanks,
>
> David
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Deploy on cloud

[Enis Afgan]

Hello Stavros,
Galaxy shells jobs to a job variety of job managers for which interfaces
have been implemented. Traditionally, this has been the DRMAA-compatible
job schedulers but as of about a year ago support for Kubernetes has been
added. This is the only container cluster manager that Galaxy natively
supports. However, my understanding is that Kubernetes (or other similar
container managers) are not really ready to take on short-running jobs like
the traditional batch clusters - but please correct me if you know
differently. Either way, take a look at this pull request where support for
Kubernetes was added for more info:
https://github.com/galaxyproject/galaxy/pull/2314

In addition, not sure if you're aware of this, but Galaxy has a cloud
manager that helps it to run in the cloud:
https://new.galaxyproject.org/cloudman/ While the current version of the
app was developed with Amazon in mind, we are currently in the process of
redoing our entire cloud stack so it becomes cloud-agnostic and
container-based. We have implemented a cloud abstraction layer in a library
called CloudBridge (http://cloudbridge.readthedocs.io/), a new CloudLaunch
that allows users to self-provision Galaxy instances (
https://beta.launch.usegalaxy.org/), and are starting work on a new version
of CloudMan that will allow the entire Galaxy service system to run in
containers as well as manage container-based jobs. We are just starting
on the implementation part of this so if you have interest in the topic and
would like to participate, please let me know.

Hope this helps,
Enis

On Fri, Mar 3, 2017 at 7:00 AM, Stavros 0 Sachtouris 
wrote:

> Hi all,
>
> we are going to deploy galaxy on ~okeanos cloud ( okeanos.grnet.gr ). We
> have already developed a working storage backend for our object store
> system ( https://github.com/galaxyproject/galaxy/pull/3611 ).
>
> Now we focus on how to make Galaxy run instances/tasks on our VMs and I
> would like some advice on how to approach this issue.
>
> What we need is a (potentially large) Galaxy deployment with the ability
> to run workflows of dockerized "tasks". Each task should run on a
> provisioned VM on our cloud. There are many tools out there to let us
> automate VM provisioning, managing docker clusters, scaling, as well as
> monitoring and accounting, and we want to provide all of these features to
> our partners in European Academia.
>
> The question is, how does Galaxy communicate with a cluster manager like
> Apache Mesos or Docker swarm? Any other tools I should look into, which are
> known to work with Galaxy? Will we have to drill into Galaxy code base to
> provide these features?
>
> Thank you in advance for you suggestions,
> Stavros Sachtouris
> developer @ www.grnet.gr
>
>
> - Upozorenje -
>
> Automatskom detekcijom utvrdjeno je da tekst ove poruke
> podsjeca na tzv. phishing poruku.
>
> AKO SE U PORUCI TRAZI DA POSALJETE VASU IRB LOZINKU ILI
> DA UNESETE IRB PODATKE NA NAVEDENOM LINKU, RADI SE O
> NAPADU S CILJEM KRADJE I ZLOUPOTREBE PODATAKA.
>
> Centar za informatiku i racunarstvo,
> Institut Rudjer Boskovic
>
> - Upozorenje -
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ansible-galaxy-tools and venv

[Enis Afgan]

Hi Matthias,
I'm not sure I understand what you're suggesting here, if anything?
Granted, I haven't used software collections but this problem feels like it
has more to do with it, where the libpython is not installed as part of the
Python install there. Perhaps a manual install of the library with the
software collection enabled would fix this (
http://stackoverflow.com/questions/26597527/how-to-install-libpython2-7-so).
The library cannot just be included into requirements.txt for example.

On Mon, Feb 27, 2017 at 2:27 AM, Matthias Bernt  wrote:

> Dear galaxy developers,
>
> Is it expected that the virtualenv in .venv/ does include the python
> binary but does not include the python shared libraries?
>
> Some more details:
>
> I have installed galaxy on CentOS 6.8 which includes the required python
> 2.7 only via software collections. For the installation of galaxy I have
> enabled the software collection by calling "source /opt/rh/python27/enable"
> (which sets PATH, LD_LIBRARY_PATH, MANPATH, XDG_DATA_DIRS, and
> PKG_CONFIG_PATH).
>
> Currently I was trying to run the ansible role ansible-galaxy-tools from
> https://github.com/galaxyproject/ansible-galaxy-tools which runs
> "/home/galaxy/galaxy/.venv/bin/python manage_bootstrap_user.py ..."
> and gives me
> /home/galaxy/galaxy/.venv/bin/python: error while loading shared
> libraries: libpython2.7.so.1.0: cannot open shared object file: No such
> file or directory.
>
> This python call would work if I again "source /opt/rh/python27/enable".
> But then the venv python binary is used with the library of the system.
> Intuitively I would expect that the library and binary belong together.
>
> Cheers,
> Matthias
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] GalaxyAdmins meetup starting soon

[Enis Afgan]

Hello again,
This month's GalaxyAdmins call with start in just a bit over an hour. We
will be taking a look at the all-new CloudLaunch application (
beta.launch.usegalaxy.org) with a live demo and an overview of the backend
architecture.

The call will be held using an online conference room
, which will open 15
minutes before the meetup.

More details about the content and access instructions are available here
https://new.galaxyproject.org/community/galaxy-admins/meetups/2017-02-16/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] GalaxyAdmins online meetup: call details

[Enis Afgan]

So you don't miss a single email in your inbox about the GalaxyAdmins call,
I'm sending the second one. This one has the necessary call details info:

The meetup's online conference room will open 15 minutes before the meetup.
The call will use Johns Hopkins' Adobe Connect server
, which in turn uses Adobe Flash.
You are encouraged to connect a few minutes early to work out any
unexpected bumps.

Complete connection details, including screenshots of the process are
available on the event page:
https://new.galaxyproject.org/community/galaxy-admins/meetups/2017-02-16/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] GalaxyAdmins online meetup: this Thursday

[Enis Afgan]

This is a reminder that we are going to have a GalaxyAdmins call really
soon now!

The talk will be held on Thursday Feb 16th at 11AM EST. See your local time
here .


with the following topic being presented:

CloudLaunch  is an application for
launching Galaxy on the Cloud instances. In this talk, an all-new version
will be presented. The new CloudLaunch (beta.launch.usegalaxy.org) allows
multiple flavors of Galaxy to be deployed, Docker images or applications
other than Galaxy. Each application can have its own custom user interface
developed to accommodate the specific application needs, which will be
demonstrated.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] GalaxyAdmins Feb online meetup

[Enis Afgan]

Hello everyone,
We are going to have a GalaxyAdmins call this month! This is a wonderful
venue to get an update on what's going on in the Galaxy community as mostly
relevant to people managing Galaxy instance(s).

The talk will be held at the 'standard' GalaxyAdmins time: on Feb 16th at
11AM EST.


Here is a short description of the topic that will be presented:

CloudLaunch is an application for launching Galaxy on the Cloud instances.
In this talk, an all-new version will be presented. The new CloudLaunch (
beta.launch.usegalaxy.org) allows multiple flavors of Galaxy to be
deployed, Docker images or applications other than Galaxy. Each application
can have its own custom user interface developed to accommodate the
specific application needs, which will be demonstrated.



The call will be held online, using an online meetup room with Adobe
Connect. The online conference room will open 15 minutes before the meetup.
The call will use Johns Hopkins' Adobe Connect server, which in turn uses
Adobe Flash. You are encouraged to connect a few minutes early to work out
any unexpected bumps.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on the Cloud with Openstack

[Enis Afgan]

When you cloned the playbook repo, did you do it recursively so any
dependent roles get installed?
For example, in my local copy, I have the following value
in roles/galaxyprojectdotorg.cloudman-image/meta/main.yml (under
dependencies):
- { role: smola.java, java_packages: 'oracle-java8-installer' }

What results do you get if you do a global directory search for
java_packages?

Sorry for delayed replies.


On Mon, Nov 14, 2016 at 12:29 PM, Katherine Beaulieu <
katherine.beaulieu...@gmail.com> wrote:

> Hi Enis,
> Yes I did go through this process and got stuck at running the images.yml
> file. I wasn't able to run with packer so am trying it without it. Here is
> the error I got stopped at:
>
> TASK [smola.java : Install Java packages] **
> 
>
> Monday 14 November 2016  17:27:15 + (0:00:00.552)   0:00:01.830
> ***
>
> failed: [localhost] (item=[u'java_packages']) => {"failed": true, "item":
> ["java_packages"], "msg": "No package matching 'java_packages' is
> available"}
>
> to retry, use: --limit @/home/ubuntu/galaxy-cloudman-playbook/image.retry
>
>
> Do you have an idea of what could be the cause of this error?
>
> Thanks for the help!
>
> On Mon, Nov 14, 2016 at 11:58 AM, Enis Afgan <enis.af...@irb.hr> wrote:
>
>> Hi Katherine,
>> A good starting point is the official documentation available here
>> https://wiki.galaxyproject.org/CloudMan/Building. It links to a few more
>> technical README's from there. Did you go through this? Were there issues
>> you encountered?
>> Overall, the complete deployment is a reasonably involved process that is
>> somewhat dependent on the infrastructure internals so it'll take some time
>> and understanding of the process to get it done. Feel free to email with
>> any questions.
>>
>>
>>
>> On Tue, Nov 8, 2016 at 1:31 PM, Katherine Beaulieu <
>> katherine.beaulieu...@gmail.com> wrote:
>>
>>> Hi Galaxy Community,
>>> Does anyone have a really great guide as to how I could get Galaxy on
>>> the Cloud up and running with our private openstack cloud orchestrated by
>>> fuel 7.0?
>>> Cheers,
>>> Katherine
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on the Cloud with Openstack

[Enis Afgan]

Hi Katherine,
A good starting point is the official documentation available here
https://wiki.galaxyproject.org/CloudMan/Building. It links to a few more
technical README's from there. Did you go through this? Were there issues
you encountered?
Overall, the complete deployment is a reasonably involved process that is
somewhat dependent on the infrastructure internals so it'll take some time
and understanding of the process to get it done. Feel free to email with
any questions.



On Tue, Nov 8, 2016 at 1:31 PM, Katherine Beaulieu <
katherine.beaulieu...@gmail.com> wrote:

> Hi Galaxy Community,
> Does anyone have a really great guide as to how I could get Galaxy on the
> Cloud up and running with our private openstack cloud orchestrated by fuel
> 7.0?
> Cheers,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy+Slurm (with elastic cluster) error: "Job output not returned from cluster"

[Enis Afgan]

Hi Marco,
You could do something similar to what we're doing with CloudMan: add a
placeholder node to Slurm. This will make Slurm accept a job but not run it
until a node is added that is actually capable of executing it. The
placeholder node is just a definition in slurm.conf that has state=future.
Check out slurm.conf we're using:
https://github.com/galaxyproject/cloudman/blob/master/cm/conftemplates/slurm.conf.default#L37

Cheers,
Enis

On Tue, Nov 1, 2016 at 1:04 PM, Marco Tangaro  wrote:

> Dear all,
> I've a issue using Galaxy with elastic cluster support.
> It is provided by integrating SLURM and a worker node is added as soon as
> jobs are submitted through the Galaxy portal.
>
> When a job is submitted, the node takes some minutes to be configured.
> After ~7 minutes, Galaxy give me a failure message "Job output not
> returned from cluster".
> On the contrary if the node is already up, everything is ok.
> I tried only a very simple job getting the UCSC human genome.
> I'm using the master galaxy branch with postgresq+nginx+uwsgi+proftpd.
>
> Here is my job_conf.xml configuration:
> https://gist.github.com/mtangaro/c0528c3d9a7b44b3bab35dbd947f2c81
>
> I'm not a slurm expert. I went through the mailing list archive, but I did
> not solved the issue.
> Thanks a lot for your help.
> Marco
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

[Enis Afgan]

Changes to the nginx config are probably best done within the CloudMan code
at this point... You can edit this file to include the changes:
https://github.com/galaxyproject/cloudman/blob/master/cm/conftemplates/nginx1.4.conf.default
If you look at the bottom of that file, with the includes, it might be
possible to just create an additional *.conf file that will get included
there. I’d start with that option.

If it becomes necessary to actually edit the original file, take a look at
this section of the docs you pointed to on how to get a cluster to use it:
https://wiki.galaxyproject.org/CloudMan/CustomizeGalaxyCloud#Using_custom_CloudMan_application

The next major version of CloudMan will have better support for config
changes but the way things are looking, we're still ways away from that
version.

On Wed, Sep 28, 2016 at 2:20 PM, Lapointe, David <david.lapoi...@tufts.edu>
wrote:

> As it turns out I will need to use CloudMan. I am a bit confused about how
> to make changes that will persist. For example, making changes to the nginx
> configs to use my SSL certs, and/or changes to setup multiple galaxy
> processes. Do I make changes to the userData.yaml  (CloudMan/UserData page)
> or follow the directions on CloudMan/CustomizeGalaxyCloud where I would
> edit the changes in and then persist using the Cloudman Admin.
>
> David
>
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> From: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> on behalf
> of David Lapointe <david.lapoi...@tufts.edu>
> Date: Friday, September 16, 2016 at 9:24 AM
> To: Enis Afgan <enis.af...@irb.hr>
>
> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.
> galaxyproject.org>
> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>
> Yes that looks good.  The git pull will work on top of the AMI (16.04)?
>  I don't think that I need cloudman to stop/start, just supervisorctl,
> which I have been using to manage galaxy.
>
> Thanks
>
> David
>
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> From: Enis Afgan <enis.af...@irb.hr>
> Date: Thursday, September 15, 2016 at 5:30 PM
> To: David Lapointe <david.lapoi...@tufts.edu>
> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.
> galaxyproject.org>
> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>
> Give you seemingly have a custom deployment, a manual upgrade is probably
> best. Instructions are available here: https://wiki.
> galaxyproject.org/CloudMan/Services/Galaxy#Updating_Galaxy_source
>
> We'll get the 16.07 release done too but juding by the way things look, it
> won't be until the next month.
>
> On Wed, Sep 14, 2016 at 11:23 AM, Lapointe, David <
> david.lapoi...@tufts.edu> wrote:
>
>> Hi Enis,
>>
>> Yes I figured what I was doing wrong. Using Cloudman to restart services
>> overwrote my confs.   Using the CLI to manage services keeps everything in
>> place, though I would like to figure out what is the optimal way to move to
>> 16.07. Wait for the AMI or update?
>> --
>> David Lapointe Ph.D.
>> Sr. Bioinformatics Specialist
>> Research Technology (RT)
>> Tufts Technology Services (TTS)
>> 16 Dearborn Road
>> Somerville MA 02144
>>
>> Phone:  617-627-5319
>> Fax: 617-627-3667
>> http://it.tufts.edu
>>
>> From: Enis Afgan <enis.af...@irb.hr>
>> Date: Wednesday, September 14, 2016 at 11:01 AM
>> To: David Lapointe <david.lapoi...@tufts.edu>
>> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyprojec
>> t.org>
>> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>>
>> Hi David,
>> I totally missed this email from you - apologies! Are you still trying to
>> do this or did you figure it out? Following the general guidelines on
>> customizing the instance is a good place to start. There is an option in
>> CloudMan to turn on non-signed SSL (from the Admin page) but the steps
>> performed may be useful in trying to do the same manually. I believe it's
>> necessary to just update and reload the Nginx configuration. A template can
>> be found here https://github.com/galaxyproject/cloudman/blob/8db478a8
>> d777bcc48ed8c838f1f9de8519a11e60/cm/conftemplates/nginx

Re: [galaxy-dev] how to make an exact clone of the usegalaxy.org

[Enis Afgan]

Hi Rui,
You can also take a look at this repo https://github.com/
ARTbio/GalaxyKickStart, and in particular this bit
https://github.com/ARTbio/GalaxyKickStart/tree/master/
extra-files/cloud_setup.

We used this to build standalone Galaxy images on the Jetstream and Google
clouds with a toolset that's as close to Main as one can get without
debugging a number of failed tool installs. Even though it's primarily used
for a cloud image setup, there is no reason for it cannot be used for a
local machine or a VM.

On Mon, Sep 26, 2016 at 11:06 AM, Martin Čech  wrote:

> The link in the first answer is not up to date because things moved,
> however it should be probably linking to here https://github.com/galaxy
> project/ephemeris/blob/master/ephemeris/shed_install.py
>
> Martin
>
> On Mon, Sep 26, 2016 at 11:02 AM Beginner TI  wrote:
>
>> Hi Martin,
>>
>> Thanks!
>>
>> The first link doesn't really work...upon going to github it gave me 404.
>> After I explored a bit more
>> it says itself that it may not work now. :-(
>>
>> However the second one looks quite promising!
>>
>> Cheers,
>> Rui
>>
>> On Mon, Sep 26, 2016 at 7:55 AM, Martin Čech  wrote:
>>
>>> Hi Rui,
>>>
>>> there is no 'mirror' button to obtain Main's toolset yet, but many
>>> pieces are in place to help you with this.
>>>
>>> Please see the following answers for guides and links:
>>>
>>> https://biostar.usegalaxy.org/p/11338/
>>> https://biostar.usegalaxy.org/p/19660/
>>>
>>> Thanks,
>>>
>>> Martin
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Sep 26, 2016 at 10:50 AM Beginner TI 
>>> wrote:
>>>
 Hey Guys,

 I have seen this question being asked quite some times, but there
 wasn't any definitive answers. Today I'm doing a new local install. Upon
 getting the latest github repo, I still only got a small portion of galaxy
 main. Is there any way to include all the tools there automatically?

 Thanks,
 Rui
 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
>>>
>>>
>>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

[Enis Afgan]

If you have a system that works, I don't see a need to change it. But yes,
it's the Galaxy install is a clone of the repo so git pull will do the
trick.

On Fri, Sep 16, 2016 at 9:24 AM, Lapointe, David <david.lapoi...@tufts.edu>
wrote:

> Yes that looks good.  The git pull will work on top of the AMI (16.04)?
>  I don't think that I need cloudman to stop/start, just supervisorctl,
> which I have been using to manage galaxy.
>
> Thanks
>
> David
>
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> From: Enis Afgan <enis.af...@irb.hr>
> Date: Thursday, September 15, 2016 at 5:30 PM
>
> To: David Lapointe <david.lapoi...@tufts.edu>
> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.
> galaxyproject.org>
> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>
> Give you seemingly have a custom deployment, a manual upgrade is probably
> best. Instructions are available here: https://wiki.
> galaxyproject.org/CloudMan/Services/Galaxy#Updating_Galaxy_source
>
> We'll get the 16.07 release done too but juding by the way things look, it
> won't be until the next month.
>
> On Wed, Sep 14, 2016 at 11:23 AM, Lapointe, David <
> david.lapoi...@tufts.edu> wrote:
>
>> Hi Enis,
>>
>> Yes I figured what I was doing wrong. Using Cloudman to restart services
>> overwrote my confs.   Using the CLI to manage services keeps everything in
>> place, though I would like to figure out what is the optimal way to move to
>> 16.07. Wait for the AMI or update?
>> --
>> David Lapointe Ph.D.
>> Sr. Bioinformatics Specialist
>> Research Technology (RT)
>> Tufts Technology Services (TTS)
>> 16 Dearborn Road
>> Somerville MA 02144
>>
>> Phone:  617-627-5319
>> Fax: 617-627-3667
>> http://it.tufts.edu
>>
>> From: Enis Afgan <enis.af...@irb.hr>
>> Date: Wednesday, September 14, 2016 at 11:01 AM
>> To: David Lapointe <david.lapoi...@tufts.edu>
>> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyprojec
>> t.org>
>> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>>
>> Hi David,
>> I totally missed this email from you - apologies! Are you still trying to
>> do this or did you figure it out? Following the general guidelines on
>> customizing the instance is a good place to start. There is an option in
>> CloudMan to turn on non-signed SSL (from the Admin page) but the steps
>> performed may be useful in trying to do the same manually. I believe it's
>> necessary to just update and reload the Nginx configuration. A template can
>> be found here https://github.com/galaxyproject/cloudman/blob/8db478a8
>> d777bcc48ed8c838f1f9de8519a11e60/cm/conftemplates/nginx_
>> server_ssl.default
>>
>> Hope this helps and, again, sorry for not getting back to you earlier.
>>
>>
>> On Thu, Aug 11, 2016 at 2:30 PM, Lapointe, David <
>> david.lapoi...@tufts.edu> wrote:
>>
>>> This is on Amazon I should add.
>>> --
>>> David Lapointe Ph.D.
>>> Sr. Bioinformatics Specialist
>>> Research Technology (RT)
>>> Tufts Technology Services (TTS)
>>> 16 Dearborn Road
>>> Somerville MA 02144
>>>
>>> Phone:  617-627-5319
>>> Fax: 617-627-3667
>>> http://it.tufts.edu
>>>
>>> From: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> on behalf
>>> of David Lapointe <david.lapoi...@tufts.edu>
>>> Date: Thursday, August 11, 2016 at 1:44 PM
>>> To: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyprojec
>>> t.org>
>>> Subject: [galaxy-dev] Modifying a Cloud Cluster Instance
>>>
>>> I have  a Cloudman instance up and running, but I would like to add SSL
>>> ( I have a signed cert and key ) and productionize it.  I see several
>>> methods documented,but it's not clear which one is preferred: edit in place
>>> then persist changes and restart  (wiki/Cloudman/CustomizeGalaxyCloud)
>>> seems best to me. Any advice?
>>>
>>> Thanks
>>>
>>> David
>>> --
>>> David Lapointe Ph.D.
>>> Sr. Bioinformatics Specialist
>>> Research Technology (RT)
>>> Tufts Technology Services (TTS)
>>> 16 Dearborn Road
>>> Somerville MA 02144
>>>
>>> Phone:  617-627-5319
&g

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

[Enis Afgan]

Give you seemingly have a custom deployment, a manual upgrade is probably
best. Instructions are available here:
https://wiki.galaxyproject.org/CloudMan/Services/Galaxy#Updating_Galaxy_source

We'll get the 16.07 release done too but juding by the way things look, it
won't be until the next month.

On Wed, Sep 14, 2016 at 11:23 AM, Lapointe, David <david.lapoi...@tufts.edu>
wrote:

> Hi Enis,
>
> Yes I figured what I was doing wrong. Using Cloudman to restart services
> overwrote my confs.   Using the CLI to manage services keeps everything in
> place, though I would like to figure out what is the optimal way to move to
> 16.07. Wait for the AMI or update?
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> From: Enis Afgan <enis.af...@irb.hr>
> Date: Wednesday, September 14, 2016 at 11:01 AM
> To: David Lapointe <david.lapoi...@tufts.edu>
> Cc: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.
> galaxyproject.org>
> Subject: Re: [galaxy-dev] Modifying a Cloud Cluster Instance
>
> Hi David,
> I totally missed this email from you - apologies! Are you still trying to
> do this or did you figure it out? Following the general guidelines on
> customizing the instance is a good place to start. There is an option in
> CloudMan to turn on non-signed SSL (from the Admin page) but the steps
> performed may be useful in trying to do the same manually. I believe it's
> necessary to just update and reload the Nginx configuration. A template can
> be found here https://github.com/galaxyproject/cloudman/blob/
> 8db478a8d777bcc48ed8c838f1f9de8519a11e60/cm/conftemplates/
> nginx_server_ssl.default
>
> Hope this helps and, again, sorry for not getting back to you earlier.
>
>
> On Thu, Aug 11, 2016 at 2:30 PM, Lapointe, David <david.lapoi...@tufts.edu
> > wrote:
>
>> This is on Amazon I should add.
>> --
>> David Lapointe Ph.D.
>> Sr. Bioinformatics Specialist
>> Research Technology (RT)
>> Tufts Technology Services (TTS)
>> 16 Dearborn Road
>> Somerville MA 02144
>>
>> Phone:  617-627-5319
>> Fax: 617-627-3667
>> http://it.tufts.edu
>>
>> From: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> on behalf
>> of David Lapointe <david.lapoi...@tufts.edu>
>> Date: Thursday, August 11, 2016 at 1:44 PM
>> To: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyprojec
>> t.org>
>> Subject: [galaxy-dev] Modifying a Cloud Cluster Instance
>>
>> I have  a Cloudman instance up and running, but I would like to add SSL (
>> I have a signed cert and key ) and productionize it.  I see several methods
>> documented,but it's not clear which one is preferred: edit in place then
>> persist changes and restart  (wiki/Cloudman/CustomizeGalaxyCloud) seems
>> best to me. Any advice?
>>
>> Thanks
>>
>> David
>> --
>> David Lapointe Ph.D.
>> Sr. Bioinformatics Specialist
>> Research Technology (RT)
>> Tufts Technology Services (TTS)
>> 16 Dearborn Road
>> Somerville MA 02144
>>
>> Phone:  617-627-5319
>> Fax: 617-627-3667
>> http://it.tufts.edu
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Modifying a Cloud Cluster Instance

[Enis Afgan]

Hi David,
I totally missed this email from you - apologies! Are you still trying to
do this or did you figure it out? Following the general guidelines on
customizing the instance is a good place to start. There is an option in
CloudMan to turn on non-signed SSL (from the Admin page) but the steps
performed may be useful in trying to do the same manually. I believe it's
necessary to just update and reload the Nginx configuration. A template can
be found here
https://github.com/galaxyproject/cloudman/blob/8db478a8d777bcc48ed8c838f1f9de8519a11e60/cm/conftemplates/nginx_server_ssl.default

Hope this helps and, again, sorry for not getting back to you earlier.


On Thu, Aug 11, 2016 at 2:30 PM, Lapointe, David 
wrote:

> This is on Amazon I should add.
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> From: galaxy-dev  on behalf
> of David Lapointe 
> Date: Thursday, August 11, 2016 at 1:44 PM
> To: "galaxy-dev@lists.galaxyproject.org"  galaxyproject.org>
> Subject: [galaxy-dev] Modifying a Cloud Cluster Instance
>
> I have  a Cloudman instance up and running, but I would like to add SSL (
> I have a signed cert and key ) and productionize it.  I see several methods
> documented,but it's not clear which one is preferred: edit in place then
> persist changes and restart  (wiki/Cloudman/CustomizeGalaxyCloud) seems
> best to me. Any advice?
>
> Thanks
>
> David
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Amazon Instance Types for Galaxy

[Enis Afgan]

Hi David,
Yes there are: t2 class it typically a bit small, particularly t2.small and
smaller. m4 and c4 classes work fine but require use of VPC (meaning you
need to select a subnet to launch into from the advanced startup options on
https://launch.usegalaxy.org/). I haven't tried all the instance types but
don't see why other classes would have issues. Please let us know if you
find out differently.

Sorry for a delayed response.

On Thu, Jun 23, 2016 at 4:57 PM, Lapointe, David 
wrote:

> Are there instance types which will not work for Galaxy/Cloudman on AWS?
> EBS volumes for example.
>
> Thanks,
> David
> --
> David Lapointe Ph.D.
> Sr. Bioinformatics Specialist
> Research Technology (RT)
> Tufts Technology Services (TTS)
> 16 Dearborn Road
> Somerville MA 02144
>
> Phone:  617-627-5319
> Fax: 617-627-3667
> http://it.tufts.edu
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Parallelism (job splitting) for ncbi_blast_plus running through CloudMan

[Enis Afgan]

>
>
> Now we have what may be a CloudMan question: our working cluster has a
> head node and 4 workers. The head node is loaded up but the workers are
> idle. I would have thought jobs should be pushing out to the workers but we
> don't see any load on these machines.
>
> So are the jobs queued just not being scheduled on the workers? If that's
the case, it is definitely not the expected behavior. What about submitting
an additional job from Galaxy or the command line - where does it go? I've
never tried the splitting thing but just wonder if there's some affinity
assigned that makes all jobs run on the same node, although it seems that
wold defeat the whole purpose of splitting...
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ERROR: PIP is not a legl parament of an Ansible Play

[Enis Afgan]

The location of that dir is defined in shed_tool_conf.xml[.sample] and
defaults to ../shed_tools. The reasoning is described here, under item #2:
https://wiki.galaxyproject.org/DevNewsBriefs/2012_10_23#Tool_Shed

On Thu, Apr 7, 2016 at 11:09 AM, William Ge <w...@centrilliontech.com> wrote:

> Thanks, Enis!
>
> Yes, that line was uncommented with the option pointing to
> “tool_conf.xml,shed_tool_conf.xml”. However, I noticed that the
> “shed_tool_conf.xml” has not been changed after running the
> ansible-playbook to install tools. One other thing: the shed_tools
> directory was one level above the Galaxy subdirectories: i.e., all
> subdirectories (config, database, tool-data, tools, static, etc) are in
> /home/galaxy/galaxy, while “shed_tools” is in /home/galaxy. Do not know why.
>
>
>
> Thanks,
>
> Bill
>
>
>
> *From:* Enis Afgan [mailto:enis.af...@irb.hr]
> *Sent:* Thursday, April 7, 2016 7:06 AM
> *To:* William Ge <w...@centrilliontech.com>
> *Cc:* Marius van den Beek <m.vandenb...@gmail.com>;
> galaxy-dev@lists.galaxyproject.org
>
> *Subject:* Re: [galaxy-dev] ERROR: PIP is not a legl parament of an
> Ansible Play
>
>
>
> Hi Bill,
>
> Do you have the value set in Galaxy's *$GALAXY_ROOT/config/galaxy.ini*
> file for setting *tool_config_file *option? I believe this variable
> should be set to a file other than the default **.sample*.
>
>
>
> On Wed, Apr 6, 2016 at 9:21 PM, William Ge <w...@centrilliontech.com>
> wrote:
>
> Hi, Marius,
>
>
>
> I got ansible running and installed the tools on my local instance of
> Galaxy. However, the tools are not shown on the tool panel on the web page.
> Wonder if the tools has been installed in the wrong place, or something
> else is wrong. Have been searching the Wiki and list without any resulst.
> Please point me to the right directions.
>
>
>
> Thanks,
>
> Bill
>
>
>
> *From:* Marius van den Beek [mailto:m.vandenb...@gmail.com]
> *Sent:* Friday, March 4, 2016 12:27 PM
> *To:* William Ge <w...@centrilliontech.com>
> *Cc:* Dannon Baker <dannon.ba...@gmail.com>;
> galaxy-dev@lists.galaxyproject.org
> *Subject:* Re: [galaxy-dev] ERROR: PIP is not a legl parament of an
> Ansible Play
>
>
>
> On 4 March 2016 at 20:47, William Ge <w...@centrilliontech.com> wrote:
>
> Thanks a lot, Marius!
>
> Sure, you're welcome!
>
>
>
> I am a newbie to this complex system and want a quick clone/install
> locally. I have spent over two weeks on the process. There are many
> configuration files and environment variables or dependencies to sort out.
> I wish the sample configuration files would include most of the options so
> that new installation just needs to comment out or delete unwanted ones.
>
>
>
> The galaxy team definitely aims for this, and usually you should (tm) be
> able to clone galaxy and use run.sh to get going.
>
> If you can tell us the steps where you were struggling we can try to
> update or clarify the documentation (A lot of it is on the wiki).
>
> Anyway, here is the error I got:
>
> [galaxy@Pegasus ansible-galaxy-tools]$ python
> files/install_tool_shed_tools.py -h
> Traceback (most recent call last):
>   File "files/install_tool_shed_tools.py", line 49, in 
> logging.captureWarnings(True)  # Capture HTTPS warngings from urllib3
> AttributeError: 'module' object has no attribute 'captureWarnings'
>
>
>
> If I run this after activating the galaxy_env and install ensible,
> bioblend, argparse and importlib, the same error came up. I guess this is
> Python 2.6 issue. Any solutions known?
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ERROR: PIP is not a legl parament of an Ansible Play

[Enis Afgan]

Hi Bill,
Do you have the value set in Galaxy's *$GALAXY_ROOT/config/galaxy.ini* file
for setting *tool_config_file *option? I believe this variable should be
set to a file other than the default **.sample*.

On Wed, Apr 6, 2016 at 9:21 PM, William Ge  wrote:

> Hi, Marius,
>
>
>
> I got ansible running and installed the tools on my local instance of
> Galaxy. However, the tools are not shown on the tool panel on the web page.
> Wonder if the tools has been installed in the wrong place, or something
> else is wrong. Have been searching the Wiki and list without any resulst.
> Please point me to the right directions.
>
>
>
> Thanks,
>
> Bill
>
>
>
> *From:* Marius van den Beek [mailto:m.vandenb...@gmail.com]
> *Sent:* Friday, March 4, 2016 12:27 PM
> *To:* William Ge 
> *Cc:* Dannon Baker ;
> galaxy-dev@lists.galaxyproject.org
> *Subject:* Re: [galaxy-dev] ERROR: PIP is not a legl parament of an
> Ansible Play
>
>
>
> On 4 March 2016 at 20:47, William Ge  wrote:
>
> Thanks a lot, Marius!
>
> Sure, you're welcome!
>
>
>
> I am a newbie to this complex system and want a quick clone/install
> locally. I have spent over two weeks on the process. There are many
> configuration files and environment variables or dependencies to sort out.
> I wish the sample configuration files would include most of the options so
> that new installation just needs to comment out or delete unwanted ones.
>
>
>
> The galaxy team definitely aims for this, and usually you should (tm) be
> able to clone galaxy and use run.sh to get going.
>
> If you can tell us the steps where you were struggling we can try to
> update or clarify the documentation (A lot of it is on the wiki).
>
> Anyway, here is the error I got:
>
> [galaxy@Pegasus ansible-galaxy-tools]$ python
> files/install_tool_shed_tools.py -h
> Traceback (most recent call last):
>   File "files/install_tool_shed_tools.py", line 49, in 
> logging.captureWarnings(True)  # Capture HTTPS warngings from urllib3
> AttributeError: 'module' object has no attribute 'captureWarnings'
>
>
>
> If I run this after activating the galaxy_env and install ensible,
> bioblend, argparse and importlib, the same error came up. I guess this is
> Python 2.6 issue. Any solutions known?
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Galaxy CloudMan with Galaxy 16.01

[Enis Afgan]

Hello!

*We have released a new version of Galaxy CloudMan on AWS.* CloudMan
 offers an easy way to get a
personal and completely functional instance of Galaxy in the cloud in just
a few minutes, without any manual configuration or imposed quotas. Once
running, you have complete control over Galaxy, including the ability to
install new tools.

Most notable changes include:
 * Galaxy 16.01 release
 * A fine-grained control over auto-scaling options
 * Several fixes to cluster sharing and cloning

See the CHANGELOG
 for a
more complete set of changes.

Give it a whirl at launch.usegalaxy.org or take a look at the Getting
Started 
documentation.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Cloudman & AWS IAM_PROFILE support?

[Enis Afgan]

Hi Ryan,
I'm not sure I really understand your question but CloudMan can run under
an IAM account - it is just necessary to give the IAM user permissions to
create EC2 and S3 resources. Let us know if that's not what you had in mind
or if you have any more questions.

Cheers,
Enis

On Mon, Mar 14, 2016 at 1:56 PM, Ryan G  wrote:

> Hi - Does Cloudman support IAM_PROFILES?   We run our instances within a
> VPC and assign an IAM_PROFILE to the ec2 instances so that they have access
> to resouces.   Does Cloudman have this support?
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on Azure

[Enis Afgan]

Hi Alex,
Assuming you'd want to set it up as a dedicated server on a VM running on
Azure, I'd suggest taking a look at the production instance setup notes:
https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer

Other admin topics are also available on the wiki:
https://wiki.galaxyproject.org/Admin

Hope this helps.

On Mon, Feb 29, 2016 at 1:40 PM, Alex Lenail  wrote:

> Hello,
>
> My name is Alex LeNail and I'm a researcher at MIT. I'm looking to set up
> a Galaxy server, and our lab has a great deal of Microsoft Azure credits,
> so we were hoping to deploy Galaxy on there.
>
> Where should I start if I wanted to set that up? I haven't really found
> any resources online with any description of how that might be
> accomplished.
>
> Thanks,
>
> --Alex
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Current state of cloudman etc. within private AWS VPCs?

[Enis Afgan]

Hi Chris,
I believe you're the first person that's tried to run this in a completely
private VPC so I don't have experience with it but can suggest a change
that might get over this initial hump at least.

My guess is that get_galaxy_dns method is why you're seeing this log
message (
https://github.com/galaxyproject/cloudman/blob/master/cm/controllers/root.py#L863)
so perhaps changing that method to return an address you've configured as
an accessible one would fix it? After changing the code, it'll also be
necessary to update CloudMan source code in the corresponding bucket; see
this page for more on that topic:
https://wiki.galaxyproject.org/CloudMan/CustomizeGalaxyCloud#Using_custom_CloudMan_application

Hope this helps. Let us know how things progress and we can update the code
if a solution is generic.

Cheers,
Enis


On Fri, Jan 29, 2016 at 10:51 AM, Chris Dagdigian  wrote:

>
> Apologies if this is asked often, I did a bit of searching and google-foo
> but could not find very new news...
>
>
> I've long been a fan/user of cloudman-managed Galaxy clusters but for the
> first time I need galaxy in an AWS environment where we absolutely cannot
> stand up any sort of public facing service or any sort of host with a
> visible EIC IP address. The hosts can access the internet through NAT etc.
> but we can't run them in a public VPC or otherwise expose them to the
> Internet at large.
>
> My experiment last night with cloudman launching inside a private VPC
> (with NAT'ed internet access and working forward/reverse DNS resolution for
> all hosts in the subnet) seemed to have failed - the key error in the
> cloudman logs was an endless iteration of "attempting to learn public
> hostname" log entries.
>
> I blew away the install when I saw that error message endlessly repeating
> because there is zero chance our hosts in this VPC subnet will ever have a
> public FDQN hostname. They all have private IPs and private (but DNS
> resolvable) hostnames.
>
> Can anyone give a current state update on cloudman inside private VPCs? Is
> this something I should have debugged further or am I better off just
> rolling my own AMI and installing galaxy and perhaps the cloudman elements
> by hand?
>
> Thanks!
>
> -Chris
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ProFTPd in CloudMan

[Enis Afgan]

Thanks for reporting this (and figuring out the solution). It's been fixed
with this commit

so any new cluster will work fine.

Cheers.

On Tue, Sep 1, 2015 at 7:42 PM, Alexander Vowinkel <
vowinkel.alexan...@gmail.com> wrote:

> Update:
>
> ProFTP gives as an answer to passive mode the INTERNAL IP to connect which
> makes FileZilla break the connection.
>
> Indeed adding MasqueradeAddressin the config file, it
> works.
>
> Where is the proftp config file being created in cloudman? How can this be
> added dynamically?
>
> Best,
> Alexander
>
> 2015-09-01 18:19 GMT-05:00 Alexander Vowinkel <
> vowinkel.alexan...@gmail.com>:
>
>> Hi,
>>
>> I get the error "Could not read from socket: ECONNRESET - Connection
>> reset by peer" when connecting with FileZilla to the Cloud Instance.
>> Using launch.usegalaxy.org
>>
>> FileZilla Log: http://pastebin.com/zfACC6wH
>> ProFTP Log: http://pastebin.com/T9JxKVCb
>>
>> With console FTP in linux I can connect, get a listing and upload files.
>> I tried to find out what the reason is, but no solution found.
>> bareFTP is also not able to connect.
>>
>> The ports for PassivePorts are open (checked in AWS)
>>
>> Any idea on this?
>>
>> Thanks,
>> Alexander
>>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy in Amazon

[Enis Afgan]

Hi Ryan,
What you're suggesting to do is still somewhat experimental but we're
continuing to work on it to make it more integrated into the Galaxy
ecosystem and more robust. There are really three general approaches:
1. Run Galaxy via CloudMan 100% on AWS. This option is most robust and
basically ready for use but, over time and if you decide to make
modifications to various pieces of the puzzle, will require an increased
understanding of how CloudMan works. It's also the most expensive option.
2. Run Galaxy UI locally and create a CloudMan cluster on demand with the
Pulsar http://pulsar.readthedocs.org/en/latest/index.html service enabled
to accept jobs from the local Galaxy. This paper describes that approach:
http://onlinelibrary.wiley.com/doi/10.1002/cpe.3536/abstract
3. Run your Galaxy UI locally and create Ansible roles/tasks to dynamically
acquire cloud instances and assemble those into a cluster. You will
probably want to use Pulsar for job management again. This option gives you
most control but also means you'll need to build the system. Nate may also
have more comments about this.

I've also put some comments about your specific questions inline.

Hope this helps clarify the situation at least. We're actively working on
this scenario so things should get easier in the future. Let us know if you
have more questions and what you decide.

Cheers,
Enis



On Wed, Aug 19, 2015 at 8:53 AM, Ryan G ngsbioinformat...@gmail.com wrote:

 Hi all - We are running a local instance of Galaxy on our internal
 infrastructure.  It seems to be going well.

 We've gotten to the point where we are ready to migrate our NGS data to
 Amazon for storage in S3.  We are also looking at how Galaxy can be used in
 Amazon.  Specifically, we are interested in understanding:

 1)  Should we run an instance of Galaxy in Amazon, or continue to run it
 locally (to minimize costs) but have it run analyses in Amazon?

 The options above summarize this scenario.


 2)  Regardless of how we run it, data will be stored in S3.  How will
 Galaxy interact with S3 for its Data Libraries?

Galaxy implements an Object Store interface that can link to S3 as a
back-end data store. It's been around for a number of years now and
demonstrated as working but it also hasn't been used in production so I'd
suggest testing this first. Galaxy configuration options for the object
store are in Galaxy's config file:
https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.ini.sample#L289


 3)  Is it even possible to separate the Galaxy web interface from the HPC
 cluster?

Yes; you either need a shared file system between the resources or use
Pulsar.


 3)  We understand Galaxy in Amazon uses CloudMan.  Can we run this in our
 VPC with our own AMI?

Yes; you can build your own version of the system with the tools and
whatever else you would like to configure. Docs on how to do this are
available here: https://wiki.galaxyproject.org/CloudMan/Building


 If anyone can provide insights into how they are using Galaxy in Amazon, I
 am very interested to hear your thoughts.

 Ryan

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cloudman galaxy - too large volume

[Enis Afgan]

All of the volumes will be SSD from now on, large or small (technically,
gp2 type as defined here
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html) but
no, it's not currently possibly to specify the type via user data.
Regardless, to specify the *volume_type* the volume needs to exist a priori
and so if it an SSD type was created, that's what it's going to be.

On Wed, Jul 15, 2015 at 11:03 AM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 Great!
 Especially also witht he IO performance.
 Is it possible to define the volume type manually by putting 'volume_type'
 to the user data?
 So I can also have smaller volumes with SSD?
 Best,
 Alexander

 2015-07-14 13:57 GMT-05:00 Enis Afgan enis.af...@irb.hr:

 I just fixed this here
 https://github.com/galaxyproject/cloudman/commit/0f6ab132958ad57a338c66387112407edbc7632d#diff-9eba488c9175de87945a13d7b7228a6dR621

 This required switching to SSD-based volumes on AWS and, besides actually
 being able to create volumes up to 16TB now, it should give us better IO
 performance now.

 On Mon, Jul 13, 2015 at 5:06 PM, Alexander Vowinkel 
 vowinkel.alexan...@gmail.com wrote:

 Hi,

 I get this error on cluster creation:

 Error creating volume: EC2ResponseError: 400 Bad Request
 InvalidParameterValueVolume of 2048GiB is too large for volume type
 standard; maximum is 1024GiB

 Is there a way to get a volume that big for galaxy?

 Best,
 Alexander

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/




___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cloudman galaxy - too large volume

[Enis Afgan]

I just fixed this here
https://github.com/galaxyproject/cloudman/commit/0f6ab132958ad57a338c66387112407edbc7632d#diff-9eba488c9175de87945a13d7b7228a6dR621

This required switching to SSD-based volumes on AWS and, besides actually
being able to create volumes up to 16TB now, it should give us better IO
performance now.

On Mon, Jul 13, 2015 at 5:06 PM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 Hi,

 I get this error on cluster creation:

 Error creating volume: EC2ResponseError: 400 Bad Request
 InvalidParameterValueVolume of 2048GiB is too large for volume type
 standard; maximum is 1024GiB

 Is there a way to get a volume that big for galaxy?

 Best,
 Alexander

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cloudman persist filesystem error

[Enis Afgan]

Hi Alexander,
I tested the process of persisting file system changes but it all worked as
expected. Looking at the log snippet you sent, the only thing that seems
off is the very first line: Completed creation of a snapshot for the volume
'None', snap id: 'snap-6f0b211d' - volume should not be None there.
Actually, it's incredible that it would be None because a volume must exist
in order to create a snapshot... Do you have more content from that log?

In the mean time, I added more safeguards if this happens again:
https://github.com/galaxyproject/cloudman/commit/337ecb95a04788e540d60be37b4f4f817257b24f

On Tue, Jun 30, 2015 at 7:32 PM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 Hi,

 from IRC this question/bug report:

 (18:06:13) avowinkel: some cloudman guy here? enis?
 (18:06:30) avowinkel: AttributeError: 'NoneType' object has no attribute
 'id'
 (18:06:50) avowinkel: when persisting galaxyIndices
 (18:07:43) avowinkel: snapshot is created, but it crashes when ...
 somewhen well after that
 (18:11:08) avowinkel: http://pastebin.com/SDp1p96K
 (18:11:10) mrscribe: Title: paster.log - Pastebin.com (at pastebin.com)

 Best,
 Alexander

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cloudman launch

[Enis Afgan]

 A) I wanted to tell you that I did the update of the galaxyFS
 with the playbook, by using the role cm.filesystem only.
 It worked -  thanks!


Excellent!


 B) How could I mount another volume to the upstarting cluster?

  - name: stuff
snap_id: snap-xyz

 Would that work?
 E.g when I want this snapshot be mounted as /mnt/stuff


 With this commit
https://github.com/galaxyproject/cloudman/commit/b56a6cd77899ac762d34a747da51ac78a14d3da7,
you can (I guess that'll be a goo reason to make a move on the next
question :) )


 C) How can I use the dev branch of galaxy with cloudman?
 There are some things only in dev I will need. How would
 that be possible?


You'll need to keep using a custom bucket and place cm.tar.gz and
cm_boot.py into that bucket. I'll send you a couple of scripts that will
make that super easy.


 D) There is an email from me to the list from two days ago.
 Could you look at that?
 It's not critical for me, but might lead to a bug.


Just replied. Thanks.
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cloudman error somewhere

[Enis Afgan]

Hi Alexander,
When you build the image, did you run the image cleanup step? RabbitMQ's
database needs to be cleaned up before building the image otherwise the
application won't start properly and the AMQP connection will never be
established.

You can also post the entire CloudMan's log somewhere and I can take a look
there for other clues.



On Tue, Jun 16, 2015 at 4:07 PM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 Maybe connected. this error (happening multiple times):

 
 Exception happened during processing of request from ('127.0.0.1', 37201)
 Traceback (most recent call last):
   File
 /home/ubuntu/.virtualenvs/CM/local/lib/python2.7/site-packages/paste/httpserver.py,
 line 1068, in process_request_in_thread
 self.finish_request(request, client_address)
   File /usr/lib/python2.7/SocketServer.py, line 334, in finish_request
 self.RequestHandlerClass(request, client_address, self)
   File /usr/lib/python2.7/SocketServer.py, line 651, in __init__
 self.finish()
   File /usr/lib/python2.7/SocketServer.py, line 710, in finish
 self.wfile.close()
   File /usr/lib/python2.7/socket.py, line 279, in close
 self.flush()
   File /usr/lib/python2.7/socket.py, line 303, in flush
 self._sock.sendall(view[write_offset:write_offset+buffer_size])
 error: [Errno 32] Broken pipe
 



 2015-06-16 13:59 GMT-05:00 Alexander Vowinkel 
 vowinkel.alexan...@gmail.com:

 Hi,

 I have the following problem:
 First, the job scheduling didn't work.

 I tried to upload a file, but the job was waiting.
 Then I checked squeue, which timed out.
 The slurm service was running.

 Then I restarted the cluster.
 Now I even have problems access the cloudman admin page.
 Meaning, the page is loading endless.
 The machine is not busy (CPU and mem @ htop)
 In the paster.log it tries this every 4 seconds:

 2015-06-16 18:35:56,547 DEBUG master:2579 Trying to setup AMQP
 connection; conn = 'cm.util.comm.CMMasterComm object at 0x7f0aa26d72d0'
 2015-06-16 18:35:56,547 DEBUG   comm:30   Setting up a new AMQP
 connection
 2015-06-16 18:35:56,548 DEBUG   comm:49   AMQP Connection
 Failure:  [Errno 111] Connection refused


 Any idea what this might be?
 Something seems to be really weird.

 I am using the next version of CloudMan (dev).
 I created an own instance with the Cloud Man playbook.

 Best,
 Alexander



 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

[Enis Afgan]

Hi Nick,
Sorry to hear you're having trouble. I just tried a couple of scenarios and
they all worked as expected (e.g., with and without elastic IPs, different
instance types).

The main CloudMan log is located in /mnt/cm/paster.log, on both master and
worker instances (if you didn't download the ssh key from cloudlaunch, you
can ssh with ubuntu username and the same password as provided on the
cloudlaunch form). The log is also available from the UI if you go to Admin
page and then click 'Show CloudMan log' under 'System controls'. If you can
share that, we can hopefully figure out what's going on.

Best,
Enis

On Wed, Jun 17, 2015 at 11:47 AM, Nicholas Dickens 
nick.dick...@glasgow.ac.uk wrote:

   Hi All,

  First time post, as a quick intro, I’m have some reasonable experience
 with EC2  AWS for developing our own pipelines, I’m comfortable in python
 and *nix flavours and I am developing a completely custom galaxy in AWS for
 members of the WTCMP, Glasgow.  In the meantime, I have thrown up a quick
 cloudman galaxy using the cloudstart and the cloudman 2.3 ami
 (ami-a7dbf6ce) in us-east-1.  Auto–scaling didn’t seem to work so I’ve
 switched it off and added nodes manually, I tried various sizes including
 ‘same as master’ instances but they just don’t start – in the EC2 console I
 can see them and see them running.  But they’re constantly pending in the
 /cloud interface and in the log they reboot 4 times and then terminate –
 apparently not responding 10:16:56 - Instance i-xx not responding
 after 4 reboots. Terminating instance.

  It’s out of the box, I editted the universe_wsgi.ini… file to disallow
 user registration and allow me to impersonate users but didn’t change
 anything else.  The only other configuration I’ve done is associate an
 elastic IP with the master instance so I can have a more static url for a
 couple of test users (if I need to destroy it and start again, etc).

  I’m new to the system so don’t know which logs are best to check…and am
 I missing something obvious?  It there a known bug when using elastic IPs?
 I’ve googled but with no joy.

  Thanks for your help and best wishes,

  Nick
  --
  Nick Dickens
 DPhil BSc ARCS

  Bioinformatics Team Leader
 Wellcome Trust Centre for Molecular Parasitology
 B6-21 SGDB
 120 University Place
 Glasgow
 G12 8TA

  Tel: +44 141 330 8282

  http://fb.me/WTCMPbix
 @WTCMPbix
 http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/

 http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/


 - Upozorenje -

 Automatskom detekcijom utvrdjeno je da tekst ove poruke
 podsjeca na tzv. phishing poruku.

  AKO SE U PORUCI TRAZI DA POSALJETE VASU IRB LOZINKU ILI
 DA UNESETE IRB PODATKE NA NAVEDENOM LINKU, RADI SE O
 NAPADU S CILJEM KRADJE I ZLOUPOTREBE PODATAKA.

  Centar za informatiku i racunarstvo,
 Institut Rudjer Boskovic

 - Upozorenje -

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Data Manager - BWA-MEM index

[Enis Afgan]

To see the data in a Data Manager, you first need to fetch the data via a
Data Manager vs. a history. So, you'll first need to install the fetch DM (
https://toolshed.g2.bx.psu.edu/repository?repository_id=cdf172d633d603c2changeset_revision=bca4c608408c)
and then click on 'Reference Genome - fetching' to fetch the appropriate
data before being able to run the BWA-MEM DM.

Hope this helps,
Enis

On Fri, Jun 5, 2015 at 6:05 PM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 Hi,

 I'm trying to index some fasta files.
 They are uploaded in data library and I also loaded them into
 Data Manager history and my local history.

 But when I open the Data Manager, I can't choose my file.
 It says in Source FASTA Sequence only No options available.

 Can somebody help?

 Best,
 Alexander

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] job queue problem with cloud man

[Enis Afgan]

Hi Alexander,
I'm glad to hear you got the jobs running. I don't think there's an
informative message that is being delivered when a job is not able to run
but looking at Galaxy (and/or CloudMan) logs might be necessary.

FYI, the version of CloudMan you're running (i.e., the currentlt released
version) is using SGE for running jobs (vs. Condor) to you would check the
status of a job via *qstat*.



On Wed, Jun 10, 2015 at 11:43 PM, Alexander Vowinkel 
vowinkel.alexan...@gmail.com wrote:

 I figured out that only jobs that used one of the files could not run.
 They wanted a tabular file, which I uploaded before as txt and
 changed then to tabular.
 It gave the message that some metadata could not be determined.
 Auto-detection somehow just worked now when I tried it the 3rd time.

 It would be good to be able to see some information about a reason
 why a job can not be executed and is waiting.
 Is that somewhere implemented?

 Best,
 Alexander

 2015-06-10 16:19 GMT-05:00 Alexander Vowinkel 
 vowinkel.alexan...@gmail.com:

 Hi list-members,

 I have a Cloud Man instance on amazon.
 I did minimal adaption (installing some tools, adding an index)

 I'm currently running only a master, which is allowed to run jobs.

 I have now 3 jobs saying: This job is waiting to run
 But in the CloudMan console it says:
 Worker status: Idle: 0 Available: 0 Requested: 0
 Also condor_q gives 0 jobs.

 Where is the problem?
 I already restarted the galaxy service via the admin console...

 Can someone help me to find whats going wrong?

 Thanks!
 Alexander

 PS: I'm running different job types (one is just Add column)
 PPS: Upload File jobs are working



 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] SSL/LDAP configuration for CloudMan/Galaxy

[Enis Afgan]

Hi David,
The nginx change will require a new AMI that includes the appropriately
compiled nginx. We've been working on automating the process of building
the image and it's captured in this Ansible playbook:
https://github.com/galaxyproject/galaxy-cloudman-playbook (the current
README is a bit of out of sync with the code but I have been working on a
updating that and will commit it later on today or tomorrow at the latest).
You'll need to edit the nginx installation procedure to include the
customizations for nginx you want.

Re. configuration changes (nginx.conf and galaxy.ini) - these are done via
CloudMan (
https://github.com/galaxyproject/cloudman/tree/master/cm/conftemplates). So
it would be necessary to create a your own S3 bucket and host CloudMan
source there with the desired customizations. I'll send you a paper that
will be presented next month that captures all the pieces that are required
for assemble a custom version of Galaxy CloudMan.

Hope this helps and please let us know if you have any more questions,
Enis

On Fri, Apr 24, 2015 at 2:25 PM, David Kovalic kova...@analome.com wrote:

 Hi,


 We are interested in running CloudMan/Galaxy with SSL and LDAP. After
 searching around on the internet it seems like this is achievable and
 probably not too difficult, but there is no current complete “cookbook
 recipe” for doing so, so I thought it best to ask questions before I go
 ahead and break stuff :)


  As I understand:

 · ngnix needs to have the LDAP module added, as the standard CM
 ngnix build doesn’t include this

 · ngnix needs to have a custom ngnix.conf file which specifies
 the use of SSL and LDAP

 · Galaxy need to have a custom configuration universe_wsgi.ini
 for LDAP use


By searching online I can’t clearly figure out:

 · How to recompile (and persist across CM cluster
 termination/restart) a new version of ngnix

 · The best way to maintain and specify a custom ngnix.conf. Is it
 possible to do this by placing the custom ngnix.conf in the cluster S3
 bucket and adding a configuration line specifying its URL (e.g.
 nginx_conf_contents: https://s3.amazonaws.com/[cm bucket ID]/ngnix.conf)
 in persistent_data.yaml file in the CM S3 bucket?

 · Where do I make the modifications such that the changes to
 universe_wsgi.ini persist across CM cluster termination/restart?

It would be great to get some experienced insight on how best to
 complete this configuration, and have it persist.


 Any guidance would be greatly appreciated. Thanks,


 David Kovalic




 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/