Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

[Enis Afgan]

Hmm - /mnt definitely should not be empty. There's nothing unusual in the
log you sent so could you please send me the one from the worker? It's in
the same location (/mnt/cm/paster.log)
If it's not there, please track the boot procedure logs as follows and can
send those logs:
1. /usr/bin/ec2autorun.log
2. /tmp/cm/cm_boot.py.log
3. /mnt/cm/paster.log

Thanks,
Enis

On Fri, Jun 19, 2015 at 5:02 PM, Nicholas Dickens <
nick.dick...@glasgow.ac.uk> wrote:

>   Thanks – I’ve attached the log. I just tried to start a worker and let
> it go to the first reboot and then copied this log.  I logged into the
> worker and it looks ok (dmesg, etc) the only noticable thing was /mnt is
> empty (just a lost+found directory) and I was expecting to see an nfs mount
> for galaxy export or something.  But I’m still finding my way round the
> system.  It may also have been the time in the reboot cycle that I was
> there.
>
>  Best wishes,
>
>  Nick
>  --
>  Nick Dickens
> DPhil BSc ARCS
>
>  Bioinformatics Team Leader
> Wellcome Trust Centre for Molecular Parasitology
> B6-21 SGDB
> 120 University Place
> Glasgow
> G12 8TA
>
>  Tel: +44 141 330 8282
>
>  http://fb.me/WTCMPbix
> @WTCMPbix
> http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/
>
> http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/
>
>   From: Enis Afgan 
> Date: Friday, 19 June 2015 17:00
> To: Nick Dickens 
> Cc: "galaxy-dev@lists.galaxyproject.org" <
> galaxy-dev@lists.galaxyproject.org>
> Subject: Re: [spam?] [galaxy-dev] Cloudman cluster not starting workers
>
>   Hi Nick,
> Sorry to hear you're having trouble. I just tried a couple of scenarios
> and they all worked as expected (e.g., with and without elastic IPs,
> different instance types).
>
>  The main CloudMan log is located in /mnt/cm/paster.log, on both master
> and worker instances (if you didn't download the ssh key from cloudlaunch,
> you can ssh with ubuntu username and the same password as provided on the
> cloudlaunch form). The log is also available from the UI if you go to Admin
> page and then click 'Show CloudMan log' under 'System controls'. If you can
> share that, we can hopefully figure out what's going on.
>
>  Best,
> Enis
>
> On Wed, Jun 17, 2015 at 11:47 AM, Nicholas Dickens <
> nick.dick...@glasgow.ac.uk> wrote:
>
>>   Hi All,
>>
>>  First time post, as a quick intro, I’m have some reasonable experience
>> with EC2 & AWS for developing our own pipelines, I’m comfortable in python
>> and *nix flavours and I am developing a completely custom galaxy in AWS for
>> members of the WTCMP, Glasgow.  In the meantime, I have thrown up a quick
>> cloudman galaxy using the cloudstart and the cloudman 2.3 ami
>> (ami-a7dbf6ce) in us-east-1.  Auto–scaling didn’t seem to work so I’ve
>> switched it off and added nodes manually, I tried various sizes including
>> ‘same as master’ instances but they just don’t start – in the EC2 console I
>> can see them and see them running.  But they’re constantly pending in the
>> /cloud interface and in the log they reboot 4 times and then terminate –
>> apparently not responding "10:16:56 - Instance i-xx not responding
>> after 4 reboots. Terminating instance".
>>
>>  It’s out of the box, I editted the universe_wsgi.ini… file to disallow
>> user registration and allow me to impersonate users but didn’t change
>> anything else.  The only other configuration I’ve done is associate an
>> elastic IP with the master instance so I can have a more static url for a
>> couple of test users (if I need to destroy it and start again, etc).
>>
>>  I’m new to the system so don’t know which logs are best to check…and am
>> I missing something obvious?  It there a known bug when using elastic IPs?
>> I’ve googled but with no joy.
>>
>>  Thanks for your help and best wishes,
>>
>>  Nick
>>  --
>>  Nick Dickens
>> DPhil BSc ARCS
>>
>>  Bioinformatics Team Leader
>> Wellcome Trust Centre for Molecular Parasitology
>> B6-21 SGDB
>> 120 University Place
>> Glasgow
>> G12 8TA
>>
>>  Tel: +44 141 330 8282
>>
>>  http://fb.me/WTCMPbix
>> @WTCMPbix
>> http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/
>>
>> http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/
>>
>>
>> - Upozorenje -
>>
>> Automatskom detekcijom utvrdjeno je da tekst ove poruke
>> podsjeca na tzv. phishing poruku.
>>
>> AKO SE U PORUCI TRAZI DA POSALJETE VASU IRB LOZINKU ILI
>> DA UNESETE IRB PODATKE NA NAVEDENOM LINKU, RADI SE O
>> NAPADU S CILJEM KRADJE I ZLOUPOTREBE PODATAKA.
>>
>> Centar za informatiku i racunarstvo,
>> Institut Rudjer Boskovic
>>
>> - Upozorenje -
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the 

Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

[Nicholas Dickens]

Thanks – I’ve attached the log. I just tried to start a worker and let it go to 
the first reboot and then copied this log.  I logged into the worker and it 
looks ok (dmesg, etc) the only noticable thing was /mnt is empty (just a 
lost+found directory) and I was expecting to see an nfs mount for galaxy export 
or something.  But I’m still finding my way round the system.  It may also have 
been the time in the reboot cycle that I was there.

Best wishes,

Nick
--
Nick Dickens
DPhil BSc ARCS

Bioinformatics Team Leader
Wellcome Trust Centre for Molecular Parasitology
B6-21 SGDB
120 University Place
Glasgow
G12 8TA

Tel: +44 141 330 8282

http://fb.me/WTCMPbix
@WTCMPbix
http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/
http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/

From: Enis Afgan mailto:enis.af...@irb.hr>>
Date: Friday, 19 June 2015 17:00
To: Nick Dickens mailto:nick.dick...@glasgow.ac.uk>>
Cc: 
"galaxy-dev@lists.galaxyproject.org" 
mailto:galaxy-dev@lists.galaxyproject.org>>
Subject: Re: [spam?] [galaxy-dev] Cloudman cluster not starting workers

Hi Nick,
Sorry to hear you're having trouble. I just tried a couple of scenarios and 
they all worked as expected (e.g., with and without elastic IPs, different 
instance types).

The main CloudMan log is located in /mnt/cm/paster.log, on both master and 
worker instances (if you didn't download the ssh key from cloudlaunch, you can 
ssh with ubuntu username and the same password as provided on the cloudlaunch 
form). The log is also available from the UI if you go to Admin page and then 
click 'Show CloudMan log' under 'System controls'. If you can share that, we 
can hopefully figure out what's going on.

Best,
Enis

On Wed, Jun 17, 2015 at 11:47 AM, Nicholas Dickens 
mailto:nick.dick...@glasgow.ac.uk>> wrote:
Hi All,

First time post, as a quick intro, I’m have some reasonable experience with EC2 
& AWS for developing our own pipelines, I’m comfortable in python and *nix 
flavours and I am developing a completely custom galaxy in AWS for members of 
the WTCMP, Glasgow.  In the meantime, I have thrown up a quick cloudman galaxy 
using the cloudstart and the cloudman 2.3 ami (ami-a7dbf6ce) in us-east-1.  
Auto–scaling didn’t seem to work so I’ve switched it off and added nodes 
manually, I tried various sizes including ‘same as master’ instances but they 
just don’t start – in the EC2 console I can see them and see them running.  But 
they’re constantly pending in the /cloud interface and in the log they reboot 4 
times and then terminate – apparently not responding "10:16:56 - Instance 
i-xx not responding after 4 reboots. Terminating instance".

It’s out of the box, I editted the universe_wsgi.ini… file to disallow user 
registration and allow me to impersonate users but didn’t change anything else. 
 The only other configuration I’ve done is associate an elastic IP with the 
master instance so I can have a more static url for a couple of test users (if 
I need to destroy it and start again, etc).

I’m new to the system so don’t know which logs are best to check…and am I 
missing something obvious?  It there a known bug when using elastic IPs? I’ve 
googled but with no joy.

Thanks for your help and best wishes,

Nick
--
Nick Dickens
DPhil BSc ARCS

Bioinformatics Team Leader
Wellcome Trust Centre for Molecular Parasitology
B6-21 SGDB
120 University Place
Glasgow
G12 8TA

Tel: +44 141 330 8282

http://fb.me/WTCMPbix
@WTCMPbix
http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/
http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/


- Upozorenje -

Automatskom detekcijom utvrdjeno je da tekst ove poruke
podsjeca na tzv. phishing poruku.

AKO SE U PORUCI TRAZI DA POSALJETE VASU IRB LOZINKU ILI
DA UNESETE IRB PODATKE NA NAVEDENOM LINKU, RADI SE O
NAPADU S CILJEM KRADJE I ZLOUPOTREBE PODATAKA.

Centar za informatiku i racunarstvo,
Institut Rudjer Boskovic

- Upozorenje -

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



CloudMan from Galaxy
Admin | Report bugs | Wiki | Screencast  
Only up to the most recent 200 lines of the file (paster.log) are shown. Show 
all | Show less | Show more | Back to admin view
2015-06-19 20:42:32,573 DEBUG master:2534 S&S: Migration..Completed; 
SGE..OK; FS object for transient_nfs..OK; FS object for galaxy..OK; FS object 
for galaxyIndices..OK; Postgres..OK; ProFTPd..OK; Galaxy..OK; 
GalaxyReports..OK; 
2015-06-19 20:43:36,82

Re: [galaxy-dev] [spam?] Cloudman cluster not starting workers

[Enis Afgan]

Hi Nick,
Sorry to hear you're having trouble. I just tried a couple of scenarios and
they all worked as expected (e.g., with and without elastic IPs, different
instance types).

The main CloudMan log is located in /mnt/cm/paster.log, on both master and
worker instances (if you didn't download the ssh key from cloudlaunch, you
can ssh with ubuntu username and the same password as provided on the
cloudlaunch form). The log is also available from the UI if you go to Admin
page and then click 'Show CloudMan log' under 'System controls'. If you can
share that, we can hopefully figure out what's going on.

Best,
Enis

On Wed, Jun 17, 2015 at 11:47 AM, Nicholas Dickens <
nick.dick...@glasgow.ac.uk> wrote:

>   Hi All,
>
>  First time post, as a quick intro, I’m have some reasonable experience
> with EC2 & AWS for developing our own pipelines, I’m comfortable in python
> and *nix flavours and I am developing a completely custom galaxy in AWS for
> members of the WTCMP, Glasgow.  In the meantime, I have thrown up a quick
> cloudman galaxy using the cloudstart and the cloudman 2.3 ami
> (ami-a7dbf6ce) in us-east-1.  Auto–scaling didn’t seem to work so I’ve
> switched it off and added nodes manually, I tried various sizes including
> ‘same as master’ instances but they just don’t start – in the EC2 console I
> can see them and see them running.  But they’re constantly pending in the
> /cloud interface and in the log they reboot 4 times and then terminate –
> apparently not responding "10:16:56 - Instance i-xx not responding
> after 4 reboots. Terminating instance".
>
>  It’s out of the box, I editted the universe_wsgi.ini… file to disallow
> user registration and allow me to impersonate users but didn’t change
> anything else.  The only other configuration I’ve done is associate an
> elastic IP with the master instance so I can have a more static url for a
> couple of test users (if I need to destroy it and start again, etc).
>
>  I’m new to the system so don’t know which logs are best to check…and am
> I missing something obvious?  It there a known bug when using elastic IPs?
> I’ve googled but with no joy.
>
>  Thanks for your help and best wishes,
>
>  Nick
>  --
>  Nick Dickens
> DPhil BSc ARCS
>
>  Bioinformatics Team Leader
> Wellcome Trust Centre for Molecular Parasitology
> B6-21 SGDB
> 120 University Place
> Glasgow
> G12 8TA
>
>  Tel: +44 141 330 8282
>
>  http://fb.me/WTCMPbix
> @WTCMPbix
> http://www.gla.ac.uk/researchinstitutes/iii/staff/nickdickens/
>
> http://www.gla.ac.uk/researchinstitutes/iii/staff/jeremymottram/comparativegenomicsofleishmania/
>
>
> - Upozorenje -
>
> Automatskom detekcijom utvrdjeno je da tekst ove poruke
> podsjeca na tzv. phishing poruku.
>
>  AKO SE U PORUCI TRAZI DA POSALJETE VASU IRB LOZINKU ILI
> DA UNESETE IRB PODATKE NA NAVEDENOM LINKU, RADI SE O
> NAPADU S CILJEM KRADJE I ZLOUPOTREBE PODATAKA.
>
>  Centar za informatiku i racunarstvo,
> Institut Rudjer Boskovic
>
> - Upozorenje -
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/