Team,

I'm up and running fine, thanks.

Thanks Dannon for the info on 'migrated_tools_conf.xml.sample', that
allowed a normal update.
Thanks Ross for the info on coverage datatype definition from
datatypes_conf.xml.
Thanks Enis for the tip that a new clustername guarantees a clean new
Galaxy.

I've found something that may be of interest. A few days ago AWS EC2 added
an m1.small and m1.medium model for 64-bit AMI's, and Galaxy starts ok with
these. But it has a timing problem that I mentioned before. On m1.medium
the jobs sit in the History queue in the gray waiting state for 1 to 30
minutes before they run. They run ok after that. On m1.small they sit there
forever and don't run. Or they certainly don't run within 5 to 6 hours.
Things work ok (or used to anyway) on t1.micro so I'm guessing there's a
'case' statement on instance type somewhere that is falling through for
m1.medium and m1.small. (A wild guess but you know how it goes, it's
usually simple  :=)

One last gremlin. My new tools are added at the top of tool_conf.xml and
have always shown up at the top of the Tools list, in a local Galaxy and
previous Cloud Galaxies. Now they show up near the bottom of the tools
list. There's no obvious ordering happening like alphabetical tool name or
file-date or name of tools/tooldir. Any ideas here appreciated.

Thanks !

Greg E


On Sat, Mar 10, 2012 at 3:36 PM, Enis Afgan <afg...@gmail.com> wrote:

> Also Greg
> Starting a new cloudman instance with a different cluster name than
> previously used will always give you a clean environment and setup so you
> can continue to always use the us-east region.
>
> Cheers,
> Enis
> On 10/03/2012 2:08 PM, "Ross" <ross.laza...@gmail.com> wrote:
>
>> Since Dannon is likely asleep, maybe this will help until he can respond:
>>
>> http://dev.list.galaxyproject.org/error-on-fastq-file-while-grooming-or-fastqc-report-or-fastqc-converter-td4396664.html
>>
>> ie: try removing the second coverage datatype definition from
>> datatypes_conf.xml and restarting the Galaxy process.
>>
>> On Sat, Mar 10, 2012 at 2:00 PM, Greg Edwards <gedwar...@gmail.com>
>> wrote:
>> > Dannon,
>> >
>> > Thanks for that. Adding 'migrated_tools_conf.xml.sample' fixed up most
>> > problems. I'm left with a couple though.
>> >
>> > 1. The tools (apart from file upload with Get Data) don't initially
>> execute.
>> > This includes my new tools and existing ones like Text Manipulation /
>> Paste
>> > Files side-by-side. They stay in the grey "waiting to run" mode for
>> about
>> > 15-30 minutes, then run.
>> >
>> > 2. When they finally run, they compute and report fine but finish in
>> the Red
>> > error colour, and stderr has reported as below.This is both my tools
>> and the
>> > system standard ones.
>> >
>> >
>> > WARNING:galaxy.datatypes.registry:Overriding conflicting datatype with
>> > extension 'coverage', using datatype from /mnt/galaxyData/tmp/tmp6o5hnt.
>> >
>> >
>> > There's an oddity I should mention. When I first fire
>> > up galaxy-cloudman-2011-03-22, a remnant of my add-on tools is already
>> > showing at the top of the Tools menu. "PTM Tools" is there, and in
>> > tools_conf.xml. But the tools are not in tools/ptmtools, so the menu is
>> > inactive. It's like I've corrupted the file systems associated
>> > with galaxy-cloudman-2011-03-22.
>> >
>> > I'm thinking of trying it all on the Singapore AWS node where I can't
>> have
>> > touched the AMI's or file sets. This has all been on US East. In fact
>> it's
>> > been on US East 1d, I could try on 1a, I never go there normally, but
>> they
>> > probably share data. I could try on US West. I'll let you know, if I
>> don't
>> > hear before.
>> >
>> > Greg.
>> >
>> >
>> >
>> >
>> > On Sat, Mar 10, 2012 at 12:14 AM, Dannon Baker <dannonba...@me.com>
>> wrote:
>> >>
>> >> Greg,
>> >>
>> >> The problem here is that the galaxy update failed to merge a change to
>> >> run.sh because of minor customizations it has.  We'll have a long term
>> fix
>> >> out for this soon, but for now what you can do is ssh in to your
>> instance
>> >> and update run.sh yourself prior to restarting galaxy.  All you need
>> to do
>> >> is add 'migrated_tools_conf.xml.sample' to the SAMPLES in
>> >> /mnt/galaxyTools/galaxy-central/run.sh, execute `sh run.sh
>> --run-daemon` (or
>> >> restart galaxy again from the admin page) and you should be good to go.
>> >>
>> >> That new AMI you're seeing is not owned by the Galaxy Team, and we
>> don't
>> >> actually know who made it.  Keep using the same
>> galaxy-cloudman-2011-03-22
>> >> for now (and we'll always have the most up-to-date AMI listed at
>> >> usegalaxy.org/cloud).  Because of Cloudman's modular volume design
>> almost
>> >> nothing resides on the AMI itself, so we can (and do) update the tools
>> and
>> >> index volumes without having to touch it.  So while the AMI reflects a
>> date
>> >> of almost a year old, the galaxy tools volume (and thus the actual
>> Galaxy
>> >> instance you're running) has been updated much more recently.
>> >>
>> >> One last note- if you're updating and copying your tools in every time,
>> >> you may want to try using the 'Persist changes' functionality
>> available in
>> >> the Cloudman admin panel.  Once you've set your instance up how you
>> want, if
>> >> you click 'Persist changes to galaxyTools', it'll create a custom
>> snapshot
>> >> of your tools volume that will be used from that point forward with
>> this
>> >> instance.
>> >>
>> >> Let me know if you have any more issues,
>> >>
>> >> -Dannon
>> >>
>> >> On Mar 9, 2012, at 2:53 AM, Greg Edwards wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > I'm trying to restart my Galaxy Cloudman service, using the same
>> >> > approach that has been successful over the last couple of months ..
>> >> >
>> >> >  - launch AMI 861460482541/galaxy-cloudman-2011-03-22 as m1.large
>> >> >  - update from Cloudman console
>> >> >  - copy in my tools etc etc
>> >> >  - restart
>> >> >  - away we go, all works
>> >> >
>> >> > However today the update fails, the log says ...
>> >> >
>> >> >
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1:
>> >> > RuntimeWarning: __builtin__.file size changed, may indicate binary
>> >> > incompatibility
>> >> >   from csamtools import *
>> >> > python path is:
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/numpy-1.6.0-py2.6-linux-x86_64-ucs4.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/boto-2.2.2-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/Whoosh-0.3.18-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/pycrypto-2.0.1-py2.6-linux-x86_64-ucs4.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/python_lzo-1.08_2.03_static-py2.6-linux-x86_64-ucs4.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/amqplib-0.6.1-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/pexpect-2.4-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/Babel-0.9.4-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/Mako-0.4.1-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/WebHelpers-0.2-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/wchartype-0.1-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/docutils-0.7-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/WebOb-0.8.5-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/Routes-1.12.3-py2.6.egg,
>> >> >
>> /mnt/galaxyTools/galaxy-central/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/eggs/Paste-1.6-py2.6.egg,
>> >> > /mnt/galaxyTools/galaxy-central/lib, /usr/lib/python2.6/,
>> >> > /usr/lib/python2.6/plat-linux2, /usr/lib/python2.6/lib-tk,
>> >> > /usr/lib/python2.6/lib-old, /usr/lib/python2.6/lib-dynload
>> >> > Traceback (most recent call last):
>> >> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/web/buildapp.py",
>> >> > line 82, in app_factory
>> >> >     app = UniverseApplication( global_conf = global_conf, **kwargs )
>> >> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/app.py", line 24,
>> in
>> >> > __init__
>> >> >     self.config.check()
>> >> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/config.py", line
>> 243,
>> >> > in check
>> >> >     tree = parse_xml( config_filename )
>> >> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/util/__init__.py",
>> >> > line 105, in parse_xml
>> >> >     tree = ElementTree.parse(fname)
>> >> >   File
>> >> >
>> "/mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
>> >> > line 859, in parse
>> >> >     tree.parse(source, parser)
>> >> >   File
>> >> >
>> "/mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
>> >> > line 576, in parse
>> >> >     source = open(source, "rb")
>> >> > IOError: [Errno 2] No such file or directory:
>> >> > './migrated_tools_conf.xml'
>> >> > Removing PID file paster.pid
>> >> >
>> >> >
>> >> > While I'm here, I see a new Galaxy Cloudman AMI
>> >> >  072133624695/galaxy-cloudman-2012-02-26.  I can't manage to start
>> that, I
>> >> > get an error as below, with all types of instance,
>> >> > (tiny/small/medium/large). Is that a recommended AMI now ?  It would
>> be good
>> >> > to have a new updated AMI.
>> >> >
>> >> > <snap1.png>
>> >> >
>> >> > Thanks !
>> >> >
>> >> > --
>> >> > Greg Edwards,
>> >> > Port Jackson Bioinformatics
>> >> > gedwar...@gmail.com
>> >> >
>> >> > ___________________________________________________________
>> >> > The Galaxy User list should be used for the discussion of
>> >> > Galaxy analysis and other features on the public server
>> >> > at usegalaxy.org.  Please keep all replies on the list by
>> >> > using "reply all" in your mail client.  For discussion of
>> >> > local Galaxy instances and the Galaxy source code, please
>> >> > use the Galaxy Development list:
>> >> >
>> >> >  http://lists.bx.psu.edu/listinfo/galaxy-dev
>> >> >
>> >> > To manage your subscriptions to this and other Galaxy lists,
>> >> > please use the interface at:
>> >> >
>> >> >  http://lists.bx.psu.edu/
>> >>
>> >
>> >
>> >
>> > --
>> > Greg Edwards,
>> > Port Jackson Bioinformatics
>> > gedwar...@gmail.com
>> >
>> >
>> > ___________________________________________________________
>> > The Galaxy User list should be used for the discussion of
>> > Galaxy analysis and other features on the public server
>> > at usegalaxy.org.  Please keep all replies on the list by
>> > using "reply all" in your mail client.  For discussion of
>> > local Galaxy instances and the Galaxy source code, please
>> > use the Galaxy Development list:
>> >
>> >  http://lists.bx.psu.edu/listinfo/galaxy-dev
>> >
>> > To manage your subscriptions to this and other Galaxy lists,
>> > please use the interface at:
>> >
>> >  http://lists.bx.psu.edu/
>>
>>
>>
>> --
>> Ross Lazarus MBBS MPH;
>> Associate Professor, Harvard Medical School;
>> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;
>>
>> ___________________________________________________________
>> The Galaxy User list should be used for the discussion of
>> Galaxy analysis and other features on the public server
>> at usegalaxy.org.  Please keep all replies on the list by
>> using "reply all" in your mail client.  For discussion of
>> local Galaxy instances and the Galaxy source code, please
>> use the Galaxy Development list:
>>
>>  http://lists.bx.psu.edu/listinfo/galaxy-dev
>>
>> To manage your subscriptions to this and other Galaxy lists,
>> please use the interface at:
>>
>>  http://lists.bx.psu.edu/
>>
>


-- 
Greg Edwards,
Port Jackson Bioinformatics
gedwar...@gmail.com
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to