Re: [galaxy-dev] Once-run galaxy archives

[Peter Cock]

On Wed, Jul 23, 2014 at 2:42 PM, John Chilton  wrote:
> Problem with automation is I could create dozens of templates over the
> next several years and consume less time in aggregate than it would
> take me to automate this. Nonetheless, there is a documentation
> component here that is important so I did enough to document - if
> someone wants to automate from there feel free.
>
> The template is just a copy of the sqlite database after a fresh
> Galaxy is launched. I usually just do this against whatever
> development instance of Galaxy I am working on. For completeness
> though I have put together a script to automate this task against a
> fresh install (I think):
>
> https://github.com/jmchilton/galaxy-downloads/blob/master/build_sqlite_template.sh
>
> Good luck!
>
> -John

Perfect - documentation target achieved :)

In terms of speeding up things like TravisCI using these SQLite
database templates, refreshing this every few schema bumps
should be enough.

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[John Chilton]

Problem with automation is I could create dozens of templates over the
next several years and consume less time in aggregate than it would
take me to automate this. Nonetheless, there is a documentation
component here that is important so I did enough to document - if
someone wants to automate from there feel free.

The template is just a copy of the sqlite database after a fresh
Galaxy is launched. I usually just do this against whatever
development instance of Galaxy I am working on. For completeness
though I have put together a script to automate this task against a
fresh install (I think):

https://github.com/jmchilton/galaxy-downloads/blob/master/build_sqlite_template.sh

Good luck!

-John



On Tue, Jul 22, 2014 at 4:13 PM, Peter Cock  wrote:
> On Tue, Jul 22, 2014 at 7:41 PM, Eric Rasche  wrote:
>> John,
>>
>> How are those generated? Would you be amenable to scripting that
>> portion and running it once a month? (...say in a cron job, with a
>> passwordless ssh key so you never have to touch it again)
>>
>> Cheers,
>> Eric
>
> How to generate it was going to be my next question too ;)
>
> I'm impressed with Eric's zeal to automate things. Having a script
> for making the SQLite template would be good - under git in the
> same repository?
>
> Peter
>
> P.S. The schema version 120 template works great, thanks!:
>
> https://travis-ci.org/peterjc/pico_galaxy/builds/30592828
> https://travis-ci.org/peterjc/galaxy_blast/builds/30592097
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Peter Cock]

On Tue, Jul 22, 2014 at 7:41 PM, Eric Rasche  wrote:
> John,
>
> How are those generated? Would you be amenable to scripting that
> portion and running it once a month? (...say in a cron job, with a
> passwordless ssh key so you never have to touch it again)
>
> Cheers,
> Eric

How to generate it was going to be my next question too ;)

I'm impressed with Eric's zeal to automate things. Having a script
for making the SQLite template would be good - under git in the
same repository?

Peter

P.S. The schema version 120 template works great, thanks!:

https://travis-ci.org/peterjc/pico_galaxy/builds/30592828
https://travis-ci.org/peterjc/galaxy_blast/builds/30592097
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

John,

How are those generated? Would you be amenable to scripting that portion and 
running it once a month? (...say in a cron job, with a passwordless ssh key so 
you never have to touch it again)

Cheers,
Eric

22.07.2014, 19:08, "John Chilton" :
> On Tue, Jul 22, 2014 at 7:51 AM, Peter Cock  wrote:
>>  On Tue, Jul 22, 2014 at 1:15 PM, Eric Rasche  wrote:
>>>  Hi Peter,
>>>
>>>  On July 22, 2014 3:15:41 AM CDT, Peter Cock  
>>> wrote:
 Given how close you can get now for minimal effort,
 this seem unnecessary.

 http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

 My TravisCI setup this fetches the latest Galaxy as
 a tar ball (from a GitHub mirror as it was faster than
 a git clone which was faster than getting the tar ball
 from BitBucket, which in turn was faster than using
 hg clone),
>>>  Yes, that post was at least part of the thinking behind this.
>>  :)
  .., and a per-migrated SQLite database
 (using a bit of Galaxy functionality originally with
 $GALAXY_TEST_DB_TEMPLATE added to speed
 up running the functional tests).
>>  Apologies for grammatical error - I pasted in the environment
>>  variable at the wrong point in the sentence.
>>>  I know I've seen that used but was never able to get that
>>>  working in practice (then again I didn't try that hard). If
>>>  that's a working/usable feature, then that is already the
>>>  majority of setup time.
>>  Yes, the creation of the test-database and all the migrations
>>  was an obvious low-hanging fruit when we were looking at
>>  making running the tool functional tests faster - although
>>  originally in the context of running the tests on a local
>>  development Galaxy instance.
>>
>>  As to using this in practise, currently my TravisCI setup has:
>>
>>  export 
>> GALAXY_TEST_DB_TEMPLATE=https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite
>>
>>  I also added that line at the start of my local copy of script
>>  run_functional_tests.sh to benefit from this while doing
>>  development. That should be all there is to it (but from
>>  memory, this is only for use with the SQLite backend).
>>
>>  John - could you add a current schema snapshot to
>>  https://github.com/jmchilton/galaxy-downloads/ please?
>
> Hey All,
>
> Love this thread and effort! Keep up the good work - would love to
> replace say blend4j's automatic travisci testing to be backed by
> dockerized -stable and -central instance.
>
> At any rate, I have uploaded a more updated sqlite template:
> https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0120.sqlite.
> The old template still exists at the same URL so hopefully this
> doesn't break anything.
>
> -John
 Note this does not cache the eggs and all the other
 side effects of the first run like creating config files,
 so there is room for some speed up.
>>>  Eggs would be nice but not the biggest thing in the world.
>>  Right. I do like your idea of automatically generated
>>  cutting-edge or each stable release Docker images
>>  though (even if I have no personal need for them at
>>  the moment).
>>
>>  Regards,
>>
>>  Peter

-- 
Eric Rasche
Programmer II
Center for Phage Technology
Texas A&M Univesity
College Station, TX 77843
Ph: 4046922048

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[John Chilton]

On Tue, Jul 22, 2014 at 7:51 AM, Peter Cock  wrote:
> On Tue, Jul 22, 2014 at 1:15 PM, Eric Rasche  wrote:
>> Hi Peter,
>>
>> On July 22, 2014 3:15:41 AM CDT, Peter Cock  
>> wrote:
>>>
>>>Given how close you can get now for minimal effort,
>>>this seem unnecessary.
>>>
>>>http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>>
>>>My TravisCI setup this fetches the latest Galaxy as
>>>a tar ball (from a GitHub mirror as it was faster than
>>>a git clone which was faster than getting the tar ball
>>>from BitBucket, which in turn was faster than using
>>>hg clone),
>>
>> Yes, that post was at least part of the thinking behind this.
>
> :)
>
>>> .., and a per-migrated SQLite database
>>>(using a bit of Galaxy functionality originally with
>>>$GALAXY_TEST_DB_TEMPLATE added to speed
>>>up running the functional tests).
>
> Apologies for grammatical error - I pasted in the environment
> variable at the wrong point in the sentence.
>
>> I know I've seen that used but was never able to get that
>> working in practice (then again I didn't try that hard). If
>> that's a working/usable feature, then that is already the
>> majority of setup time.
>
> Yes, the creation of the test-database and all the migrations
> was an obvious low-hanging fruit when we were looking at
> making running the tool functional tests faster - although
> originally in the context of running the tests on a local
> development Galaxy instance.
>
> As to using this in practise, currently my TravisCI setup has:
>
> export 
> GALAXY_TEST_DB_TEMPLATE=https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite
>
> I also added that line at the start of my local copy of script
> run_functional_tests.sh to benefit from this while doing
> development. That should be all there is to it (but from
> memory, this is only for use with the SQLite backend).
>
> John - could you add a current schema snapshot to
> https://github.com/jmchilton/galaxy-downloads/ please?

Hey All,

Love this thread and effort! Keep up the good work - would love to
replace say blend4j's automatic travisci testing to be backed by
dockerized -stable and -central instance.

At any rate, I have uploaded a more updated sqlite template:
https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0120.sqlite.
The old template still exists at the same URL so hopefully this
doesn't break anything.

-John

>
>>>Note this does not cache the eggs and all the other
>>>side effects of the first run like creating config files,
>>>so there is room for some speed up.
>>
>> Eggs would be nice but not the biggest thing in the world.
>
> Right. I do like your idea of automatically generated
> cutting-edge or each stable release Docker images
> though (even if I have no personal need for them at
> the moment).
>
> Regards,
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

For my part:Script/infra to generate brancheshttps://cpt.tamu.edu/gitlab/rasche.eric/docker-branch-generator/tree/masterGenerated branches:https://cpt.tamu.edu/gitlab/rasche.eric/generated-docker-branches/branches/recentNeed to patch up a couple issues, but I'm pretty much done on my end. Whenever you're ready, we can move this to github and someone can just run a cron job once a month. :)22.07.2014, 15:35, "Björn Grüning" : :) great I like it! Will do it shortly! Am 22.07.2014 15:36, schrieb Eric Rasche:  Hi Björn,  22.07.2014, 14:26, "Björn Grüning" :Hi Eric, That sounds like a pretty good idea.  If there was a pre-built image available for whatever release I wanted to test against I could just cache it and (hopefully) get my tests running a bit faster.  I'm not sure if anyone else is already doing this? Also, I remember there being mentioned pre-building docker images for each release of Galaxy, which would accomplish something similar, but I'm not really sure how that's being handled.  I think Björn's Docker image is kept up to date with Galaxy stable each time it's built https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51. So, this could be handled by modifying his Dockerfile to build Galaxy at whatever tagged release you want to test against. I will try hard to create with every Galaxy stable release a new Galaxy docker image. You can create a github branch with a specific tag, that will end up as a new tagged version of the main Galaxy Docker image. Try hard to create? No no, what can we do to automatically create these? I'm not so familiar with how one might build a galaxy release specific docker image, but if you can provide a generalized process, let's stick it in a CI server/cron job somewhere and never worry about it again!The hardest part is to remind myself ;)The procedure is:1) create a new branch for the galaxy-docker github account2) change the version string in the git-clone command in the Dockerfile3) login into the docker-index site and re-associate a new tag to thenew branch ... click the build buttonI could simplify that a little bit if the galaxy-docker image has itsown repository. The docker-index has a build-on-push feature.But currently every image (all branches) are build again. There is noway to only trigger one branch build. Until that is fixed in thedocker-index I will do it manually.So you see any improvements in that setup? Let me know!  Naturally! 1 and 2 could be automated out with a script. 3 could probably be fixed with a script, but that requires parsing pages/crafting cURL queries and that's less pleasant.  Let's have a new repository just for galaxy-docker images. I'll write up a script to check for updates and create+push branches as needed, we can place this in a cron/CI job and have it email you whenever it's run to say "hey, associate the branch/trigger a build"Cheers,Bjoern One downside to docker is that you need to get it installed on your CI server, which may or may not be possible (needs a very recent kernel for example). So true. SL7 for the win! :) Docker, Docker, Docker! Bjoern Aaron On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche  wrote: Hi Aaron, Yeah, absolutely understandable. I want my tools tested early and often. I abuse my CI server for everything, especially for building and packaging software. In this case I was imagining that I might have it produce an archive on every tagged release, as well as producing a "daily" archive. All of these would be available on some ftp/http server somewhere with symlinks for latest archives (e.g. galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that work for your use case as well? Eric On 07/21/2014 03:02 PM, Aaron Petkau wrote: Hello Eric, Your right about that, downloading the archive, installing all the eggs, and then updating the database takes a bit of time (especially if you're like me and like re-running tests on nearly every change you make :P). I think it would be cool to have a pre-package Galaxy for integration testing which is quick to setup.  I once thought of downloading Björn's Docker image from Galaxy Bootstrap and using it that way, but thinking is about as far as I got with that one.  One problem I could see is it would have to be re-built on every release of Galaxy you want to test against (whereas mercurial cloning/pulling makes sure you're always up to date with the latest code). Aaron On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche > wrote:   Hi Aaron,   Good points, I was considering using g

Re: [galaxy-dev] Once-run galaxy archives

[Aaron Petkau]

That's awesome Björn and Eric.  And, I'll also have to go through your
Travis CI document Peter.  It looks really cool.

Aaron


On Tue, Jul 22, 2014 at 9:35 AM, Björn Grüning 
wrote:

> :) great I like it!
> Will do it shortly!
>
> Am 22.07.2014 15:36, schrieb Eric Rasche:
>
>  Hi Björn,
>>
>> 22.07.2014, 14:26, "Björn Grüning" :
>>
>>>   Hi Eric,
>>>
That sounds like a pretty good idea.  If there was a pre-built image
>>available for whatever release I wanted to test against I could
>> just
>>
>cache
>
>>it and (hopefully) get my tests running a bit faster.  I'm not sure
>>
>if
>
>>anyone else is already doing this?
>>
>>Also, I remember there being mentioned pre-building docker images
>> for
>>
>each
>
>>release of Galaxy, which would accomplish something similar, but
>> I'm
>>
>not
>
>>really sure how that's being handled.  I think Björn's Docker image
>>
>is kept
>
>>up to date with Galaxy stable each time it's built
>>
>https://github.com/bgruening/docker-recipes/blob/master/
> galaxy/Dockerfile#L51.
>
>>So, this could be handled by modifying his Dockerfile to build
>> Galaxy
>>
>at
>
>>whatever tagged release you want to test against.
>>
>I will try hard to create with every Galaxy stable release a new
> Galaxy
>
>docker image. You can create a github branch with a specific tag,
> that
>will end up as a new tagged version of the main Galaxy Docker image.
>
Try hard to create? No no, what can we do to automatically create
 these? I'm not so familiar with how one might build a galaxy release
 specific docker image, but if you can provide a generalized process, let's
 stick it in a CI server/cron job somewhere and never worry about it again!

>>>   The hardest part is to remind myself ;)
>>>   The procedure is:
>>>
>>>   1) create a new branch for the galaxy-docker github account
>>>   2) change the version string in the git-clone command in the Dockerfile
>>>   3) login into the docker-index site and re-associate a new tag to the
>>>   new branch ... click the build button
>>>
>>>   I could simplify that a little bit if the galaxy-docker image has its
>>>   own repository. The docker-index has a build-on-push feature.
>>>   But currently every image (all branches) are build again. There is no
>>>   way to only trigger one branch build. Until that is fixed in the
>>>   docker-index I will do it manually.
>>>
>>>   So you see any improvements in that setup? Let me know!
>>>
>>
>> Naturally! 1 and 2 could be automated out with a script. 3 could probably
>> be fixed with a script, but that requires parsing pages/crafting cURL
>> queries and that's less pleasant.
>>
>> Let's have a new repository just for galaxy-docker images. I'll write up
>> a script to check for updates and create+push branches as needed, we can
>> place this in a cron/CI job and have it email you whenever it's run to say
>> "hey, associate the branch/trigger a build"
>>
>>Cheers,
>>>   Bjoern
>>>
One downside to docker is that you need to get it installed on your
>>
>CI
>
>>server, which may or may not be possible (needs a very recent
>> kernel
>>
>for
>
>>example).
>>
>So true. SL7 for the win! :)
>
>Docker, Docker, Docker!
>Bjoern
>
>>Aaron
>>
>>On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche <
>> rasche.e...@yandex.ru>
>>
>wrote:
>
>>Hi Aaron,
>>>
>>>Yeah, absolutely understandable. I want my tools tested early and
>>>
>>often.
>
>>I abuse my CI server for everything, especially for building and
>>>packaging software. In this case I was imagining that I might have
>>>
>>it
>
>>produce an archive on every tagged release, as well as producing a
>>>"daily" archive. All of these would be available on some ftp/http
>>>
>>server
>
>>somewhere with symlinks for latest archives (e.g.
>>>galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that
>>>
>>work
>
>>for your use case as well?
>>>
>>>Eric
>>>
>>>On 07/21/2014 03:02 PM, Aaron Petkau wrote:
>>>
Hello Eric,

Your right about that, downloading the archive, installing all
 the

>>>eggs,
>
>>and then updating the database takes a bit of time (especially if

>>>you're
>
>>like me and like re-running tests on nearly every change you make

>>>:P).
>
>>I think it would be cool to have a pre-package Galaxy for

>>>integration
>
>>testing which is quick to se

Re: [galaxy-dev] Once-run galaxy archives

[Björn Grüning]

:) great I like it!
Will do it shortly!

Am 22.07.2014 15:36, schrieb Eric Rasche:

Hi Björn,

22.07.2014, 14:26, "Björn Grüning" :

  Hi Eric,

   That sounds like a pretty good idea.  If there was a pre-built image
   available for whatever release I wanted to test against I could just

   cache

   it and (hopefully) get my tests running a bit faster.  I'm not sure

   if

   anyone else is already doing this?

   Also, I remember there being mentioned pre-building docker images for

   each

   release of Galaxy, which would accomplish something similar, but I'm

   not

   really sure how that's being handled.  I think Björn's Docker image

   is kept

   up to date with Galaxy stable each time it's built

   
https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.

   So, this could be handled by modifying his Dockerfile to build Galaxy

   at

   whatever tagged release you want to test against.

   I will try hard to create with every Galaxy stable release a new Galaxy

   docker image. You can create a github branch with a specific tag, that
   will end up as a new tagged version of the main Galaxy Docker image.

   Try hard to create? No no, what can we do to automatically create these? I'm 
not so familiar with how one might build a galaxy release specific docker 
image, but if you can provide a generalized process, let's stick it in a CI 
server/cron job somewhere and never worry about it again!

  The hardest part is to remind myself ;)
  The procedure is:

  1) create a new branch for the galaxy-docker github account
  2) change the version string in the git-clone command in the Dockerfile
  3) login into the docker-index site and re-associate a new tag to the
  new branch ... click the build button

  I could simplify that a little bit if the galaxy-docker image has its
  own repository. The docker-index has a build-on-push feature.
  But currently every image (all branches) are build again. There is no
  way to only trigger one branch build. Until that is fixed in the
  docker-index I will do it manually.

  So you see any improvements in that setup? Let me know!


Naturally! 1 and 2 could be automated out with a script. 3 could probably be 
fixed with a script, but that requires parsing pages/crafting cURL queries and 
that's less pleasant.

Let's have a new repository just for galaxy-docker images. I'll write up a script to 
check for updates and create+push branches as needed, we can place this in a cron/CI job 
and have it email you whenever it's run to say "hey, associate the branch/trigger a 
build"


  Cheers,
  Bjoern

   One downside to docker is that you need to get it installed on your

   CI

   server, which may or may not be possible (needs a very recent kernel

   for

   example).

   So true. SL7 for the win! :)

   Docker, Docker, Docker!
   Bjoern

   Aaron

   On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche 

   wrote:

   Hi Aaron,

   Yeah, absolutely understandable. I want my tools tested early and

   often.

   I abuse my CI server for everything, especially for building and
   packaging software. In this case I was imagining that I might have

   it

   produce an archive on every tagged release, as well as producing a
   "daily" archive. All of these would be available on some ftp/http

   server

   somewhere with symlinks for latest archives (e.g.
   galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that

   work

   for your use case as well?

   Eric

   On 07/21/2014 03:02 PM, Aaron Petkau wrote:

   Hello Eric,

   Your right about that, downloading the archive, installing all the

   eggs,

   and then updating the database takes a bit of time (especially if

   you're

   like me and like re-running tests on nearly every change you make

   :P).

   I think it would be cool to have a pre-package Galaxy for

   integration

   testing which is quick to setup.  I once thought of downloading

   Björn's

   Docker image from Galaxy Bootstrap and using it that way, but

   thinking

   is about as far as I got with that one.  One problem I could see is

   it

   would have to be re-built on every release of Galaxy you want to

   test

   against (whereas mercurial cloning/pulling makes sure you're always

   up

   to date with the latest code).

   Aaron

   On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche mailto:rasche.e...@yandex.ru>> wrote:

 Hi Aaron,

 Good points, I was considering using galaxy bootstrap. This is
 mostly for the CI folk who want to download an archive, unpack

   it,

 and be ready to install/test their tools. The hg clone and

   egg/db

 steps seem like unnecessary overhead for this specific use

   case.

 Cheers,

 Eric

   ___
   Please keep all replies on the list by using "reply all"
   in your mail client.  To manage your subscriptions to this
   and other Galaxy lists, please use the int

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

Hi Björn,

22.07.2014, 14:26, "Björn Grüning" :
>  Hi Eric,
   That sounds like a pretty good idea.  If there was a pre-built image
   available for whatever release I wanted to test against I could just
>>>   cache
   it and (hopefully) get my tests running a bit faster.  I'm not sure
>>>   if
   anyone else is already doing this?

   Also, I remember there being mentioned pre-building docker images for
>>>   each
   release of Galaxy, which would accomplish something similar, but I'm
>>>   not
   really sure how that's being handled.  I think Björn's Docker image
>>>   is kept
   up to date with Galaxy stable each time it's built
>>>   
>>> https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.
   So, this could be handled by modifying his Dockerfile to build Galaxy
>>>   at
   whatever tagged release you want to test against.
>>>   I will try hard to create with every Galaxy stable release a new Galaxy
>>>
>>>   docker image. You can create a github branch with a specific tag, that
>>>   will end up as a new tagged version of the main Galaxy Docker image.
>>   Try hard to create? No no, what can we do to automatically create these? 
>> I'm not so familiar with how one might build a galaxy release specific 
>> docker image, but if you can provide a generalized process, let's stick it 
>> in a CI server/cron job somewhere and never worry about it again!
>  The hardest part is to remind myself ;)
>  The procedure is:
>
>  1) create a new branch for the galaxy-docker github account
>  2) change the version string in the git-clone command in the Dockerfile
>  3) login into the docker-index site and re-associate a new tag to the
>  new branch ... click the build button
>
>  I could simplify that a little bit if the galaxy-docker image has its
>  own repository. The docker-index has a build-on-push feature.
>  But currently every image (all branches) are build again. There is no
>  way to only trigger one branch build. Until that is fixed in the
>  docker-index I will do it manually.
>
>  So you see any improvements in that setup? Let me know!

Naturally! 1 and 2 could be automated out with a script. 3 could probably be 
fixed with a script, but that requires parsing pages/crafting cURL queries and 
that's less pleasant.

Let's have a new repository just for galaxy-docker images. I'll write up a 
script to check for updates and create+push branches as needed, we can place 
this in a cron/CI job and have it email you whenever it's run to say "hey, 
associate the branch/trigger a build"

>  Cheers,
>  Bjoern
   One downside to docker is that you need to get it installed on your
>>>   CI
   server, which may or may not be possible (needs a very recent kernel
>>>   for
   example).
>>>   So true. SL7 for the win! :)
>>>
>>>   Docker, Docker, Docker!
>>>   Bjoern
   Aaron

   On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche 
>>>   wrote:
>   Hi Aaron,
>
>   Yeah, absolutely understandable. I want my tools tested early and
>>>   often.
>   I abuse my CI server for everything, especially for building and
>   packaging software. In this case I was imagining that I might have
>>>   it
>   produce an archive on every tagged release, as well as producing a
>   "daily" archive. All of these would be available on some ftp/http
>>>   server
>   somewhere with symlinks for latest archives (e.g.
>   galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that
>>>   work
>   for your use case as well?
>
>   Eric
>
>   On 07/21/2014 03:02 PM, Aaron Petkau wrote:
>>   Hello Eric,
>>
>>   Your right about that, downloading the archive, installing all the
>>>   eggs,
>>   and then updating the database takes a bit of time (especially if
>>>   you're
>>   like me and like re-running tests on nearly every change you make
>>>   :P).
>>   I think it would be cool to have a pre-package Galaxy for
>>>   integration
>>   testing which is quick to setup.  I once thought of downloading
>>>   Björn's
>>   Docker image from Galaxy Bootstrap and using it that way, but
>>>   thinking
>>   is about as far as I got with that one.  One problem I could see is
>>>   it
>>   would have to be re-built on every release of Galaxy you want to
>>>   test
>>   against (whereas mercurial cloning/pulling makes sure you're always
>>>   up
>>   to date with the latest code).
>>
>>   Aaron
>>
>>   On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche >   > wrote:
>>
>> Hi Aaron,
>>
>> Good points, I was considering using galaxy bootstrap. This is
>> mostly for the CI folk who want to download an archive, unpack
>>>   it,
>> and be ready to install/test their tools. The hg clone and
>>>   egg/db
>> steps seem like unnecessary overhead for this specific use
>>>

Re: [galaxy-dev] Once-run galaxy archives

[Björn Grüning]

Hi Eric,


That sounds like a pretty good idea.  If there was a pre-built image
available for whatever release I wanted to test against I could just

cache

it and (hopefully) get my tests running a bit faster.  I'm not sure

if

anyone else is already doing this?

Also, I remember there being mentioned pre-building docker images for

each

release of Galaxy, which would accomplish something similar, but I'm

not

really sure how that's being handled.  I think Björn's Docker image

is kept

up to date with Galaxy stable each time it's built


https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.

So, this could be handled by modifying his Dockerfile to build Galaxy

at

whatever tagged release you want to test against.


I will try hard to create with every Galaxy stable release a new Galaxy

docker image. You can create a github branch with a specific tag, that
will end up as a new tagged version of the main Galaxy Docker image.


Try hard to create? No no, what can we do to automatically create these? I'm 
not so familiar with how one might build a galaxy release specific docker 
image, but if you can provide a generalized process, let's stick it in a CI 
server/cron job somewhere and never worry about it again!


The hardest part is to remind myself ;)
The procedure is:

1) create a new branch for the galaxy-docker github account
2) change the version string in the git-clone command in the Dockerfile
3) login into the docker-index site and re-associate a new tag to the 
new branch ... click the build button


I could simplify that a little bit if the galaxy-docker image has its 
own repository. The docker-index has a build-on-push feature.
But currently every image (all branches) are build again. There is no 
way to only trigger one branch build. Until that is fixed in the 
docker-index I will do it manually.


So you see any improvements in that setup? Let me know!
Cheers,
Bjoern


One downside to docker is that you need to get it installed on your

CI

server, which may or may not be possible (needs a very recent kernel

for

example).


So true. SL7 for the win! :)

Docker, Docker, Docker!
Bjoern


Aaron


On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche 

wrote:



Hi Aaron,

Yeah, absolutely understandable. I want my tools tested early and

often.


I abuse my CI server for everything, especially for building and
packaging software. In this case I was imagining that I might have

it

produce an archive on every tagged release, as well as producing a
"daily" archive. All of these would be available on some ftp/http

server

somewhere with symlinks for latest archives (e.g.
galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that

work

for your use case as well?

Eric

On 07/21/2014 03:02 PM, Aaron Petkau wrote:

Hello Eric,

Your right about that, downloading the archive, installing all the

eggs,

and then updating the database takes a bit of time (especially if

you're

like me and like re-running tests on nearly every change you make

:P).

I think it would be cool to have a pre-package Galaxy for

integration

testing which is quick to setup.  I once thought of downloading

Björn's

Docker image from Galaxy Bootstrap and using it that way, but

thinking

is about as far as I got with that one.  One problem I could see is

it

would have to be re-built on every release of Galaxy you want to

test

against (whereas mercurial cloning/pulling makes sure you're always

up

to date with the latest code).

Aaron


On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche mailto:rasche.e...@yandex.ru>> wrote:

  Hi Aaron,

  Good points, I was considering using galaxy bootstrap. This is
  mostly for the CI folk who want to download an archive, unpack

it,

  and be ready to install/test their tools. The hg clone and

egg/db

  steps seem like unnecessary overhead for this specific use

case.


  Cheers,

  Eric








___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Peter Cock]

On Tue, Jul 22, 2014 at 1:15 PM, Eric Rasche  wrote:
> Hi Peter,
>
> On July 22, 2014 3:15:41 AM CDT, Peter Cock  wrote:
>>
>>Given how close you can get now for minimal effort,
>>this seem unnecessary.
>>
>>http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>
>>My TravisCI setup this fetches the latest Galaxy as
>>a tar ball (from a GitHub mirror as it was faster than
>>a git clone which was faster than getting the tar ball
>>from BitBucket, which in turn was faster than using
>>hg clone),
>
> Yes, that post was at least part of the thinking behind this.

:)

>> .., and a per-migrated SQLite database
>>(using a bit of Galaxy functionality originally with
>>$GALAXY_TEST_DB_TEMPLATE added to speed
>>up running the functional tests).

Apologies for grammatical error - I pasted in the environment
variable at the wrong point in the sentence.

> I know I've seen that used but was never able to get that
> working in practice (then again I didn't try that hard). If
> that's a working/usable feature, then that is already the
> majority of setup time.

Yes, the creation of the test-database and all the migrations
was an obvious low-hanging fruit when we were looking at
making running the tool functional tests faster - although
originally in the context of running the tests on a local
development Galaxy instance.

As to using this in practise, currently my TravisCI setup has:

export 
GALAXY_TEST_DB_TEMPLATE=https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite

I also added that line at the start of my local copy of script
run_functional_tests.sh to benefit from this while doing
development. That should be all there is to it (but from
memory, this is only for use with the SQLite backend).

John - could you add a current schema snapshot to
https://github.com/jmchilton/galaxy-downloads/ please?

>>Note this does not cache the eggs and all the other
>>side effects of the first run like creating config files,
>>so there is room for some speed up.
>
> Eggs would be nice but not the biggest thing in the world.

Right. I do like your idea of automatically generated
cutting-edge or each stable release Docker images
though (even if I have no personal need for them at
the moment).

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

Hi Peter,

On July 22, 2014 3:15:41 AM CDT, Peter Cock  wrote:
>On Mon, Jul 21, 2014 at 6:51 PM, Eric Rasche 
>wrote:
>> Currently the checkout options consist of hg clones, and archives
>that
>> mercurial produces.
>>
>> Having pulled or cloned galaxy a few times lately, I'm wondering if
>anyone
>> would have a use for a once-run galaxy instance in an archive? I.e.,
>I'd
>> clone, run once to grab eggs and do the db migration, then re-tar
>result and
>> store online. Might cut down on build/test times for those who are
>using
>> travis or other CIs. Thoughts/opinions?
>
>Hi Eric,
>
>Given how close you can get now for minimal effort,
>this seem unnecessary.
>
>http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>
>My TravisCI setup this fetches the latest Galaxy as
>a tar ball (from a GitHub mirror as it was faster than
>a git clone which was faster than getting the tar ball
>from BitBucket, which in turn was faster than using

Yes, that post was at least part of the thinking behind this.

>hg clone), and a per-migrated SQLite database
>(using a bit of Galaxy functionality originally with
>$GALAXY_TEST_DB_TEMPLATE added to speed
>up running the functional tests).

I know I've seen that used but was never able to get that working in practice 
(then again I didn't try that hard). If that's a working/usable feature, then 
that is already the majority of setup time. Eggs would be nice but not the 
biggest thing in the world.

>Note this does not cache the eggs and all the other
>side effects of the first run like creating config files,
>so there is room for some speed up.
>
>Regards,
>
>Peter

Cheers,
Eric
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

Hi Björn,

On July 22, 2014 3:17:38 AM CDT, "Björn Grüning"  
wrote:
>Hi Aaron and Eric,
>
>Am 21.07.2014 22:58, schrieb Aaron Petkau:
>> Hello Eric,
>>
>> That sounds like a pretty good idea.  If there was a pre-built image
>> available for whatever release I wanted to test against I could just
>cache
>> it and (hopefully) get my tests running a bit faster.  I'm not sure
>if
>> anyone else is already doing this?
>>
>> Also, I remember there being mentioned pre-building docker images for
>each
>> release of Galaxy, which would accomplish something similar, but I'm
>not
>> really sure how that's being handled.  I think Björn's Docker image
>is kept
>> up to date with Galaxy stable each time it's built
>>
>https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.
>> So, this could be handled by modifying his Dockerfile to build Galaxy
>at
>> whatever tagged release you want to test against.
>
>I will try hard to create with every Galaxy stable release a new Galaxy
>
>docker image. You can create a github branch with a specific tag, that 
>will end up as a new tagged version of the main Galaxy Docker image.

Try hard to create? No no, what can we do to automatically create these? I'm 
not so familiar with how one might build a galaxy release specific docker 
image, but if you can provide a generalized process, let's stick it in a CI 
server/cron job somewhere and never worry about it again! 

>> One downside to docker is that you need to get it installed on your
>CI
>> server, which may or may not be possible (needs a very recent kernel
>for
>> example).
>
>So true. SL7 for the win! :)
>
>Docker, Docker, Docker!
>Bjoern
>
>> Aaron
>>
>>
>> On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche 
>wrote:
>>
>>> Hi Aaron,
>>>
>>> Yeah, absolutely understandable. I want my tools tested early and
>often.
>>>
>>> I abuse my CI server for everything, especially for building and
>>> packaging software. In this case I was imagining that I might have
>it
>>> produce an archive on every tagged release, as well as producing a
>>> "daily" archive. All of these would be available on some ftp/http
>server
>>> somewhere with symlinks for latest archives (e.g.
>>> galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that
>work
>>> for your use case as well?
>>>
>>> Eric
>>>
>>> On 07/21/2014 03:02 PM, Aaron Petkau wrote:
 Hello Eric,

 Your right about that, downloading the archive, installing all the
>eggs,
 and then updating the database takes a bit of time (especially if
>you're
 like me and like re-running tests on nearly every change you make
>:P).
 I think it would be cool to have a pre-package Galaxy for
>integration
 testing which is quick to setup.  I once thought of downloading
>Björn's
 Docker image from Galaxy Bootstrap and using it that way, but
>thinking
 is about as far as I got with that one.  One problem I could see is
>it
 would have to be re-built on every release of Galaxy you want to
>test
 against (whereas mercurial cloning/pulling makes sure you're always
>up
 to date with the latest code).

 Aaron


 On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche >>> > wrote:

  Hi Aaron,

  Good points, I was considering using galaxy bootstrap. This is
  mostly for the CI folk who want to download an archive, unpack
>it,
  and be ready to install/test their tools. The hg clone and
>egg/db
  steps seem like unnecessary overhead for this specific use
>case.

  Cheers,

  Eric


>>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Björn Grüning]

Hi Aaron and Eric,

Am 21.07.2014 22:58, schrieb Aaron Petkau:

Hello Eric,

That sounds like a pretty good idea.  If there was a pre-built image
available for whatever release I wanted to test against I could just cache
it and (hopefully) get my tests running a bit faster.  I'm not sure if
anyone else is already doing this?

Also, I remember there being mentioned pre-building docker images for each
release of Galaxy, which would accomplish something similar, but I'm not
really sure how that's being handled.  I think Björn's Docker image is kept
up to date with Galaxy stable each time it's built
https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.
So, this could be handled by modifying his Dockerfile to build Galaxy at
whatever tagged release you want to test against.


I will try hard to create with every Galaxy stable release a new Galaxy 
docker image. You can create a github branch with a specific tag, that 
will end up as a new tagged version of the main Galaxy Docker image.



One downside to docker is that you need to get it installed on your CI
server, which may or may not be possible (needs a very recent kernel for
example).


So true. SL7 for the win! :)

Docker, Docker, Docker!
Bjoern


Aaron


On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche  wrote:


Hi Aaron,

Yeah, absolutely understandable. I want my tools tested early and often.

I abuse my CI server for everything, especially for building and
packaging software. In this case I was imagining that I might have it
produce an archive on every tagged release, as well as producing a
"daily" archive. All of these would be available on some ftp/http server
somewhere with symlinks for latest archives (e.g.
galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that work
for your use case as well?

Eric

On 07/21/2014 03:02 PM, Aaron Petkau wrote:

Hello Eric,

Your right about that, downloading the archive, installing all the eggs,
and then updating the database takes a bit of time (especially if you're
like me and like re-running tests on nearly every change you make :P).
I think it would be cool to have a pre-package Galaxy for integration
testing which is quick to setup.  I once thought of downloading Björn's
Docker image from Galaxy Bootstrap and using it that way, but thinking
is about as far as I got with that one.  One problem I could see is it
would have to be re-built on every release of Galaxy you want to test
against (whereas mercurial cloning/pulling makes sure you're always up
to date with the latest code).

Aaron


On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche mailto:rasche.e...@yandex.ru>> wrote:

 Hi Aaron,

 Good points, I was considering using galaxy bootstrap. This is
 mostly for the CI folk who want to download an archive, unpack it,
 and be ready to install/test their tools. The hg clone and egg/db
 steps seem like unnecessary overhead for this specific use case.

 Cheers,

 Eric








___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Peter Cock]

On Mon, Jul 21, 2014 at 6:51 PM, Eric Rasche  wrote:
> Currently the checkout options consist of hg clones, and archives that
> mercurial produces.
>
> Having pulled or cloned galaxy a few times lately, I'm wondering if anyone
> would have a use for a once-run galaxy instance in an archive? I.e., I'd
> clone, run once to grab eggs and do the db migration, then re-tar result and
> store online. Might cut down on build/test times for those who are using
> travis or other CIs. Thoughts/opinions?

Hi Eric,

Given how close you can get now for minimal effort,
this seem unnecessary.

http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

My TravisCI setup this fetches the latest Galaxy as
a tar ball (from a GitHub mirror as it was faster than
a git clone which was faster than getting the tar ball
from BitBucket, which in turn was faster than using
hg clone), and a per-migrated SQLite database
(using a bit of Galaxy functionality originally with
$GALAXY_TEST_DB_TEMPLATE added to speed
up running the functional tests).

Note this does not cache the eggs and all the other
side effects of the first run like creating config files,
so there is room for some speed up.

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Aaron Petkau]

Hello Eric,

That sounds like a pretty good idea.  If there was a pre-built image
available for whatever release I wanted to test against I could just cache
it and (hopefully) get my tests running a bit faster.  I'm not sure if
anyone else is already doing this?

Also, I remember there being mentioned pre-building docker images for each
release of Galaxy, which would accomplish something similar, but I'm not
really sure how that's being handled.  I think Björn's Docker image is kept
up to date with Galaxy stable each time it's built
https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51.
So, this could be handled by modifying his Dockerfile to build Galaxy at
whatever tagged release you want to test against.

One downside to docker is that you need to get it installed on your CI
server, which may or may not be possible (needs a very recent kernel for
example).

Aaron


On Mon, Jul 21, 2014 at 3:12 PM, Eric Rasche  wrote:

> Hi Aaron,
>
> Yeah, absolutely understandable. I want my tools tested early and often.
>
> I abuse my CI server for everything, especially for building and
> packaging software. In this case I was imagining that I might have it
> produce an archive on every tagged release, as well as producing a
> "daily" archive. All of these would be available on some ftp/http server
> somewhere with symlinks for latest archives (e.g.
> galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that work
> for your use case as well?
>
> Eric
>
> On 07/21/2014 03:02 PM, Aaron Petkau wrote:
> > Hello Eric,
> >
> > Your right about that, downloading the archive, installing all the eggs,
> > and then updating the database takes a bit of time (especially if you're
> > like me and like re-running tests on nearly every change you make :P).
> > I think it would be cool to have a pre-package Galaxy for integration
> > testing which is quick to setup.  I once thought of downloading Björn's
> > Docker image from Galaxy Bootstrap and using it that way, but thinking
> > is about as far as I got with that one.  One problem I could see is it
> > would have to be re-built on every release of Galaxy you want to test
> > against (whereas mercurial cloning/pulling makes sure you're always up
> > to date with the latest code).
> >
> > Aaron
> >
> >
> > On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche  > > wrote:
> >
> > Hi Aaron,
> >
> > Good points, I was considering using galaxy bootstrap. This is
> > mostly for the CI folk who want to download an archive, unpack it,
> > and be ready to install/test their tools. The hg clone and egg/db
> > steps seem like unnecessary overhead for this specific use case.
> >
> > Cheers,
> >
> > Eric
> >
> >
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

Hi Aaron,

Yeah, absolutely understandable. I want my tools tested early and often.

I abuse my CI server for everything, especially for building and
packaging software. In this case I was imagining that I might have it
produce an archive on every tagged release, as well as producing a
"daily" archive. All of these would be available on some ftp/http server
somewhere with symlinks for latest archives (e.g.
galaxy-latest-release.tgz and galaxy-latest-daily.tgz). Would that work
for your use case as well?

Eric

On 07/21/2014 03:02 PM, Aaron Petkau wrote:
> Hello Eric,
> 
> Your right about that, downloading the archive, installing all the eggs,
> and then updating the database takes a bit of time (especially if you're
> like me and like re-running tests on nearly every change you make :P). 
> I think it would be cool to have a pre-package Galaxy for integration
> testing which is quick to setup.  I once thought of downloading Björn's
> Docker image from Galaxy Bootstrap and using it that way, but thinking
> is about as far as I got with that one.  One problem I could see is it
> would have to be re-built on every release of Galaxy you want to test
> against (whereas mercurial cloning/pulling makes sure you're always up
> to date with the latest code).
> 
> Aaron
> 
> 
> On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche  > wrote:
> 
> Hi Aaron,
> 
> Good points, I was considering using galaxy bootstrap. This is
> mostly for the CI folk who want to download an archive, unpack it,
> and be ready to install/test their tools. The hg clone and egg/db
> steps seem like unnecessary overhead for this specific use case.
> 
> Cheers,
> 
> Eric
> 
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Aaron Petkau]

Hello Eric,

Your right about that, downloading the archive, installing all the eggs,
and then updating the database takes a bit of time (especially if you're
like me and like re-running tests on nearly every change you make :P).  I
think it would be cool to have a pre-package Galaxy for integration testing
which is quick to setup.  I once thought of downloading Björn's Docker
image from Galaxy Bootstrap and using it that way, but thinking is about as
far as I got with that one.  One problem I could see is it would have to be
re-built on every release of Galaxy you want to test against (whereas
mercurial cloning/pulling makes sure you're always up to date with the
latest code).

Aaron


On Mon, Jul 21, 2014 at 2:45 PM, Eric Rasche  wrote:

> Hi Aaron,
>
> Good points, I was considering using galaxy bootstrap. This is mostly for
> the CI folk who want to download an archive, unpack it, and be ready to
> install/test their tools. The hg clone and egg/db steps seem like
> unnecessary overhead for this specific use case.
>
> Cheers,
>
> Eric
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Eric Rasche]

Hi Aaron,Good points, I was considering using galaxy bootstrap. This is mostly for the CI folk who want to download an archive, unpack it, and be ready to install/test their tools. The hg clone and egg/db steps seem like unnecessary overhead for this specific use case.Cheers,Eric
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Once-run galaxy archives

[Aaron Petkau]

Hello Eric,

I think that's a great idea.  You may be interested in checking out Björn's
Galaxy Docker image
https://github.com/bgruening/docker-recipes/tree/master/galaxy which is
very similar to what you're describing.  A few other testing frameworks,
like Galaxy bootstrap https://github.com/jmchilton/galaxy-bootstrap, use
there own caching mechanism too.

Aaron


On Mon, Jul 21, 2014 at 12:51 PM, Eric Rasche  wrote:

> Currently the checkout options consist of hg clones, and archives that
> mercurial produces.
>
> Having pulled or cloned galaxy a few times lately, I'm wondering if anyone
> would have a use for a once-run galaxy instance in an archive? I.e., I'd
> clone, run once to grab eggs and do the db migration, then re-tar result
> and store online. Might cut down on build/test times for those who are
> using travis or other CIs. Thoughts/opinions?
>
>
> Cheers,
> Eric
> --
> Eric Rasche
> Programmer II
> Center for Phage Technology
> Texas A&M Univesity
> College Station, TX 77843
> Ph: 4046922048 
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/