Dear galaxy users,here at European Institute of Oncology we're deploying a brand new galaxy install, but facing problems with history (re)importing.
We have configured a galaxy running on one node where users took a "sneek peek" on features while developing a real deployment on nfs and cluster integration. Now I have to migrate histories to new galaxy install, I'm doing that on behalf my users because we choose to integrate with an apache proxy that does reverse proxying and ldap authentication.
Yesterday I produced four tar.gz for a user, placed on a temporary webserver and then proceeded with reimport on new instance: two where reimported, two not.
All those files are relatively small (from 1MB to 13MB), and no clues from galaxy debug log:
SUCCESSFUL ONE:galaxy.jobs DEBUG 2013-12-18 13:29:32,718 (354) Working directory for job is: /srv/galaxy/galaxy-dist/database/job_working_directory/000/354 galaxy.jobs.handler DEBUG 2013-12-18 13:29:32,738 (354) Dispatching to local runner galaxy.jobs DEBUG 2013-12-18 13:29:32,826 (354) Persisting job destination (destination id: local)
galaxy.jobs.handler INFO 2013-12-18 13:29:32,864 (354) Job dispatchedgalaxy.jobs.runners.local DEBUG 2013-12-18 13:29:33,151 (354) executing: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-37.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpLAAF5z --url galaxy.jobs DEBUG 2013-12-18 13:29:33,326 (354) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2013-12-18 13:29:34,451 execution finished: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-37.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpLAAF5z --url
galaxy.jobs DEBUG 2013-12-18 13:29:34,644 job 354 endedgalaxy.datatypes.metadata DEBUG 2013-12-18 13:29:34,644 Cleaning up external metadata files
UNSUCCESSFUL ONE:galaxy.jobs DEBUG 2013-12-18 13:30:42,824 (392) Working directory for job is: /srv/galaxy/galaxy-dist/database/job_working_directory/000/392 galaxy.jobs.handler DEBUG 2013-12-18 13:30:42,844 (392) Dispatching to local runner galaxy.jobs DEBUG 2013-12-18 13:30:42,934 (392) Persisting job destination (destination id: local)
galaxy.jobs.handler INFO 2013-12-18 13:30:42,963 (392) Job dispatchedgalaxy.jobs.runners.local DEBUG 2013-12-18 13:30:43,236 (392) executing: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-138.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpH8buNE --url galaxy.jobs DEBUG 2013-12-18 13:30:43,400 (392) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2013-12-18 13:30:47,524 execution finished: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-138.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpH8buNE --url
galaxy.jobs DEBUG 2013-12-18 13:30:47,711 job 392 endedgalaxy.datatypes.metadata DEBUG 2013-12-18 13:30:47,712 Cleaning up external metadata files
sorry for line wraps, I also attached logs.I cloned your repository on my development machine to try to recreate/debug this issue but unfortunately this time I'm unable to import successfully any history (plain galaxy instance, sqlite, no ext user, no job_conf, either default and stable branches).
I'm attaching the new galaxy instance universe_wsgi.ini and job_conf.xml, as you can see they are pretty straight-forward.
What I'm asking to you : which is the correct way for debugging this sort of things ? No luck putting some debugging code in
* lib/galaxy/tools/actions/history_imp_exp.py * lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py maybe I'm missing something ...also, I'd like to contribute some patch on a couple of minor glitches I found, but I'll write to -dev list for that
Thank you, and thanks for this great tool :) ciao -- Alessandro Ogier Specialista Reti e Sistemi - Area Ricerca Direzione ICT Gruppo IEO/CCM Istituto Europeo di Oncologia Via Serio, 15 - 20139 Milan, Italy P +39 02 94375269 M +39 334 6841627 (9am-6am CET) E alessandro.og...@ieo.eu W http://www.ieo.it
galaxy.jobs DEBUG 2013-12-18 13:29:32,718 (354) Working directory for job is: /srv/galaxy/galaxy-dist/database/job_working_directory/000/354 galaxy.jobs.handler DEBUG 2013-12-18 13:29:32,738 (354) Dispatching to local runner galaxy.jobs DEBUG 2013-12-18 13:29:32,826 (354) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2013-12-18 13:29:32,864 (354) Job dispatched galaxy.jobs.runners.local DEBUG 2013-12-18 13:29:33,151 (354) executing: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-37.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpLAAF5z --url galaxy.jobs DEBUG 2013-12-18 13:29:33,326 (354) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2013-12-18 13:29:34,451 execution finished: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-37.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpLAAF5z --url galaxy.jobs DEBUG 2013-12-18 13:29:34,644 job 354 ended galaxy.datatypes.metadata DEBUG 2013-12-18 13:29:34,644 Cleaning up external metadata files
galaxy.jobs DEBUG 2013-12-18 13:30:42,824 (392) Working directory for job is: /srv/galaxy/galaxy-dist/database/job_working_directory/000/392 galaxy.jobs.handler DEBUG 2013-12-18 13:30:42,844 (392) Dispatching to local runner galaxy.jobs DEBUG 2013-12-18 13:30:42,934 (392) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2013-12-18 13:30:42,963 (392) Job dispatched galaxy.jobs.runners.local DEBUG 2013-12-18 13:30:43,236 (392) executing: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-138.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpH8buNE --url galaxy.jobs DEBUG 2013-12-18 13:30:43,400 (392) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2013-12-18 13:30:47,524 execution finished: export GALAXY_SLOTS="1"; python /srv/galaxy/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://debian.ieo.eu/drosano/dalia-138.tar.gz /srv/galaxy/galaxy-dist/database/tmp/tmpH8buNE --url galaxy.jobs DEBUG 2013-12-18 13:30:47,711 job 392 ended galaxy.datatypes.metadata DEBUG 2013-12-18 13:30:47,712 Cleaning up external metadata files
<?xml version="1.0"?>
<!-- A sample job config that explicitly configures job running the way it is configured by default (if there is no explicit config). -->
<job_conf>
<plugins>
<plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
</plugins>
<handlers default="handlers">
<handler id="handler0" tags="handlers"/>
<handler id="handler1" tags="handlers"/>
</handlers>
<destinations>
<destination id="local" runner="local"/>
</destinations>
</job_conf>
universe_wsgi.ini.gz
Description: GNU Zip compressed data
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
