Re: [galaxy-dev] concatenate_datasets docker image can't create output
Can you send me a screenshot of the Tool form right before you hit submit. It is odd that two commands are executed barely differing at all. There were some recent UI changes and I want to make sure that you are passing two inputs once and not submitting one input twice in the newly named batch mode? container.sh is generated in command_factory.py using containers.py - here are some relevant links. https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default -John On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: I'm still trying to debug this. How do I change the working directory? I can't seem to find out how all those directory mounts get passed to docker - the catDocker.xml appears to generate a 'cat' command that I assume ends up in ()/job_working_directory/000/12/container.sh but I can't find where that's happening - container.sh isn't listed in any file under galaxy-dist. job_conf.xml lists the directories that get passed as variables - again, when and where are those defined? Thanks, -Jeltje On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: I tried the second solution (setting everything to rw) and while I can confirm that the command is changed accordingly, the results are the same: Two empty output files and the same error message. Command from paster.log: docker run -e GALAXY_SLOTS=$GALAXY_SLOTS -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat ; fi; sh -c exit $return_code -Jeltje On Fri, Jan 23, 2015 at 12:24 PM, John Chilton jmchil...@gmail.com wrote: My first thought is the bug mentioned here - https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html along with potential work arounds. Are you able to confirm that this is or is not the problem at all? -John On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: Hi, I'm following instructions at https://github.com/apetkau/galaxy-hackathon-2014. When I try to run the concatenate-datasets on two input files, two odd things happen. First, TWO nearly identical commands are generated in the History panel, only differing in their output filename. In paster.log, only the second one shows up. Second, the program fails, with this info in the History: An error occurred with this dataset: Galaxy slots passed through contain as 1 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh: line 2: can't create /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat This happens for both commands in the History panel, only the output filename in the other error is dataset_15.dat Oddly, the dataset_14.dat and dataset_15.dat are both created during this command, they just end up empty. Paster.log: galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker inspect busybox:ubuntu-14.04 /dev/null 21 [ $? -ne 0 ] docker pull busybox:ubuntu-14.04 /dev/null 21 docker run -e GALAXY_SLOTS=$GALAXY_SLOTS -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw -v
Re: [galaxy-dev] concatenate_datasets docker image can't create output
See attached for screenshot! Thanks for the links. On Mon, Jan 26, 2015 at 11:37 AM, John Chilton jmchil...@gmail.com wrote: Can you send me a screenshot of the Tool form right before you hit submit. It is odd that two commands are executed barely differing at all. There were some recent UI changes and I want to make sure that you are passing two inputs once and not submitting one input twice in the newly named batch mode? container.sh is generated in command_factory.py using containers.py - here are some relevant links. https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default -John On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: I'm still trying to debug this. How do I change the working directory? I can't seem to find out how all those directory mounts get passed to docker - the catDocker.xml appears to generate a 'cat' command that I assume ends up in ()/job_working_directory/000/12/container.sh but I can't find where that's happening - container.sh isn't listed in any file under galaxy-dist. job_conf.xml lists the directories that get passed as variables - again, when and where are those defined? Thanks, -Jeltje On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: I tried the second solution (setting everything to rw) and while I can confirm that the command is changed accordingly, the results are the same: Two empty output files and the same error message. Command from paster.log: docker run -e GALAXY_SLOTS=$GALAXY_SLOTS -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat ; fi; sh -c exit $return_code -Jeltje On Fri, Jan 23, 2015 at 12:24 PM, John Chilton jmchil...@gmail.com wrote: My first thought is the bug mentioned here - https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html along with potential work arounds. Are you able to confirm that this is or is not the problem at all? -John On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: Hi, I'm following instructions at https://github.com/apetkau/galaxy-hackathon-2014. When I try to run the concatenate-datasets on two input files, two odd things happen. First, TWO nearly identical commands are generated in the History panel, only differing in their output filename. In paster.log, only the second one shows up. Second, the program fails, with this info in the History: An error occurred with this dataset: Galaxy slots passed through contain as 1 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh: line 2: can't create /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat This happens for both commands in the History panel, only the output filename in the other error is dataset_15.dat Oddly, the dataset_14.dat and dataset_15.dat are both created during this command, they just end up empty. Paster.log: galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker inspect busybox:ubuntu-14.04 /dev/null 21 [ $? -ne 0 ] docker pull busybox:ubuntu-14.04 /dev/null 21 docker run -e GALAXY_SLOTS=$GALAXY_SLOTS -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro -v
Re: [galaxy-dev] concatenate_datasets docker image can't create output
My first thought is the bug mentioned here - https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html along with potential work arounds. Are you able to confirm that this is or is not the problem at all? -John On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren jeltje.van.ba...@gmail.com wrote: Hi, I'm following instructions at https://github.com/apetkau/galaxy-hackathon-2014. When I try to run the concatenate-datasets on two input files, two odd things happen. First, TWO nearly identical commands are generated in the History panel, only differing in their output filename. In paster.log, only the second one shows up. Second, the program fails, with this info in the History: An error occurred with this dataset: Galaxy slots passed through contain as 1 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh: line 2: can't create /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat This happens for both commands in the History panel, only the output filename in the other error is dataset_15.dat Oddly, the dataset_14.dat and dataset_15.dat are both created during this command, they just end up empty. Paster.log: galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker inspect busybox:ubuntu-14.04 /dev/null 21 [ $? -ne 0 ] docker pull busybox:ubuntu-14.04 /dev/null 21 docker run -e GALAXY_SLOTS=$GALAXY_SLOTS -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat ; fi; sh -c exit $return_code galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing job script: /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job destination (destination id: docker_local) What could be happening here, and how do I fix it? Thanks, -Jeltje ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
