Re: [galaxy-dev] Problems with Galaxy cluster using grid engine

2015-10-07 Thread Richard Polich
Hi John,

Yes, I came across a listing to enable this in /config/galaxy.ini for 
Gridengine. Peter van Heusden brought this error to my attention.  I have now 
commented this out and can now upload files.

#drmaa_external_runjob_script = 
/share/apps/galaxy/scripts/drmaa_external_runner.py
#drmaa_external_killjob_script = 
/share/apps/galaxy/scripts/drmaa_external_killer.py
#external_chown_script = /share/apps/galaxy/scripts/external_chown_script.py

Thank you very much,
Richard

On Oct 7, 2015, at 8:40 AM, John Chilton 
mailto:jmchil...@gmail.com>> wrote:

Do you have drmaa_external_** options set in config/galaxy.ini? It
seems like maybe you do. I would try to get Galaxy working without
those first, just submitting everything as the Galaxy user.

-John

On Fri, Oct 2, 2015 at 4:05 PM, Richard Polich 
mailto:rpol...@txbiomed.org>> wrote:
Hi All,

Trying to setup a Galaxy cluster using Rocks Gridengine

OS is Centos 6.5.
psql (9.1.18)
shell bash

Getting error messages in paster.log below. I can submit jobs to Gridengine
using qsub so this is not an issue. But when trying to "Upload File from you
computer”, history indicates jobs does not complete.

Any help would be appreciated.

galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,272 Changing
ownership of /share/apps/galaxy/database/tmp/upload_file_data_xvpaYs with:
/usr/bin/sudo -E /share/apps/galaxy/scripts/external_chown_script.py
/share/apps/galaxy/database/tmp/upload_file_data_xvpaYs rpolich 507
galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,297 Changing
ownership of uploaded file
/share/apps/galaxy/database/tmp/upload_file_data_xvpaYs failed: sudo: no tty
present and no askpass program specified

galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,297 Changing
ownership of /share/apps/galaxy/database/tmp/tmplIgC3n with: /usr/bin/sudo
-E /share/apps/galaxy/scripts/external_chown_script.py
/share/apps/galaxy/database/tmp/tmplIgC3n rpolich 507
galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,323 Changing
ownership of uploaded file /share/apps/galaxy/database/tmp/tmplIgC3n failed:
sudo: no tty present and no askpass program specified

galaxy.tools.actions.upload_common INFO 2015-10-02 09:35:02,357 tool upload1
created job id 101
galaxy.tools.execute DEBUG 2015-10-02 09:35:02,423 Tool [upload1] created
job [101] (332.351 ms)
206.124.61.6 - - [02/Oct/2015:09:34:59 -0500] "POST /api/tools HTTP/1.1" 200
- "http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel
Mac OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
206.124.61.6 - - [02/Oct/2015:09:35:02 -0500] "GET
/api/histories/1fad1eaf5f4f1766/contents HTTP/1.1" 200 -
"http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel
Mac OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
galaxy.jobs DEBUG 2015-10-02 09:35:02,676 (101) Working directory for job
is: /share/apps/galaxy/database/job_working_directory/000/101
galaxy.jobs.handler DEBUG 2015-10-02 09:35:02,682 (101) Dispatching to drmaa
runner
galaxy.jobs DEBUG 2015-10-02 09:35:02,894 (101) Persisting job destination
(destination id: sge_default)
galaxy.jobs.runners DEBUG 2015-10-02 09:35:02,903 Job [101] queued (220.456
ms)
galaxy.jobs.handler INFO 2015-10-02 09:35:02,958 (101) Job dispatched
galaxy.jobs.command_factory INFO 2015-10-02 09:35:03,821 Built script
[/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh]
for tool
command[/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh]
galaxy.jobs.runners DEBUG 2015-10-02 09:35:04,010 (101) command is:
/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh;
return_code=$?; python
"/share/apps/galaxy/database/job_working_directory/000/101/set_metadata_QaaegG.py"
"/share/apps/galaxy/database/tmp/tmpglze54"
"/share/apps/galaxy/database/job_working_directory/000/101/galaxy.json"
"/share/apps/galaxy/database/job_working_directory/000/101/metadata_in_HistoryDatasetAssociation_71_7DAllZ,/share/apps/galaxy/database/job_working_directory/000/101/metadata_kwds_HistoryDatasetAssociation_71_YiPkTL,/share/apps/galaxy/database/job_working_directory/000/101/metadata_out_HistoryDatasetAssociation_71_JbkolS,/share/apps/galaxy/database/job_working_directory/000/101/metadata_results_HistoryDatasetAssociation_71_d93tKG,/share/apps/galaxy/database/job_working_directory/000/101/galaxy_dataset_71.dat,/share/apps/galaxy/database/job_working_directory/000/101/metadata_override_HistoryDatasetAssociation_71_ih81Fj"
5242880; sh -c "exit $return_code"
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,074 (101) submitting
file /share/apps/galaxy/database/job_working_directory/000/101/galaxy_101.sh
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,075 (101) native
specification is: -q galaxy.q -V
galaxy.jobs DEBUG 2015-10-02 09:35:04,075 (101) Changing ownership of
working directory with: /usr/bin/sudo -E
/share/apps/galaxy/scripts/external_chown_script.py
/share/apps/galaxy/database/job_workin

Re: [galaxy-dev] Problems with Galaxy cluster using grid engine

2015-10-07 Thread John Chilton
Do you have drmaa_external_** options set in config/galaxy.ini? It
seems like maybe you do. I would try to get Galaxy working without
those first, just submitting everything as the Galaxy user.

-John

On Fri, Oct 2, 2015 at 4:05 PM, Richard Polich  wrote:
> Hi All,
>
> Trying to setup a Galaxy cluster using Rocks Gridengine
>
> OS is Centos 6.5.
> psql (9.1.18)
> shell bash
>
> Getting error messages in paster.log below. I can submit jobs to Gridengine
> using qsub so this is not an issue. But when trying to "Upload File from you
> computer”, history indicates jobs does not complete.
>
> Any help would be appreciated.
>
> galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,272 Changing
> ownership of /share/apps/galaxy/database/tmp/upload_file_data_xvpaYs with:
> /usr/bin/sudo -E /share/apps/galaxy/scripts/external_chown_script.py
> /share/apps/galaxy/database/tmp/upload_file_data_xvpaYs rpolich 507
> galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,297 Changing
> ownership of uploaded file
> /share/apps/galaxy/database/tmp/upload_file_data_xvpaYs failed: sudo: no tty
> present and no askpass program specified
>
> galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,297 Changing
> ownership of /share/apps/galaxy/database/tmp/tmplIgC3n with: /usr/bin/sudo
> -E /share/apps/galaxy/scripts/external_chown_script.py
> /share/apps/galaxy/database/tmp/tmplIgC3n rpolich 507
> galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,323 Changing
> ownership of uploaded file /share/apps/galaxy/database/tmp/tmplIgC3n failed:
> sudo: no tty present and no askpass program specified
>
> galaxy.tools.actions.upload_common INFO 2015-10-02 09:35:02,357 tool upload1
> created job id 101
> galaxy.tools.execute DEBUG 2015-10-02 09:35:02,423 Tool [upload1] created
> job [101] (332.351 ms)
> 206.124.61.6 - - [02/Oct/2015:09:34:59 -0500] "POST /api/tools HTTP/1.1" 200
> - "http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel
> Mac OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
> 206.124.61.6 - - [02/Oct/2015:09:35:02 -0500] "GET
> /api/histories/1fad1eaf5f4f1766/contents HTTP/1.1" 200 -
> "http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel
> Mac OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
> galaxy.jobs DEBUG 2015-10-02 09:35:02,676 (101) Working directory for job
> is: /share/apps/galaxy/database/job_working_directory/000/101
> galaxy.jobs.handler DEBUG 2015-10-02 09:35:02,682 (101) Dispatching to drmaa
> runner
> galaxy.jobs DEBUG 2015-10-02 09:35:02,894 (101) Persisting job destination
> (destination id: sge_default)
> galaxy.jobs.runners DEBUG 2015-10-02 09:35:02,903 Job [101] queued (220.456
> ms)
> galaxy.jobs.handler INFO 2015-10-02 09:35:02,958 (101) Job dispatched
> galaxy.jobs.command_factory INFO 2015-10-02 09:35:03,821 Built script
> [/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh]
> for tool
> command[/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh]
> galaxy.jobs.runners DEBUG 2015-10-02 09:35:04,010 (101) command is:
> /share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh;
> return_code=$?; python
> "/share/apps/galaxy/database/job_working_directory/000/101/set_metadata_QaaegG.py"
> "/share/apps/galaxy/database/tmp/tmpglze54"
> "/share/apps/galaxy/database/job_working_directory/000/101/galaxy.json"
> "/share/apps/galaxy/database/job_working_directory/000/101/metadata_in_HistoryDatasetAssociation_71_7DAllZ,/share/apps/galaxy/database/job_working_directory/000/101/metadata_kwds_HistoryDatasetAssociation_71_YiPkTL,/share/apps/galaxy/database/job_working_directory/000/101/metadata_out_HistoryDatasetAssociation_71_JbkolS,/share/apps/galaxy/database/job_working_directory/000/101/metadata_results_HistoryDatasetAssociation_71_d93tKG,/share/apps/galaxy/database/job_working_directory/000/101/galaxy_dataset_71.dat,/share/apps/galaxy/database/job_working_directory/000/101/metadata_override_HistoryDatasetAssociation_71_ih81Fj"
> 5242880; sh -c "exit $return_code"
> galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,074 (101) submitting
> file /share/apps/galaxy/database/job_working_directory/000/101/galaxy_101.sh
> galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,075 (101) native
> specification is: -q galaxy.q -V
> galaxy.jobs DEBUG 2015-10-02 09:35:04,075 (101) Changing ownership of
> working directory with: /usr/bin/sudo -E
> /share/apps/galaxy/scripts/external_chown_script.py
> /share/apps/galaxy/database/job_working_directory/000/101 rpolich 507
> galaxy.jobs ERROR 2015-10-02 09:35:04,102 (101) Failed to change ownership
> of /share/apps/galaxy/database/job_working_directory/000/101, making
> world-writable instead
> Traceback (most recent call last):
>   File "/share/apps/galaxy/lib/galaxy/jobs/__init__.py", line 1649, in
> change_ownership_for_run
> self._change_ownership( self.user_system_pwent[0], str(
> self.user_system_pwent[3] ) )
>   File "/share/apps/g

[galaxy-dev] Problems with Galaxy cluster using grid engine

2015-10-02 Thread Richard Polich
Hi All,

Trying to setup a Galaxy cluster using Rocks Gridengine

OS is Centos 6.5.
psql (9.1.18)
shell bash

Getting error messages in paster.log below. I can submit jobs to Gridengine 
using qsub so this is not an issue. But when trying to "Upload File from you 
computer”, history indicates jobs does not complete.

Any help would be appreciated.

galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,272 Changing 
ownership of /share/apps/galaxy/database/tmp/upload_file_data_xvpaYs with: 
/usr/bin/sudo -E /share/apps/galaxy/scripts/external_chown_script.py 
/share/apps/galaxy/database/tmp/upload_file_data_xvpaYs rpolich 507
galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,297 Changing 
ownership of uploaded file 
/share/apps/galaxy/database/tmp/upload_file_data_xvpaYs failed: sudo: no tty 
present and no askpass program specified

galaxy.tools.actions.upload_common DEBUG 2015-10-02 09:35:02,297 Changing 
ownership of /share/apps/galaxy/database/tmp/tmplIgC3n with: /usr/bin/sudo -E 
/share/apps/galaxy/scripts/external_chown_script.py 
/share/apps/galaxy/database/tmp/tmplIgC3n rpolich 507
galaxy.tools.actions.upload_common WARNING 2015-10-02 09:35:02,323 Changing 
ownership of uploaded file /share/apps/galaxy/database/tmp/tmplIgC3n failed: 
sudo: no tty present and no askpass program specified

galaxy.tools.actions.upload_common INFO 2015-10-02 09:35:02,357 tool upload1 
created job id 101
galaxy.tools.execute DEBUG 2015-10-02 09:35:02,423 Tool [upload1] created job 
[101] (332.351 ms)
206.124.61.6 - - [02/Oct/2015:09:34:59 -0500] "POST /api/tools HTTP/1.1" 200 - 
"http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel Mac 
OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
206.124.61.6 - - [02/Oct/2015:09:35:02 -0500] "GET 
/api/histories/1fad1eaf5f4f1766/contents HTTP/1.1" 200 - 
"http://galaxy.txbiomedgenetics.org:8080/"; "Mozilla/5.0 (Macintosh; Intel Mac 
OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0"
galaxy.jobs DEBUG 2015-10-02 09:35:02,676 (101) Working directory for job is: 
/share/apps/galaxy/database/job_working_directory/000/101
galaxy.jobs.handler DEBUG 2015-10-02 09:35:02,682 (101) Dispatching to drmaa 
runner
galaxy.jobs DEBUG 2015-10-02 09:35:02,894 (101) Persisting job destination 
(destination id: sge_default)
galaxy.jobs.runners DEBUG 2015-10-02 09:35:02,903 Job [101] queued (220.456 ms)
galaxy.jobs.handler INFO 2015-10-02 09:35:02,958 (101) Job dispatched
galaxy.jobs.command_factory INFO 2015-10-02 09:35:03,821 Built script 
[/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh] for 
tool 
command[/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh]
galaxy.jobs.runners DEBUG 2015-10-02 09:35:04,010 (101) command is: 
/share/apps/galaxy/database/job_working_directory/000/101/tool_script.sh; 
return_code=$?; python 
"/share/apps/galaxy/database/job_working_directory/000/101/set_metadata_QaaegG.py"
 "/share/apps/galaxy/database/tmp/tmpglze54" 
"/share/apps/galaxy/database/job_working_directory/000/101/galaxy.json" 
"/share/apps/galaxy/database/job_working_directory/000/101/metadata_in_HistoryDatasetAssociation_71_7DAllZ,/share/apps/galaxy/database/job_working_directory/000/101/metadata_kwds_HistoryDatasetAssociation_71_YiPkTL,/share/apps/galaxy/database/job_working_directory/000/101/metadata_out_HistoryDatasetAssociation_71_JbkolS,/share/apps/galaxy/database/job_working_directory/000/101/metadata_results_HistoryDatasetAssociation_71_d93tKG,/share/apps/galaxy/database/job_working_directory/000/101/galaxy_dataset_71.dat,/share/apps/galaxy/database/job_working_directory/000/101/metadata_override_HistoryDatasetAssociation_71_ih81Fj"
 5242880; sh -c "exit $return_code"
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,074 (101) submitting file 
/share/apps/galaxy/database/job_working_directory/000/101/galaxy_101.sh
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,075 (101) native 
specification is: -q galaxy.q -V
galaxy.jobs DEBUG 2015-10-02 09:35:04,075 (101) Changing ownership of working 
directory with: /usr/bin/sudo -E 
/share/apps/galaxy/scripts/external_chown_script.py 
/share/apps/galaxy/database/job_working_directory/000/101 rpolich 507
galaxy.jobs ERROR 2015-10-02 09:35:04,102 (101) Failed to change ownership of 
/share/apps/galaxy/database/job_working_directory/000/101, making 
world-writable instead
Traceback (most recent call last):
  File "/share/apps/galaxy/lib/galaxy/jobs/__init__.py", line 1649, in 
change_ownership_for_run
self._change_ownership( self.user_system_pwent[0], str( 
self.user_system_pwent[3] ) )
  File "/share/apps/galaxy/lib/galaxy/jobs/__init__.py", line 1643, in 
_change_ownership
assert p.returncode == 0
AssertionError
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,102 (101) submitting with 
credentials: rpolich [uid: 1006]
galaxy.jobs.runners.drmaa DEBUG 2015-10-02 09:35:04,104 (101) Job script for 
external submission is: /share/apps/galaxy/database/gridengine/10