[galaxy-user] Attempting to use parallelism in BWA

2012-07-17 Thread Kevin C. Dorff
Hi,

I am using Python 2.7. I am running with an hg checked out, up to date
version of https://bitbucket.org/galaxy/galaxy-dist/;.

I am very new to Galaxy. I have an instance setup *and running* locally. I
am able to perform routine actions such as uploading files, processing with
Picard, and aligning with BWA. I have been able to make the jobs execute
out our SGE cluster.

In order to better utilize my cluster, I attempted to enable parallelism of
the tasks that support it (such as BWA) by configuring the *
universe_wsgi.ini* file with the parameter

   use_tasked_jobs = True

When I do this and attempt execute a BWA alignment on a 4.1Gb fastq file, I
am given the error


Result 256 from mv
/pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/job_working_directory/000/21/task_1/dataset_21.dat
/pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/files/000/dataset_21.dat
/pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/job_working_directory/000/21/task_0:
Traceback (most recent call last):
  File ./scripts/extract_dataset_part.py, line 25, in ?
import galaxy.model.mapping #need to load this before we unpickle, in
order to setup properties assigned by the mappers
  File
/pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/lib/galaxy/model/__init__.py,
line 13, in ?
import galaxy.datatypes.registry
  File
/pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/lib/galaxy/datatypes/registry.py,
line 146
finally:
  ^
SyntaxError: invalid syntax
The alignment failed.


If you need the complete error message, contact me and I can supply you
with it. It's too large to paste in-line and attachments aren't allowed.

Is there more I need to do to enable parallel jobs (splitting of input,
align, then merge)?

Any suggestions are greatly appreciated.

Kevin
___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using reply all in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-user] Attempting to use parallelism in BWA

2012-07-17 Thread Scott McManus
Hi Kevin- 

That's truly odd. The error happens when the offending file is imported, which 
means that 
it is a syntax issue that should have been seen earlier. I'll take a look at 
this soon (unless someone 
else beats me to it). 

In the meantime, of course, do not use the use_tasked_jobs on this instance of 
Galaxy. If others 
have seen similar issues, then please speak up. 

-Scott 

- Original Message -

 Hi,

 I am using Python 2.7. I am running with an hg checked out, up to
 date version of  https://bitbucket.org/galaxy/galaxy-dist/ .

 I am very new to Galaxy. I have an instance setup and running
 locally. I am able to perform routine actions such as uploading
 files, processing with Picard, and aligning with BWA. I have been
 able to make the jobs execute out our SGE cluster.

 In order to better utilize my cluster, I attempted to enable
 parallelism of the tasks that support it (such as BWA) by
 configuring the universe_wsgi.ini file with the parameter

 use_tasked_jobs = True

 When I do this and attempt execute a BWA alignment on a 4.1Gb fastq
 file, I am given the error

  Result 256 from mv
  /pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/job_working_directory/000/21/task_1/dataset_21.dat
  /pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/files/000/dataset_21.dat
 
  /pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/database/job_working_directory/000/21/task_0:
 
  Traceback (most recent call last):
 
  File ./scripts/extract_dataset_part.py, line 25, in ?
 
  import galaxy.model.mapping #need to load this before we unpickle,
  in
  order to setup properties assigned by the mappers
 
  File
  /pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/lib/galaxy/model/__init__.py,
  line 13, in ?
 
  import galaxy.datatypes.registry
 
  File
  /pbtech_mounts/fclab_store006/gobyweb_dat/galaxy-python/lib/galaxy/datatypes/registry.py,
  line 146
 
  finally:
 
  ^
 
  SyntaxError: invalid syntax
 
  The alignment failed.
 
 If you need the complete error message, contact me and I can supply
 you with it. It's too large to paste in-line and attachments aren't
 allowed.

 Is there more I need to do to enable parallel jobs (splitting of
 input, align, then merge)?

 Any suggestions are greatly appreciated.

 Kevin
 ___
 The Galaxy User list should be used for the discussion of
 Galaxy analysis and other features on the public server
 at usegalaxy.org. Please keep all replies on the list by
 using reply all in your mail client. For discussion of
 local Galaxy instances and the Galaxy source code, please
 use the Galaxy Development list:

 http://lists.bx.psu.edu/listinfo/galaxy-dev

 To manage your subscriptions to this and other Galaxy lists,
 please use the interface at:

 http://lists.bx.psu.edu/___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using reply all in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/