On Feb 8, 2012, at 9:32 PM, Fields, Christopher J wrote:
'samtools sort' seems to be running on our server end as well (not on the
cluster). I may look into it a bit more myself. Snapshot of top off our
server (you can see our local runner as well):
PID USER PR NI VIRT RES SHR
On Feb 8, 2012, at 11:58 AM, Ryan Golhar wrote:
Hi Nate - I finally got a chance to look at this briefly, but I must admit,
my Python skills are lacking. In the Bam class in binary.py, all I see are
calls to
proc = subprocess.Popen( args=command, shell=True, cwd=tmp_dir, stderr=open(
On Feb 13, 2012, at 9:45 AM, Nate Coraor wrote:
On Feb 8, 2012, at 9:32 PM, Fields, Christopher J wrote:
'samtools sort' seems to be running on our server end as well (not on the
cluster). I may look into it a bit more myself. Snapshot of top off our
server (you can see our local runner
On Feb 13, 2012, at 11:52 AM, Fields, Christopher J wrote:
On Feb 13, 2012, at 9:45 AM, Nate Coraor wrote:
On Feb 8, 2012, at 9:32 PM, Fields, Christopher J wrote:
'samtools sort' seems to be running on our server end as well (not on the
cluster). I may look into it a bit more myself.
On Thu, Feb 9, 2012 at 2:57 AM, Fields, Christopher J
cjfie...@illinois.edu wrote:
Forgot to add, but this also seems tied to the same problem
Ryan's describing. IIRC Galaxy also runs 'samtools sort' after
certain jobs, correct?
chris
This sounds like part of the BAM grooming (assuming
Hi Nate - I finally got a chance to look at this briefly, but I must admit,
my Python skills are lacking. In the Bam class in binary.py, all I see are
calls to
proc = subprocess.Popen( args=command, shell=True, cwd=tmp_dir,
stderr=open( stderr_name, 'wb' ) )
which, to me, look like calls to
'samtools sort' seems to be running on our server end as well (not on the
cluster). I may look into it a bit more myself. Snapshot of top off our
server (you can see our local runner as well):
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
3950 galaxy
Forgot to add, but this also seems tied to the same problem Ryan's describing.
IIRC Galaxy also runs 'samtools sort' after certain jobs, correct?
chris
On Feb 8, 2012, at 8:32 PM, Fields, Christopher J wrote:
'samtools sort' seems to be running on our server end as well (not on the
On Jan 18, 2012, at 11:54 AM, Ryan Golhar wrote:
Nate - Is there a specific place in the Galaxy code that forks the samtools
index on bam files on the cluster or the head node? I really need to track
this down.
Hey Ryan,
Sorry it's taken so long, I've been pretty busy. The relevant code
Galaxy shouldn't be trying to do that, but it also shouldn't cause metadata to
fail.
On Jan 20, 2012, at 10:52 AM, Ryan Golhar wrote:
Thanks Nate. I'll play with that. Could it be that Galaxy is trying to
reset the permissions or ownership of the imported BAM files. I'm not
copying them
Just wanted to add that we have consistently seen this issue of 'samtools
index' running locally on our install. We are using SGE scheduler. Thanks for
pointing out details in the code Nate.
--
Shantanu.
On Jan 20, 2012, at 9:35 AM, Nate Coraor wrote:
On Jan 18, 2012, at 11:54 AM, Ryan
Nate - Is there a specific place in the Galaxy code that forks the samtools
index on bam files on the cluster or the head node? I really need to track
this down.
On Fri, Jan 13, 2012 at 12:54 PM, Ryan Golhar
ngsbioinformat...@gmail.comwrote:
I re-uploaded 3 BAM files using the Upload system
On Jan 12, 2012, at 11:41 PM, Ryan Golhar wrote:
Any ideas as to how to fix this? We are interested in using Galaxy to host
all our NGS data. If indexing on the head node is going to happen, then this
is going to be an extremely slow process.
Could you post the contents of
I re-uploaded 3 BAM files using the Upload system filepaths
On Fri, Jan 13, 2012 at 10:53 AM, Nate Coraor n...@bx.psu.edu wrote:
On Jan 12, 2012, at 11:41 PM, Ryan Golhar wrote:
Any ideas as to how to fix this? We are interested in using Galaxy to
host all our NGS data. If indexing on the
I re-uploaded 3 BAM files using the Upload system file paths. runner0.log
shows:
galaxy.jobs DEBUG 2012-01-13 12:50:08,442 dispatching job 76 to pbs runner
galaxy.jobs INFO 2012-01-13 12:50:08,555 job 76 dispatched
galaxy.jobs.runners.pbs DEBUG 2012-01-13 12:50:08,697 (76) submitting file
Any ideas as to how to fix this? We are interested in using Galaxy to host
all our NGS data. If indexing on the head node is going to happen, then
this is going to be an extremely slow process.
___
Please keep all replies on the list by
On Jan 10, 2012, at 10:20 PM, Ryan Golhar wrote:
On Tue, Jan 10, 2012 at 11:43 AM, Ryan Golhar ngsbioinformat...@gmail.com
wrote:
Hi Ryan,
You could check it in lib/galaxy/config.py, after it's read. By any chance,
are you using galaxy-central vs. galaxy-dist? It's possible that due
it is also set to True:
[galaxy@bic galaxy-dist]$ grep Ryan *.log
runner0.log:galaxy.jobs WARNING 2012-01-10 22:17:26,381 Ryan Golhar -
self.set_metadata_externally = True
Clearly something else is going on here. On my last import of BAM file,
even after samtools finished indexing the
On Jan 11, 2012, at 10:56 AM, Ryan Golhar wrote:
it is also set to True:
[galaxy@bic galaxy-dist]$ grep Ryan *.log
runner0.log:galaxy.jobs WARNING 2012-01-10 22:17:26,381 Ryan Golhar -
self.set_metadata_externally = True
Clearly something else is going on here. On my last import of
On Wed, Jan 11, 2012 at 11:16 AM, Nate Coraor n...@bx.psu.edu wrote:
On Jan 11, 2012, at 10:56 AM, Ryan Golhar wrote:
it is also set to True:
[galaxy@bic galaxy-dist]$ grep Ryan *.log
runner0.log:galaxy.jobs WARNING 2012-01-10 22:17:26,381 Ryan Golhar -
self.set_metadata_externally =
On Jan 9, 2012, at 2:38 PM, Ryan Golhar wrote:
On Fri, Jan 6, 2012 at 12:55 PM, Ryan Golhar ngsbioinformat...@gmail.com
wrote:
This indicates that set_meta is running locally, in the runner process. Can
you make sure there's not a typo in your config? The other possibility is
that
On Tue, Jan 10, 2012 at 11:43 AM, Ryan Golhar
ngsbioinformat...@gmail.comwrote:
Hi Ryan,
You could check it in lib/galaxy/config.py, after it's read. By any
chance, are you using galaxy-central vs. galaxy-dist? It's possible that
due to a bug I recently fixed and a certain combination of
On Fri, Jan 6, 2012 at 12:55 PM, Ryan Golhar ngsbioinformat...@gmail.comwrote:
This indicates that set_meta is running locally, in the runner process.
Can you make sure there's not a typo in your config? The other
possibility is that external metadata setting failed and it's being retried
On Jan 5, 2012, at 2:48 PM, Ryan Golhar wrote:
On Thu, Jan 5, 2012 at 11:59 AM, Nate Coraor n...@bx.psu.edu wrote:
On Jan 5, 2012, at 11:41 AM, Ryan Golhar wrote:
I set it to run on the cluster:
[galaxy@bic galaxy-dist]$ grep upload1 universe_wsgi.runner.ini
#upload1 = local:///
This indicates that set_meta is running locally, in the runner process.
Can you make sure there's not a typo in your config? The other
possibility is that external metadata setting failed and it's being retried
internally (if that was true, you'd see messages indicated such in the
server
On Jan 4, 2012, at 6:44 PM, Ryan Golhar wrote:
On Wed, Jan 4, 2012 at 5:17 PM, Ryan Golhar ngsbioinformat...@gmail.com
wrote:
I'm adding Data Libraries to my local galaxy instance. I'm doing this by
importing directories that contain bam and bai files. I see the bam/bai
files get added
On Jan 4, 2012, at 6:44 PM, Ryan Golhar wrote:
On Wed, Jan 4, 2012 at 5:17 PM, Ryan Golhar ngsbioinformat...@gmail.com
wrote:
I'm adding Data Libraries to my local galaxy instance. I'm doing this
by importing directories that contain bam and bai files. I see the bam/bai
files get added
On Jan 5, 2012, at 11:29 AM, Ryan Golhar wrote:
On Jan 4, 2012, at 6:44 PM, Ryan Golhar wrote:
On Wed, Jan 4, 2012 at 5:17 PM, Ryan Golhar ngsbioinformat...@gmail.com
wrote:
I'm adding Data Libraries to my local galaxy instance. I'm doing this by
importing directories that contain
I set it to run on the cluster:
[galaxy@bic galaxy-dist]$ grep upload1 universe_wsgi.runner.ini
#upload1 = local:///
On Thu, Jan 5, 2012 at 11:33 AM, Nate Coraor n...@bx.psu.edu wrote:
On Jan 5, 2012, at 11:29 AM, Ryan Golhar wrote:
On Jan 4, 2012, at 6:44 PM, Ryan Golhar wrote:
On
On Jan 5, 2012, at 11:41 AM, Ryan Golhar wrote:
I set it to run on the cluster:
[galaxy@bic galaxy-dist]$ grep upload1 universe_wsgi.runner.ini
#upload1 = local:///
Could you set use_heartbeat = True in the runner's config file and then check
the resulting heartbeat log files created in
I'm adding Data Libraries to my local galaxy instance. I'm doing this by
importing directories that contain bam and bai files. I see the bam/bai
files get added on the admin page and the Message is This job is running.
qstat shows the job run and complete. I checked my runner0.log and it
On Wed, Jan 4, 2012 at 5:17 PM, Ryan Golhar ngsbioinformat...@gmail.comwrote:
I'm adding Data Libraries to my local galaxy instance. I'm doing this by
importing directories that contain bam and bai files. I see the bam/bai
files get added on the admin page and the Message is This job is
On Wed, Jan 4, 2012 at 5:17 PM, Ryan Golhar
ngsbioinformat...@gmail.comwrote:
I'm adding Data Libraries to my local galaxy instance. I'm doing this by
importing directories that contain bam and bai files. I see the bam/bai
files get added on the admin page and the Message is This job is
33 matches
Mail list logo
