Re: [galaxy-dev] Trackster error after upgrade

2012-11-14 Thread Joachim Jacob

Hi Jeremy,

Thanks for you assistance. I would love to try your suggestions, but 
Galaxy won't start anymore.


I have found the root cause of the problems: my home partition has 
become too small, due to a fat 'job_working_directory'. I didn't expect 
it to become so big (especially with tophat output). Now I have the 
job_working_directory mounted on our network share. So I restart 
Galaxy, and got this error:


Traceback (most recent call last):
 File 
/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py, line 
36, in app_factory

   app = UniverseApplication( global_conf = global_conf, **kwargs )
 File /home/galaxy/galaxy-dist/lib/galaxy/app.py, line 85, in 
__init__
   self.toolbox = tools.ToolBox( tool_configs, self.config.tool_path, 
self )
 File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 
84, in __init__

   self.load_integrated_tool_panel_keys()
 File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 
236, in load_integrated_tool_panel_keys

   tree = util.parse_xml( self.integrated_tool_panel_config )
 File /home/galaxy/galaxy-dist/lib/galaxy/util/__init__.py, line 
143, in parse_xml

   tree = ElementTree.parse(fname)
 File 
/home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py, 
line 859, in parse

   tree.parse(source, parser)
 File 
/home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py, 
line 584, in parse

   self._root = parser.close()
 File 
/home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py, 
line 1251, in close

   self._parser.Parse(, 1) # end of data
ExpatError: unclosed token: line 109, column 4
Removing PID file main.pid


Before dragging up my backup, any idea how to fix this otherwise? I 
suspect that mercurial could not pull all changes in, is that possible?



Cheers,
Joachim


On Wed 14 Nov 2012 04:25:53 AM CET, Jeremy Goecks wrote:

One dataset I want to view in Trackster has an error
ConverterDependencyException: 'A dependency (bgzip) was in an error state.


I've seen this once or twice before but haven't been able to track down the 
issue yet.

Here are two workaround that may help you out:

(1) Add the dataset to a new/saved visualization and then click 'try again' to 
try reindexing the dataset;

(2) copy the dataset using History Options -- Copy Datasets, then delete the 
failing dataset and try indexing the new dataset.

If the dataset continues to fail, please make it available to me and I'll take 
a look.

Best,
J.


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] Can't edit galaxy workflow

2012-11-14 Thread Dannon Baker
This changeset is only in galaxy-central, at the moment, and not galaxy-dist.  
You have a few options here:

Pull directly from -central to the revision indicated (though this means you 
will be updating your instance to less thoroughly tested galaxy-central code):

hg pull -u -r 5013377e0bf7 http://bitbucket.org/galaxy/galaxy-central/

Or, probably preferable to that, you could apply just the individual changeset 
as a patch (which will merge in silently when you do update -dist).  Simply 
execute the following within your galaxy-dist directory:

curl https://bitbucket.org/galaxy/galaxy-central/changeset/5013377e0bf7/raw  
modulefix.patch
hg patch --no-commit modulefix.patch

-Dannon


On Nov 14, 2012, at 8:04 AM, Sanjarbek Hudaiberdiev hudai...@icgeb.org wrote:

 Thanks a lot Dannon, for your reply.
 
 But hg keeps saying following when I try to pull: abort: unknown revision 
 '5013377e0bf7'!
 
 Is there any other way of manually getting it?
 
 (Sorry, I'm not an expert on those new SCMs)
 Sanjar.
 
 On 11/13/2012 06:33 PM, Dannon Baker wrote:
 Sanjar,
 
 This is fixed as of 5013377e0bf7.  This may not be in the next distribution, 
 but will be in the one after that.  Of course, you can manually pull the 
 change from galaxy-central at any time.
 
 -Dannon
 
 
 On Nov 13, 2012, at 9:45 AM, Sanjarbek Hudaiberdiev hudai...@icgeb.org 
 wrote:
 
 I tried to reply to similar posts, but couldn't figure out how to do it. So 
 posting again:
 
 Galaxy giving this error when editing workflow, just after creating 
 workflow from existing history:
 
 URL: 
 http://localhost/galaxy/workflow/load_workflow?id=df7a1f0c02a5b08e_=1352817462148
 Module weberror.evalexception.middleware:364 in respond
  app_iter = self.application(environ, detect_start_response)
 Module paste.debug.prints:98 in __call__
  environ, self.app)
 Module paste.wsgilib:539 in intercept_output
  app_iter = application(environ, replacement_start_response)
 Module paste.recursive:80 in __call__
  return self.application(environ, start_response)
 Module paste.httpexceptions:632 in __call__
  return self.application(environ, start_response)
 Module galaxy.web.framework.base:160 in __call__
  body = method( trans, **kwargs )
 Module galaxy.web.framework:73 in decorator
  return simplejson.dumps( func( self, trans, *args, **kwargs ) )
 Module galaxy.webapps.galaxy.controllers.workflow:733 in load_workflow
  'tooltip': module.get_tooltip( static_path=url_for( '/static' ) ),
 Module galaxy.workflow.modules:258 in get_tooltip
  return self.tool.help.render( static_path=static_path )
 AttributeError: 'NoneType' object has no attribute 'render'
 
 Could anyone help me to solve this problem?
 
 Thanks,
 Sanjar.
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/
 

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] Can't edit galaxy workflow

2012-11-14 Thread Sanjarbek Hudaiberdiev

Thanks a lot Dannon, for your reply.

But hg keeps saying following when I try to pull: abort: unknown 
revision '5013377e0bf7'!


Is there any other way of manually getting it?

(Sorry, I'm not an expert on those new SCMs)
Sanjar.

On 11/13/2012 06:33 PM, Dannon Baker wrote:

Sanjar,

This is fixed as of 5013377e0bf7.  This may not be in the next distribution, 
but will be in the one after that.  Of course, you can manually pull the change 
from galaxy-central at any time.

-Dannon


On Nov 13, 2012, at 9:45 AM, Sanjarbek Hudaiberdiev hudai...@icgeb.org wrote:


I tried to reply to similar posts, but couldn't figure out how to do it. So 
posting again:

Galaxy giving this error when editing workflow, just after creating workflow 
from existing history:

URL: 
http://localhost/galaxy/workflow/load_workflow?id=df7a1f0c02a5b08e_=1352817462148
Module weberror.evalexception.middleware:364 in respond

  app_iter = self.application(environ, detect_start_response)

Module paste.debug.prints:98 in __call__

  environ, self.app)

Module paste.wsgilib:539 in intercept_output

  app_iter = application(environ, replacement_start_response)

Module paste.recursive:80 in __call__

  return self.application(environ, start_response)

Module paste.httpexceptions:632 in __call__

  return self.application(environ, start_response)

Module galaxy.web.framework.base:160 in __call__

  body = method( trans, **kwargs )

Module galaxy.web.framework:73 in decorator

  return simplejson.dumps( func( self, trans, *args, **kwargs ) )

Module galaxy.webapps.galaxy.controllers.workflow:733 in load_workflow

  'tooltip': module.get_tooltip( static_path=url_for( '/static' ) ),

Module galaxy.workflow.modules:258 in get_tooltip

  return self.tool.help.render( static_path=static_path )

AttributeError: 'NoneType' object has no attribute 'render'

Could anyone help me to solve this problem?

Thanks,
Sanjar.
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


[galaxy-dev] Galaxy tool error report from yaxi...@gmail.com

2012-11-14 Thread yaximik

GALAXY TOOL ERROR REPORT


This error report was sent from the Galaxy instance hosted on the server
127.0.0.1:7070
-
This is in reference to dataset id 13 from history id 6
-
You should be able to view the history containing the related history item

13: FastQC_SC2T252P15_S1_L001_R1_001.fastq.html

by logging in as a Galaxy admin user to the Galaxy instance referenced above
and pointing your browser to the following link.

127.0.0.1:7070/history/view?id=e03374982e5c9d8a
-
The user 'yaxi...@gmail.com' provided the following information:

This was returned upon attempt to run FasQC on one Illumina gromed dataset.

Local galaxy install on RHEL58
-
job id: 7
tool id: fastqc
-
job command line:
python /home/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i 
/home/GalaxyUpload/SC2T252P15_S1_L001_R1_001.fastq -d 
/home/galaxy/galaxy-dist/database/job_working_directory/000/7/dataset_13_files
 -o /home/galaxy/galaxy-dist/database/files/000/dataset_13.dat -n FastQC -f 
fastq -j SC2T252P15_S1_L001_R1_001.fastq -e 
/home/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc
-
job stderr:
Traceback (most recent call last):
  File /home/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py, line 158, in 
module
assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find 
executable %s' % opts.executable
AssertionError: ##rgFastQC.py error - cannot find executable 
/home/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc

-
job stdout:

-
job info:
None
-
job traceback:
None
-
(This is an automated message).
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Working with files outside galaxy without copying?

2012-11-14 Thread Samuel Lampa

Hi,

We are looking into ways to integrate galaxy into the workflows of users 
at our cluster, with lots of NGS users running all and any kind of 
analyses on their typically huge amounts of data. For this we use a 
parallel file system, available on all compute nodes.


This file system, although approx 1PB in size, is constantly filling up, 
and thus we are not very attracted by the idea of copying files into/out 
of galaxy for each analysis.


Thus, we would be interested to know what are the options for working 
with existing/external(to galaxy) file systems?


Eg. would it be possible to link files into some kind of galaxy file 
system (I'm not totally clear about how galaxy stores it's data, 
although I found out that stuff is created in database/files), from outside?


 ... or is there any work going on for selecting any file system path 
as input in galaxy workflows?


... or any other hints?

As said, I'm quite new to galaxy, trying to grok my head around how we 
can use it, so all hints are welcome.


Best Regards
// Samuel


--
Developer at SNIC-UPPMAX www.uppmax.uu.se
Developer at Dept of Pharm Biosciences www.farmbio.uu.se
Twitter - twitter.com/samuellampa
Blog - saml.rilspace.org
G+ - gplus.to/saml

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] Working with files outside galaxy without copying?

2012-11-14 Thread Jorrit Boekel
This may be implemented, I believe some people run Galaxy on the cluster,
but if not:

Galaxy normally operates on files stored in the database/files folder. If
your Galaxy instance has access to the files you need on your parallel fs,
I guess you could start by writing a tool that creates links to your files
when given the path to the input file. Another tool may then move the
dataset into the desired place while creating a link in the database folder.

Sounds like a hack to me, but may work.

cheers,
jorrit




On Wed, Nov 14, 2012 at 2:52 PM, Samuel Lampa samuel.la...@gmail.comwrote:

 Hi,

 We are looking into ways to integrate galaxy into the workflows of users
 at our cluster, with lots of NGS users running all and any kind of analyses
 on their typically huge amounts of data. For this we use a parallel file
 system, available on all compute nodes.

 This file system, although approx 1PB in size, is constantly filling up,
 and thus we are not very attracted by the idea of copying files into/out of
 galaxy for each analysis.

 Thus, we would be interested to know what are the options for working with
 existing/external(to galaxy) file systems?

 Eg. would it be possible to link files into some kind of galaxy file
 system (I'm not totally clear about how galaxy stores it's data, although I
 found out that stuff is created in database/files), from outside?

  ... or is there any work going on for selecting any file system path as
 input in galaxy workflows?

 ... or any other hints?

 As said, I'm quite new to galaxy, trying to grok my head around how we can
 use it, so all hints are welcome.

 Best Regards
 // Samuel


 --
 Developer at SNIC-UPPMAX www.uppmax.uu.se
 Developer at Dept of Pharm Biosciences www.farmbio.uu.se
 Twitter - twitter.com/samuellampa
 Blog - saml.rilspace.org
 G+ - gplus.to/saml

 __**_
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Working with files outside galaxy without copying?

2012-11-14 Thread Samuel Lampa

Hi Jorrit, thanks for the input!

On 11/14/2012 03:12 PM, Jorrit Boekel wrote:
This may be implemented, I believe some people run Galaxy on the cluster, 


Yes, but AFAIK, most cluster job runners use some kind of file staging 
(copying) into local storage on the compute nodes OR, require you do 
have the data uploaded in the galaxy file area, and to have this file 
are availble NFS mounted on all compute nodes.


It would be very interesting if there is an existing solution for 
running jobs on files without moving/copying them at all, though!


Best
// Samuel

Galaxy normally operates on files stored in the database/files folder. 
If your Galaxy instance has access to the files you need on your 
parallel fs, I guess you could start by writing a tool that creates 
links to your files when given the path to the input file. Another 
tool may then move the dataset into the desired place while creating a 
link in the database folder.


Sounds like a hack to me, but may work.

cheers,
jorrit




On Wed, Nov 14, 2012 at 2:52 PM, Samuel Lampa samuel.la...@gmail.com 
mailto:samuel.la...@gmail.com wrote:


Hi,

We are looking into ways to integrate galaxy into the workflows of
users at our cluster, with lots of NGS users running all and any
kind of analyses on their typically huge amounts of data. For this
we use a parallel file system, available on all compute nodes.

This file system, although approx 1PB in size, is constantly
filling up, and thus we are not very attracted by the idea of
copying files into/out of galaxy for each analysis.

Thus, we would be interested to know what are the options for
working with existing/external(to galaxy) file systems?

Eg. would it be possible to link files into some kind of galaxy
file system (I'm not totally clear about how galaxy stores it's
data, although I found out that stuff is created in
database/files), from outside?

 ... or is there any work going on for selecting any file system
path as input in galaxy workflows?

... or any other hints?

As said, I'm quite new to galaxy, trying to grok my head around
how we can use it, so all hints are welcome.

Best Regards
// Samuel


-- 
Developer at SNIC-UPPMAX www.uppmax.uu.se http://www.uppmax.uu.se

Developer at Dept of Pharm Biosciences www.farmbio.uu.se
http://www.farmbio.uu.se
Twitter - twitter.com/samuellampa http://twitter.com/samuellampa
Blog - saml.rilspace.org http://saml.rilspace.org
G+ - gplus.to/saml http://gplus.to/saml

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/





--
Developer at SNIC-UPPMAX www.uppmax.uu.se
Developer at Dept of Pharm Biosciences www.farmbio.uu.se
Twitter - twitter.com/samuellampa
Blog - saml.rilspace.org
G+ - gplus.to/saml

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] Trackster error after upgrade

2012-11-14 Thread Daniel Blankenberg
Hi Joachim,

It looks like your integrated_tool_panel.xml file has been corrupted somehow. 
You can move this file to a backup location (or delete) and it should be 
automatically regenerated on Galaxy startup. If this fixes the issue, could you 
share a copy of the file? It might be helpful for us to try to figure out what 
happened to it.


Thanks for using Galaxy,

Dan


On Nov 14, 2012, at 4:25 AM, Joachim Jacob wrote:

 Hi Jeremy,
 
 Thanks for you assistance. I would love to try your suggestions, but Galaxy 
 won't start anymore.
 
 I have found the root cause of the problems: my home partition has become too 
 small, due to a fat 'job_working_directory'. I didn't expect it to become so 
 big (especially with tophat output). Now I have the job_working_directory 
 mounted on our network share. So I restart Galaxy, and got this error:
 
 Traceback (most recent call last):
 File /home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py, line 
 36, in app_factory
   app = UniverseApplication( global_conf = global_conf, **kwargs )
 File /home/galaxy/galaxy-dist/lib/galaxy/app.py, line 85, in __init__
   self.toolbox = tools.ToolBox( tool_configs, self.config.tool_path, self )
 File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 84, in 
 __init__
   self.load_integrated_tool_panel_keys()
 File /home/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py, line 236, in 
 load_integrated_tool_panel_keys
   tree = util.parse_xml( self.integrated_tool_panel_config )
 File /home/galaxy/galaxy-dist/lib/galaxy/util/__init__.py, line 143, in 
 parse_xml
   tree = ElementTree.parse(fname)
 File 
 /home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py,
  line 859, in parse
   tree.parse(source, parser)
 File 
 /home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py,
  line 584, in parse
   self._root = parser.close()
 File 
 /home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py,
  line 1251, in close
   self._parser.Parse(, 1) # end of data
 ExpatError: unclosed token: line 109, column 4
 Removing PID file main.pid
 
 
 Before dragging up my backup, any idea how to fix this otherwise? I suspect 
 that mercurial could not pull all changes in, is that possible?
 
 
 Cheers,
 Joachim
 
 
 On Wed 14 Nov 2012 04:25:53 AM CET, Jeremy Goecks wrote:
 One dataset I want to view in Trackster has an error
 ConverterDependencyException: 'A dependency (bgzip) was in an error state.
 
 I've seen this once or twice before but haven't been able to track down the 
 issue yet.
 
 Here are two workaround that may help you out:
 
 (1) Add the dataset to a new/saved visualization and then click 'try again' 
 to try reindexing the dataset;
 
 (2) copy the dataset using History Options -- Copy Datasets, then delete 
 the failing dataset and try indexing the new dataset.
 
 If the dataset continues to fail, please make it available to me and I'll 
 take a look.
 
 Best,
 J.
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] [galaxy-user] Warning message

2012-11-14 Thread Daniel Blankenberg
Hi Rolando,

Please keep all replies on list.

Without a little more information its hard to guess what is happening. Can you 
provide the log output for when you try to start the server? 

Is this a cloud instance? If I had to make a guess, it would be that there is a 
permissions problem due to the copy. Can you make sure that when you copied the 
.sample file over the .xml file that you did it as the galaxy user? 


Thanks for using Galaxy,

Dan

On Nov 13, 2012, at 10:08 PM, rolandomantil...@gmail.com wrote:

 I just copy the file datatypes_conf.xml.sample over datatypes_conf.xml like 
 you told me and now I can't start galaxy it tells me Galaxy deamon not 
 running,please help
 Sent from my Verizon Wireless BlackBerry
 From: Daniel Blankenberg d...@bx.psu.edu
 Date: Tue, 13 Nov 2012 10:10:43 -0500
 To: Rolando Mantillarolandomantil...@gmail.com
 Cc: galaxy-u...@lists.bx.psu.edu
 Subject: Re: [galaxy-user] Warning message
 
 Hi Rolando,
 
 You'll need to remove the old blast datatypes from your datatypes_conf.xml 
 file. If you haven't added any new datatypes manually yourself, you can copy 
 the datatypes_conf.xml.sample file over of your datatypes_conf.xml file.
 
 Also, in the future questions about local Galaxy instances should be sent to 
 the galaxy-dev mailing list. 
 
 
 Thanks for using Galaxy,
 
 Dan
 
 
 On Nov 12, 2012, at 10:31 PM, Rolando Mantilla wrote:
 
 I'm having issues with the FASTQ_Groomer. What I have done it first I 
 downloaded an SRA file created by an Ion torrent sequencer from the NCBI 
 site. Then used the fastq-dump app from the NCBI site to covert the .sra 
 file to .fastq file. When I uploaded the data into galaxy it recognized it 
 as a fastq(as it should) but when I try to run the FASTQ groomer I get the 
 message and warnings below. I also have already downloaded the the 
 blast_datatypes tool from the tool_shed. I truly don't know what the issue 
 is, any help
 An error occurred running this job: Groomed 12376 sanger reads into sanger 
 reads.   

 WARNING:galaxy.datatypes.registry:Error loading datatype with extension 
 'blastxml': 'module' object has no attribute 'BlastXml'  
 WARNING:galaxy.datatypes.registry:Overriding conflicting datatype with 
 extension 'blastxml', using datatype from /mnt/galaxyData/tmp/tmpdP_cZ7.
 ___
 The Galaxy User list should be used for the discussion of
 Galaxy analysis and other features on the public server
 at usegalaxy.org.  Please keep all replies on the list by
 using reply all in your mail client.  For discussion of
 local Galaxy instances and the Galaxy source code, please
 use the Galaxy Development list:
 
  http://lists.bx.psu.edu/listinfo/galaxy-dev
 
 To manage your subscriptions to this and other Galaxy lists,
 please use the interface at:
 
  http://lists.bx.psu.edu/
 

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Working with files outside galaxy without copying?

2012-11-14 Thread Hans-Rudolf Hotz

Hi Samuel

Have you looked at Data Libraries ?
(http://wiki.galaxyproject.org/Admin/Data%20Libraries/Libraries )

also look at the configurations in universe_wsgi.ini:

library_import_dir
user_library_import_dir
allow_library_path_paste

which gives you many option to access data out side of the Galaxy 
database/ folder - without duplicating the data.


Regards, Hans-Rudolf


On 11/14/2012 03:23 PM, Samuel Lampa wrote:

Hi Jorrit, thanks for the input!

On 11/14/2012 03:12 PM, Jorrit Boekel wrote:

This may be implemented, I believe some people run Galaxy on the cluster,


Yes, but AFAIK, most cluster job runners use some kind of file staging
(copying) into local storage on the compute nodes OR, require you do
have the data uploaded in the galaxy file area, and to have this file
are availble NFS mounted on all compute nodes.

It would be very interesting if there is an existing solution for
running jobs on files without moving/copying them at all, though!

Best
// Samuel


Galaxy normally operates on files stored in the database/files folder.
If your Galaxy instance has access to the files you need on your
parallel fs, I guess you could start by writing a tool that creates
links to your files when given the path to the input file. Another
tool may then move the dataset into the desired place while creating a
link in the database folder.

Sounds like a hack to me, but may work.

cheers,
jorrit




On Wed, Nov 14, 2012 at 2:52 PM, Samuel Lampa samuel.la...@gmail.com
mailto:samuel.la...@gmail.com wrote:

Hi,

We are looking into ways to integrate galaxy into the workflows of
users at our cluster, with lots of NGS users running all and any
kind of analyses on their typically huge amounts of data. For this
we use a parallel file system, available on all compute nodes.

This file system, although approx 1PB in size, is constantly
filling up, and thus we are not very attracted by the idea of
copying files into/out of galaxy for each analysis.

Thus, we would be interested to know what are the options for
working with existing/external(to galaxy) file systems?

Eg. would it be possible to link files into some kind of galaxy
file system (I'm not totally clear about how galaxy stores it's
data, although I found out that stuff is created in
database/files), from outside?

 ... or is there any work going on for selecting any file system
path as input in galaxy workflows?

... or any other hints?

As said, I'm quite new to galaxy, trying to grok my head around
how we can use it, so all hints are welcome.

Best Regards
// Samuel


-- Developer at SNIC-UPPMAX www.uppmax.uu.se
http://www.uppmax.uu.se
Developer at Dept of Pharm Biosciences www.farmbio.uu.se
http://www.farmbio.uu.se
Twitter - twitter.com/samuellampa http://twitter.com/samuellampa
Blog - saml.rilspace.org http://saml.rilspace.org
G+ - gplus.to/saml http://gplus.to/saml

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/






___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


Re: [galaxy-dev] Can't edit galaxy workflow

2012-11-14 Thread Scott McManus

Make sure that you are pulling from galaxy-central and not galaxy-dist.
I believe something like:
hg pull -r rev# -u http://bitbucket.org/galaxy/galaxy-central

should work. The problem may be that you are pointing to galaxy-dist by
default, which will not know about that revision (yet).

-Scott

- Original Message -
 Thanks a lot Dannon, for your reply.
 
 But hg keeps saying following when I try to pull: abort: unknown
 revision '5013377e0bf7'!
 
 Is there any other way of manually getting it?
 
 (Sorry, I'm not an expert on those new SCMs)
 Sanjar.
 
 On 11/13/2012 06:33 PM, Dannon Baker wrote:
  Sanjar,
 
  This is fixed as of 5013377e0bf7.  This may not be in the next
  distribution, but will be in the one after that.  Of course, you
  can manually pull the change from galaxy-central at any time.
 
  -Dannon
 
 
  On Nov 13, 2012, at 9:45 AM, Sanjarbek Hudaiberdiev
  hudai...@icgeb.org wrote:
 
  I tried to reply to similar posts, but couldn't figure out how to
  do it. So posting again:
 
  Galaxy giving this error when editing workflow, just after
  creating workflow from existing history:
 
  URL:
  http://localhost/galaxy/workflow/load_workflow?id=df7a1f0c02a5b08e_=1352817462148
  Module weberror.evalexception.middleware:364 in respond
app_iter = self.application(environ, detect_start_response)
  Module paste.debug.prints:98 in __call__
environ, self.app)
  Module paste.wsgilib:539 in intercept_output
app_iter = application(environ, replacement_start_response)
  Module paste.recursive:80 in __call__
return self.application(environ, start_response)
  Module paste.httpexceptions:632 in __call__
return self.application(environ, start_response)
  Module galaxy.web.framework.base:160 in __call__
body = method( trans, **kwargs )
  Module galaxy.web.framework:73 in decorator
return simplejson.dumps( func( self, trans, *args, **kwargs )
)
  Module galaxy.webapps.galaxy.controllers.workflow:733 in
  load_workflow
'tooltip': module.get_tooltip( static_path=url_for( '/static'
) ),
  Module galaxy.workflow.modules:258 in get_tooltip
return self.tool.help.render( static_path=static_path )
  AttributeError: 'NoneType' object has no attribute 'render'
 
  Could anyone help me to solve this problem?
 
  Thanks,
  Sanjar.
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
 
http://lists.bx.psu.edu/
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
   http://lists.bx.psu.edu/
 
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Galaxy November 14, 2012 Distribution News Brief

2012-11-14 Thread Jennifer Jackson
*Galaxy November 14, 2012 Distribution  News Brief 
http://wiki.galaxyproject.org/News/2012_11_14_DistributionNewsBrief*


*
Complete News Brief http://wiki.galaxyproject.org/DevNewsBriefs/2012_11_14
*

*Highlights:*

 *

   *NGS: Picard (beta)* have moved from the *Galaxy distribution
   https://bitbucket.org/galaxy/galaxy-dist* to the*Galaxy Main Tool
   Shed* http://toolshed.g2.bx.psu.edu/.

 *

   The *Galaxy Project* is now using /Sphinx http://sphinx-doc.org/
   Python/ to document the *galaxy-central
   http://galaxy-central.readthedocs.org* and *galaxy-dist*
   http://galaxy-dist.readthedocs.org code base.

 *

   The *Intergalactic Utilities Commission*
   http://wiki.g2.bx.psu.edu/ReviewingToolShedRepositories will soon
   begin reviewing repositories in the *Galaxy Main Tool Shed
   http://toolshed.g2.bx.psu.edu/* (better repos, better tools)

 *

   *Tool Shed http://toolshed.g2.bx.psu.edu/* *best practice*
   advice: a single tool or a suite of tools per repository?
   http://wiki.galaxyproject.org/AToolOrASuitePerRepository

 *

   *Tophat http://tophat.cbcb.umd.edu/*, *Tophat2
   http://tophat.cbcb.umd.edu/manual.html*, and *Cuffdiff
   http://cufflinks.cbcb.umd.edu/manual.html#cuffdiff* *updated* to
   accept gene annotations in *GFF3
   http://wiki.galaxyproject.org/Learn/Datatypes#GFF3* format.

 *

   Multiple *enhancements* to the *API*
   http://wiki.galaxyproject.org/DevNewsBriefs/2012_11_14#APItargeting user
   and history actions.

 *

   Plus *updates to* *CloudLaunch http://usegalaxy.org/cloud*, new
   *Security Fixes*
   http://wiki.galaxyproject.org/DevNewsBriefs/2012_11_14#Security_Fixes,
   and several usability enhancements for *Datasets*, *Datatypes*,
   *Tools*, and *Tool Shed* functions.

/
http://getgalaxy.org///

/http://bitbucket.org/galaxy/galaxy-dist/

/http://galaxy-dist.readthedocs.org/


new: $ hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist

upgrade: $ hg pull -u -r 5dcbbdfe1087

*Thanks for using Galaxy!*

Jennifer Jackson http://wiki.galaxyproject.org/JenniferJackson  the 
Galaxy Team http://wiki.galaxyproject.org/Galaxy%20Team


/http://galaxyproject.org/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Join (Operate on Genomic Intervals)

2012-11-14 Thread George Maliha
Hello,

I am using the Galaxy public server to conduct a join operating on genomic
intervals. However, my job has not completed in over 2 days. Is there a
bug, and what can I do to speed up the process?

Thank you,

George
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy-related job opening at the FMI in Basel, Switzerland

2012-11-14 Thread Hotz, Hans-Rudolf
Hi

I apologize for abusing the list for a job add:

We are looking for a R/Bioconductor person which is familiar with Galaxy to
help us expanding our sets of NGS tools in Galaxy, see

http://www.fmi.ch/Careers/JobDetails.html?jobID=158



Please feel free to contact me for more details

Regards, Hans-Rudolf



Hans-Rudolf Hotz, PhD
Bioinformatics Support

Friedrich Miescher Institute for Biomedical Research
Maulbeerstrasse 66
4058 Basel/Switzerland


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] ftp file upload error question

2012-11-14 Thread Tony Szempruch
Dear Galaxy Devs,

I have tried to upload 2 files though the ftp upload using FileZila. I
have been getting the following error:

Response:   220 ProFTPD 1.3.4b Server (Galaxy Main Server FTP)
[:::128.118.250.4]
Command:USER **
Response:   331 Password required for **
Command:PASS *
Response:   530 Sorry, the maximum number of clients (3) for this user are
already connected.
Error:  Critical error
Error:  Could not connect to server

I have seen that some one else had this issue and posted online.
What should I do to try to get around this issue?

Thank you

Tony Szempruch

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/