[galaxy-dev] Multiple file box in Galaxy XML

2011-11-28 Thread Roger Ngo
Hi all, I was wondering if it was possible for a user to specify the number
of files to upload in Galaxy and have multiple file input boxes to upload
updated on the wrapper page accordingly.

A user will have from one to many files needed to be uploaded from a job.

Right now I can see that galaxy does have the option for 1 file with the
param file. With an options tagset, I can have the user specify a range.
The easiest solution I can think of right now is to have a 1-N option tag
set and create 1-N file upload boxes corresponding to the option number.

Is there a better way already built in to Galaxy to do this?

Thanks!
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Barcode Splitter: bug

2011-11-28 Thread Yunling Shi
Hi,
 
I tried to use Barcode Splitter in Galaxy and could not get it work. In the 
Library to split pull down list, I could not get anything. I would expect it 
provides me information just like Barcodes to use pulldown list. Is this a bug?
 
Yunling ___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Question about from_work_dir

2011-11-28 Thread Aurélien Bernard

Hi everybody,


I'm currently trying to create a wrapper for one of my program.

This program generates several output file and the name of each output 
file is derived from the name of the main input file.



_In the .xml file I try to get my output files with the following tag_:

outputs
data from_work_dir=${input_file}.out label=${input_file} main 
output format=txt /

...
/outputs

_The corresponding input file tag_:

param name=input_file type=data label=Input label format=alr 
help=Requested file type: ALR /



The problem is that the content of the from_work_dir attribute is not 
interpreted by galaxy (Contrary to the label attribute) and therefore I 
get this error message:


   galaxy.jobs DEBUG 2011-11-28 17:44:24,431 finish(): Could not move
   /home/galaxy/galaxy-dist/database/job_working_directory/326/${input_file}.out
   to /home/galaxy/galaxy-dist/database/files/000/dataset_684.dat as
   directed by from_work_dir


Do I made a syntax error ? Do I misunderstand the function of the 
from_work_dir attribute?



Thank you in advance for your help.

Best regards,


Aurélien Bernard

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Question about from_work_dir

2011-11-28 Thread Jeremy Goecks
Aurelien,

The particular behavior that you need isn't currently supported because the 
value of from_work_dir isn't treated as a template. To work around this 
limitation, you can do the move/copy operations in your tool wrapper; see the 
wrappers for Bowtie and Sicer for examples on how to do this.

Good luck,
J.

On Nov 28, 2011, at 12:21 PM, Aurélien Bernard wrote:

 Hi everybody,
 
 
 I'm currently trying to create a wrapper for one of my program.
 
 This program generates several output file and the name of each output file 
 is derived from the name of the main input file.
 
 
 In the .xml file I try to get my output files with the following tag:
 
 outputs
 data from_work_dir=${input_file}.out label=${input_file} main output 
 format=txt /
 ...
 /outputs
 
 The corresponding input file tag:
 
 param name=input_file type=data label=Input label format=alr 
 help=Requested file type: ALR /
 
 
 The problem is that the content of the from_work_dir attribute is not 
 interpreted by galaxy (Contrary to the label attribute) and therefore I get 
 this error message:
 
 galaxy.jobs DEBUG 2011-11-28 17:44:24,431 finish(): Could not move 
 /home/galaxy/galaxy-dist/database/job_working_directory/326/${input_file}.out 
 to /home/galaxy/galaxy-dist/database/files/000/dataset_684.dat as directed by 
 from_work_dir
 
 Do I made a syntax error ? Do I misunderstand the function of the 
 from_work_dir attribute ?
 
 
 Thank you in advance for your help.
 
 Best regards,
 
 
 Aurélien Bernard
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Running shell script

2011-11-28 Thread Nate Coraor
On Nov 21, 2011, at 11:09 AM, Toqa Manasrah wrote:

 Hi everybody,
  
 I am integrating my software ViSpA inside galaxy and I have two questions:
 1: the main program is a shell script (.sh) which call another shell script, 
 which call Java executables. should I include any interpreter inside the .xml 
 command  tag?
 and what that should be?
  
 2. how to guarantee that the output files destination are on the use local 
 host rather than the server which is running galaxy?

Hi Tuqa,

Can you clarify this second question?  I'm not sure what you're asking.

Thanks,
--nate

  
 Thanks and regards,
  
 Tuqa.
 ___
 The Galaxy User list should be used for the discussion of
 Galaxy analysis and other features on the public server
 at usegalaxy.org.  Please keep all replies on the list by
 using reply all in your mail client.  For discussion of
 local Galaxy instances and the Galaxy source code, please
 use the Galaxy Development list:
 
  http://lists.bx.psu.edu/listinfo/galaxy-dev
 
 To manage your subscriptions to this and other Galaxy lists,
 please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] Can't view data files from published histories, only imported histories

2011-11-28 Thread Liisa Koski
Hi Jeremy,
I'm trying to view boxplots (png) and FastQC (html) output. I  can view 
other output types like tabular in the published histories, but not png or 
html.

Thanks,
Liisa

Liisa Koski 
Bioinformatics Programmer
Phone: 518-309-3079, E-Mail: liisa.ko...@dnalandmarks.ca
Postal Address: DNA LandMarks Inc, St-jean-sur-Richelieu, J3B 6X3 Québec 
CANADA
DNA LandMarks - une compagnie de BASF Plant Science / a BASF Plant Science 
company
Confidentiality notice: The information contained in this e-mail is 
confidential and may be the subject of legal professional privilege. It is 
intended for the authorized use of the individual or entity addressed. If 
the receiver or reader of this message is not the intended recipient, you 
are hereby notified that any disclosure, copying, distribution or use of 
the contents of this message is prohibited. If this email is received in 
error, please accept our apologies, delete all copies from your system, 
and notify us at supp...@dnalandmarks.ca.Confidentialité: 
L'information contenue dans ce courriel est confidentielle et peut être 
assujettie aux règles concernant le secret professionel. L'information 
contenue dans ce courriel est autorisée uniquement pour l'individu ou 
l'entité légale adressée. Si le récipiendaire ou le lecteur de ce message 
n'est pas celui ou celle prévue, vous êtes tenu de ne pas présenter, 
copier, distribuer ou utiliser le contenu de ce message. Si ce courriel 
est reçu par erreur, veuillez nous en excuser, veuillez détruire toutes 
copies de votre système nous informer à supp...@dnalandmarks.ca.



From:
Jeremy Goecks jeremy.goe...@emory.edu
To:
Liisa Koski liisa.ko...@dnalandmarks.ca
Cc:
galaxy-dev@lists.bx.psu.edu
Date:
2011-11-28 13:23
Subject:
Re: [galaxy-dev] Can't view data files from published histories, only 
imported histories



Liisa,

This functionality works fine in some instances, e.g. for this history:

http://main.g2.bx.psu.edu/u/cartman/h/repeats

I'd guess that it's related to the particular dataset type. What type of 
dataset are you trying to view when you see this error? Can you view any 
datasets from the history?

Thanks,
J.

On Nov 28, 2011, at 1:11 PM, Liisa Koski wrote:

Hi, 
I found a weird bug. I am trying to view data files by clicking on the 
'eye' icon from a published history on my local galaxy installation. When 
I click on the eye I get a 'Server Error' and in the log file I get the 
following. 

Error - type 'exceptions.NameError': global name 'data' is not defined 
URL: http://domain:8080/u/user/d/8bdb720fee635874 
File 
'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', 
line 143 in __call__ 
  app_iter = self.application(environ, start_response) 
File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in 
__call__ 
  return self.application(environ, start_response) 
File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 
632 in __call__ 
  return self.application(environ, start_response) 
File 'galaxy_dist/lib/galaxy/web/framework/base.py', line 160 in __call__ 
  body = method( trans, **kwargs ) 
File 'galaxy_dist/lib/galaxy/web/controllers/dataset.py', line 693 in 
display_by_username_and_slug 
  trans.response.set_content_type( data.get_mime() ) 
NameError: global name 'data' is not defined 

If I import the history into my own history pane I can click on the eye 
and see the data with no errors. 

Any ideas how I can see the data from the published histories? 

Thanks, 
Liisa 

Liisa Koski 
Bioinformatics Programmer
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] [galaxy-user] Upload executable in galaxy

2011-11-28 Thread Nate Coraor
On Nov 23, 2011, at 11:10 AM, Giota Kottara wrote:

 Dear galaxy developers, 
 
 I am trying to integrate an executable tool in a private instance of galaxy. 
 I stored the configuration .xml file and the executable in tools folder of 
 Galaxy and the tool appears in the homepage. Nevertheless when i am running 
 the tool it doesn't seem to find the executable. Am I forgetting something? 

Hi Giota,

I've moved this over to the galaxy-dev list since it pertains to tool 
development.

Executables will need to be on the $PATH of the user running Galaxy.  Could you 
share the error messages you receive when you run the tool?

--nate

 
 Thank you in advance
 Giota Kottara
 ___
 The Galaxy User list should be used for the discussion of
 Galaxy analysis and other features on the public server
 at usegalaxy.org.  Please keep all replies on the list by
 using reply all in your mail client.  For discussion of
 local Galaxy instances and the Galaxy source code, please
 use the Galaxy Development list:
 
  http://lists.bx.psu.edu/listinfo/galaxy-dev
 
 To manage your subscriptions to this and other Galaxy lists,
 please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] Error running tool

2011-11-28 Thread Nate Coraor

On Nov 25, 2011, at 8:23 AM, Toqa Manasrah wrote:

 Hello everybody,
 I am trying to run my tool Vispa but every time i try to i got this error 
 messages:
  
 An error occurred running this job: 
 /home/galaxy/galaxy-dist/tools/vispa/main.bash: line 13: ./parser_v.2.sh: No 
 such file or directory
 /home/galaxy/galaxy-dist/tools/vispa/main.bash: line 16: ./vispa.bash: No 
 such file or directory
  
 although all these files are reside under the same directory and have the 
 permission 755.

Hi Tuqa,

The working directory used to execute your job is not the directory in which 
the tool itself resides.  Your main script should use something like `basename 
$0` to determine what directory it's in, and then use that directory to call 
other scripts in the same directory.

--nate

  
 how can I solve the problem?
  
 Thanks and regards,
 Tuqa.
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Shogun Toolbox?

2011-11-28 Thread Kipper Fletez-Brant
Dear List,

I am new to developing for Galaxy and haven't found a good answer to this 
question yet.  Does Galaxy Test Toolshed use or recognize Shogun Toolbox 
(http://www.shogun-toolbox.org/)?  I'm considering writing a module for Galaxy, 
but the underlying Python code relies on Shogun, and I haven't seen a 
suppported 3rd party software list, so I can't tell if Galaxy would be able to 
run the module without my rewriting it not to depend on Shogun.

Thank you,
Kipper Fletez-Brant
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] Shogun Toolbox?

2011-11-28 Thread Vipin TS
Dear Kipper,

We are running most of our tools with the help of Shogun toolbox. Just
configure a recent version of Shogun toolbox to your local machine and add
the python path to your program. By doing this you will be able to call
shogun functionalities in your programs. (Depends on the configuring path
you will be able to find the shared object file and main library files)
Recent version of shogun can be find at
ftp://shogun-toolbox.org/shogun/releases/1.0/sources/shogun-1.0.0.tar.bz2

HTH,
Vipin
Friedrich Miescher Laboratory
of the Max Planck Society
Spemannstrasse 39, 72076
Tuebingen, Germany


Dear List,

 I am new to developing for Galaxy and haven't found a good answer to this
 question yet.  Does Galaxy Test Toolshed use or recognize Shogun Toolbox (
 http://www.shogun-toolbox.org/)?  I'm considering writing a module for
 Galaxy, but the underlying Python code relies on Shogun, and I haven't seen
 a suppported 3rd party software list, so I can't tell if Galaxy would be
 able to run the module without my rewriting it not to depend on Shogun.

 Thank you,
 Kipper Fletez-Brant
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Can't view data files from published histories, only imported histories

2011-11-28 Thread Jeremy Goecks
Liisa,

I'm not able to reproduce your issue on our public server. For instance, here's 
a history that includes both FastQC output and a boxplot; both display 
correctly:

http://main.g2.bx.psu.edu/u/jeremy/h/unnamed-history-2

(Note that I just committed a fix to galaxy-central so that the images in the 
FastQC output show up correctly.)

Some questions that might help us figure out the problem:
*can you reproduce your problem on a public Galaxy instance, such as main or 
test?
*is your Galaxy instance up to date?
*is there something unusual about the datasets that you're using, e.g. are they 
imported on non-standard in some way?

J.

On Nov 28, 2011, at 1:35 PM, Liisa Koski wrote:

 Hi Jeremy, 
 I'm trying to view boxplots (png) and FastQC (html) output. I  can view other 
 output types like tabular in the published histories, but not png or html. 
 
 Thanks, 
 Liisa 
 
 Liisa Koski 
 Bioinformatics Programmer
 Phone: 518-309-3079, E-Mail: liisa.ko...@dnalandmarks.ca
 Postal Address: DNA LandMarks Inc, St-jean-sur-Richelieu, J3B 6X3 Québec 
 CANADA
 
 DNA LandMarks - une compagnie de BASF Plant Science / a BASF Plant Science 
 company
 
 Confidentiality notice: The information contained in this e-mail is 
 confidential and may be the subject of legal professional privilege. It is 
 intended for the authorized use of the individual or entity addressed. If the 
 receiver or reader of this message is not the intended recipient, you are 
 hereby notified that any disclosure, copying, distribution or use of the 
 contents of this message is prohibited. If this email is received in error, 
 please accept our apologies, delete all copies from your system, and notify 
 us at supp...@dnalandmarks.ca.Confidentialité: L'information contenue 
 dans ce courriel est confidentielle et peut être assujettie aux règles 
 concernant le secret professionel. L'information contenue dans ce courriel 
 est autorisée uniquement pour l'individu ou l'entité légale adressée. Si le 
 récipiendaire ou le lecteur de ce message n'est pas celui ou celle prévue, 
 vous êtes tenu de ne pas présenter, copier, distribuer ou utiliser le contenu 
 de ce message. Si ce courriel est reçu par erreur, veuillez nous en excuser, 
 veuillez détruire toutes copies de votre système nous informer à 
 supp...@dnalandmarks.ca. 
 
 
 From: Jeremy Goecks jeremy.goe...@emory.edu
 To:   Liisa Koski liisa.ko...@dnalandmarks.ca
 Cc:   galaxy-dev@lists.bx.psu.edu
 Date: 2011-11-28 13:23
 Subject:  Re: [galaxy-dev] Can't view data files from published 
 histories, only imported histories
 
 
 
 
 Liisa, 
 
 This functionality works fine in some instances, e.g. for this history: 
 
 http://main.g2.bx.psu.edu/u/cartman/h/repeats 
 
 I'd guess that it's related to the particular dataset type. What type of 
 dataset are you trying to view when you see this error? Can you view any 
 datasets from the history? 
 
 Thanks, 
 J. 
 
 On Nov 28, 2011, at 1:11 PM, Liisa Koski wrote: 
 
 Hi, 
 I found a weird bug. I am trying to view data files by clicking on the 'eye' 
 icon from a published history on my local galaxy installation. When I click 
 on the eye I get a 'Server Error' and in the log file I get the following. 
 
 Error - type 'exceptions.NameError': global name 'data' is not defined 
 URL: http://domain:8080/u/user/d/8bdb720fee635874 
 File 
 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', 
 line 143 in __call__ 
  app_iter = self.application(environ, start_response) 
 File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in 
 __call__ 
  return self.application(environ, start_response) 
 File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 
 in __call__ 
  return self.application(environ, start_response) 
 File 'galaxy_dist/lib/galaxy/web/framework/base.py', line 160 in __call__ 
  body = method( trans, **kwargs ) 
 File 'galaxy_dist/lib/galaxy/web/controllers/dataset.py', line 693 in 
 display_by_username_and_slug 
  trans.response.set_content_type( data.get_mime() ) 
 NameError: global name 'data' is not defined 
 
 If I import the history into my own history pane I can click on the eye and 
 see the data with no errors. 
 
 Any ideas how I can see the data from the published histories? 
 
 Thanks, 
 Liisa 
 
 Liisa Koski 
 Bioinformatics Programmer
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/
 
 
 

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Shogun Toolbox?

2011-11-28 Thread Vipin TS
Hi Kipper,

Sorry I got confused, Yes you are right, both ways are fine I think.

--Vipin


Vipin,

 Thanks for quick reply. Pardon my asking, as I said I am new to Galaxy
 development, but if I wanted to share my module, would other users need to
 install Shogun separately, or could I include it as a dependency or 'egg'?



 On Nov 28, 2011, at 2:28 PM, Vipin TS vipin...@gmail.com wrote:

 Dear Kipper,

 We are running most of our tools with the help of Shogun toolbox. Just
 configure a recent version of Shogun toolbox to your local machine and add
 the python path to your program. By doing this you will be able to call
 shogun functionalities in your programs. (Depends on the configuring path
 you will be able to find the shared object file and main library files)
 Recent version of shogun can be find at
 ftp://shogun-toolbox.org/shogun/releases/1.0/sources/shogun-1.0.0.tar.bz2

 HTH,
 Vipin
 Friedrich Miescher Laboratory
 of the Max Planck Society
 Spemannstrasse 39, 72076
 Tuebingen, Germany


 Dear List,

 I am new to developing for Galaxy and haven't found a good answer to this
 question yet.  Does Galaxy Test Toolshed use or recognize Shogun Toolbox (
 http://www.shogun-toolbox.org/)?  I'm considering writing a module for
 Galaxy, but the underlying Python code relies on Shogun, and I haven't seen
 a suppported 3rd party software list, so I can't tell if Galaxy would be
 able to run the module without my rewriting it not to depend on Shogun.

 Thank you,
 Kipper Fletez-Brant
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/



___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] only showing first lines of large output file

2011-11-28 Thread Edward Kirton
hi matthias,

i've also been getting metadata errors (below) for several weeks, but it
works with the set_metadata_externally option (in the universe config file)
set to False.  for large files, it could take many minutes for your job
server to check the metadata, and no new jobs will be run or completed jobs
picked up during this delay, but at least it works reliably.

fyi, the time these errors appeared roughly coincided with when i moved the
data files (database/) to a different disk than the galaxy code.  i don't
know for sure if that's the issue or not, but may be a clue; i will
experiment when i get a chance.

galaxy.datatypes.metadata DEBUG 2011-11-28 12:39:01,129 setting metadata
externally failed for Hist
oryDatasetAssociation 449: External set_meta() not called
galaxy.jobs DEBUG 2011-11-28 12:39:01,221 job 355 ended
galaxy.datatypes.metadata DEBUG 2011-11-28 12:39:01,357 Cleaning up
external metadata files
galaxy.datatypes.metadata DEBUG 2011-11-28 12:39:01,496 Failed to cleanup
MetadataTempFile temp fil
es from
../../../groupdirs/Galaxy/dev/database/tmp/metadata_out_HistoryDatasetAssociation_449_sB_cT
B: No JSON object could be decoded: line 1 column 0 (char 0)


On Wed, Nov 16, 2011 at 3:00 AM, Mattias de Hollander 
m.dehollan...@nioo.knaw.nl wrote:

 **
 Hello,

 I just noticed I get metadata errors in the paster.log file. Can someone
 tell me what they are and if they are related to above problem?

 galaxy.jobs.runners.drmaa DEBUG 2011-11-16 11:55:28,037 (840/588) state 
 change: job finished normally
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:28,231 setting metadata 
 externally failed for HistoryDatasetAssociation 1680: [Errno 2] No such file 
 or directory: 
 '../../galaxyData/tmp/metadata_in_HistoryDatasetAssociation_1680_N7JVtq'
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:28,325 setting metadata 
 externally failed for HistoryDatasetAssociation 1679: [Errno 2] No such file 
 or directory: 
 '../../galaxyData/tmp/metadata_in_HistoryDatasetAssociation_1679_oTzTG_'
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:28,634 setting metadata 
 externally failed for HistoryDatasetAssociation 1678: [Errno 2] No such file 
 or directory: 
 '../../galaxyData/tmp/metadata_in_HistoryDatasetAssociation_1678_MgW15e'galaxy.jobs
  DEBUG 2011-11-16 11:55:29,103 job 840 ended
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:29,103 Cleaning up external 
 metadata files
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:29,152 Failed to cleanup 
 MetadataTempFile temp files from 
 ../../galaxyData/tmp/metadata_out_HistoryDatasetAssociation_1678_UTGZz8: 
 [Errno 2] No such file or directory: 
 '../../galaxyData/tmp/metadata_out_HistoryDatasetAssociation_1678_UTGZz8'
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:29,152 Failed to cleanup 
 external metadata file (filename_in) for HistoryDatasetAssociation_1678: 
 [Errno 2] No such file or directory: 
 '../../galaxyData/tmp/metadata_in_HistoryDatasetAssociation_1678_MgW15e'
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:29,152 Failed to cleanup 
 external metadata file (filename_out) for HistoryDatasetAssociation_1678: 
 [Errno 2] No such file or directory: 
 '../../galaxyData/tmp/metadata_out_HistoryDatasetAssociation_1678_UTGZz8'
 galaxy.datatypes.metadata DEBUG 2011-11-16 11:55:29,153 Failed to cleanup 
 external metadata file (filename_kwds) for HistoryDatasetAssociation_1678: 
 [Errno 2] No such file or directory: 
 '../../galaxyData/tmp/metadata_kwds_HistoryDatasetAssociation_1678_3SDMLo'


 And of course, how to solve it :)

 Any help is appreciated.


 Mattias

 --
 Bioinformatician
 Netherlands Institute of Ecology (NIOO-KNAW)
 Wageningen, the Netherlands


 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Job output not returned from cluster

2011-11-28 Thread Joseph Hargitai
Ed,

we had the classic goof on our cluster with this. 4 nodes could not see the 
/home/galaxy folder due to a missing entry in /etc/fstab. When the jobs hit 
those nodes (which explains the randomness) we got the error message.

Bothersome was the lack of good logs to go on. The error message was too 
generic - however I discovered that Galaxy was depositing the error and our 
messages in the /pbs folder and you could briefly read them before they got 
deleted. There the message was the classic SGE input/output message - 
/home/galaxy file not found.

Hence my follow up question - how can I have galaxy NOT to delete these SGE 
error and out files?

best,
joe


From: Edward Kirton [eskir...@lbl.gov]
Sent: Monday, November 28, 2011 4:15 PM
To: Nate Coraor
Cc: Joseph Hargitai; galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Job output not returned from cluster

hi, we've had this issue too -- in short, the cluster node(s) finish writing 
outfiles to disk, but the file system (inode metadata) isn't updated at the 
galaxy server yet when galaxy checks for the files.

turning the metadata caching off (as recommended on the galaxy wiki) isn't an 
option for me (and the performance hit would be significant), so i added some 
loops around the file checking (5sec sleep and retry up to 6 times).  there 
were a couple of places this probably should be done (not just .[eo]* log files 
but also the outfiles).

i am testing these hacks now but due to the intermittent nature of these 
errors, it'll be a few days before i know if this is working as expected.  once 
vetted, i will put these minor edits in a clone of galaxy-central so the 
changes can be picked up.

ed

On Mon, Oct 24, 2011 at 10:24 AM, Nate Coraor 
n...@bx.psu.edumailto:n...@bx.psu.edu wrote:
Joseph Hargitai wrote:
 Nate,

 this error is intermittent. You resubmit the same job twice or three time and 
 then it works.  Once we are over the midterm exams - which use galaxy - we 
 will try to switch the filesystem from autofs to hard mount. We suspect this 
 to be the issue.

Ah, I suspect this is attribute caching in NFS.  Try mounting with the
option 'noac' and see if it solves the problem.

 Could we suppress e and o SGE style to resolve this issue, or Galaxy wants 
 the o?

The filename is unimportant, but I doubt it's the cause.

 Do you have an idea about the url build for galaxy - ucsc page return when 
 the url is :8080/galaxy and not just /galaxy?

Not off the top of my head.  I have this message marked, I'll take a
look as soon as I have time.

--nate


 thanks,
 joe

 
 From: Nate Coraor [n...@bx.psu.edumailto:n...@bx.psu.edu]
 Sent: Friday, October 21, 2011 10:26 AM
 To: Joseph Hargitai
 Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu
 Subject: Re: [galaxy-dev] Job output not returned from cluster

 Joseph Hargitai wrote:
 
  Hi,
 
  i was browsing through the list and found many entries for this issue but 
  not a definite answer.
 
  We are actually running into this error for simple file uploads from the 
  internal filesystem.

 Hi Joe,

 This error occurs when the job's standard output and error files are not
 found where Galaxy expects them, namely:

 cluster_files_directory/job_id.o
 cluster_files_directory/job_id.e

 Please check your queueing system to make sure it can correctly deliver
 these back from the execution hosts to the specified filesystem.

 --nate

 
  thanks,
  joe
 

  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
 
http://lists.bx.psu.edu/




___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Shogun Toolbox?

2011-11-28 Thread Kipper Fletez-Brant
Vipin,

Thanks for quick reply. Pardon my asking, as I said I am new to Galaxy 
development, but if I wanted to share my module, would other users need to 
install Shogun separately, or could I include it as a dependency or 'egg'?



On Nov 28, 2011, at 2:28 PM, Vipin TS vipin...@gmail.com wrote:

 Dear Kipper, 
 
 We are running most of our tools with the help of Shogun toolbox. Just 
 configure a recent version of Shogun toolbox to your local machine and add 
 the python path to your program. By doing this you will be able to call 
 shogun functionalities in your programs. (Depends on the configuring path you 
 will be able to find the shared object file and main library files) Recent 
 version of shogun can be find at 
 ftp://shogun-toolbox.org/shogun/releases/1.0/sources/shogun-1.0.0.tar.bz2
 
 HTH, 
 Vipin
 Friedrich Miescher Laboratory 
 of the Max Planck Society
 Spemannstrasse 39, 72076 
 Tuebingen, Germany
 
 
 Dear List,
 
 I am new to developing for Galaxy and haven't found a good answer to this 
 question yet.  Does Galaxy Test Toolshed use or recognize Shogun Toolbox 
 (http://www.shogun-toolbox.org/)?  I'm considering writing a module for 
 Galaxy, but the underlying Python code relies on Shogun, and I haven't seen a 
 suppported 3rd party software list, so I can't tell if Galaxy would be able 
 to run the module without my rewriting it not to depend on Shogun.
 
 Thank you,
 Kipper Fletez-Brant
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/
 
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/