date:20150120

Re: [galaxy-dev] Tool development - Selecting a single item from input dataset.

2015-01-20 Thread Peter Cock

I think this is the (relatively new) Galaxy ability to automatically
run N copies of your tool given N input files, making N outputs
and is related to the collections work.

(This is possible if your tool takes a single input file)

Peter

On Tue, Jan 20, 2015 at 6:17 PM, Vimalkumar Velayudhan
 wrote:
> Hi all,
>
> I am trying to create a select box with the possibility of selecting only a
> single item from the input dataset (figure 1). This works fine but the
> option for selecting multiple files is still visible (figure 2). The
> multiple="false" attribute has no effect.
>
> Figure: http://i.imgur.com/oJVFCoF.png
>
> I have the following in my XML.
>
> label="Select Ribo-Seq alignment file" multiple="false" >
> 
>
> Any suggestions?
>
> galaxy-dist revision 5f4c13d622b8
>
>
> Regards,
> Vimalkumar Velayudhan
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Partial automation for generating those twisty R dependency tool shed installation sequences

2015-01-20 Thread Ross

In case anyone needs to prepare a complete set of twisty R/BioC
dependencies in the right order for automated toolshed installation,
https://testtoolshed.g2.bx.psu.edu/view/fubar/biocdepgen_r_3_1_2 contains a
dependency package generator for the iuc package_r_3_1_2 - eg for DESeq2,
you end up with 46 dependent tarballs in addition to DESeq2 itself !!!

Please, DO NOT install this on a public instance - like the tool factory 2
it's intended only for tool development on throw away instances.

Not much documentation - set the tardir to where you want the tarballs
saved, the XML and path prefix needed before the tarball name in
tool_dependencies.xml to point to where the toolshed should download those
packages from (I use a github repo eg) and the names of all the R/BioC
packages showing in sessionInfo() as "other attached packages" after your R
script has run successfully.

I hope this helps minimise the pain for other tool developers.
Let me know how you go, or if anyone wants specific versions for other r
base packages please.



On Tue, Jan 20, 2015 at 8:06 PM, Ross  wrote:

> Hi Bjorn,
> Thanks for caring :)
>
> R/Bioc dependencies are determined by the target R package.
> This realisation made me change tool factory 2 in the test toolshed so it
> now allows R (or perl or python - not bash!) dependency package_r_
> selection. It still allows the local system R but that's not recommended
> for anything reproducible.
> The generated tool will always use that R and so I can generate an RBioC
> dependency generator using that script
> https://wiki.galaxyproject.org/SetUpREnvironment for any available
> toolshed package_r_
> Feedback always appreciated...
>
>
> On Sun, Jan 18, 2015 at 9:25 PM, Björn Grüning 
> wrote:
>
>> Hi Ross,
>>
>> Am 13.01.2015 um 00:13 schrieb Ross:
>> > Hi  Björn,
>> > I'm a bit old fashioned and think I prefer a proper Galaxy tool rather
>> than
>> > a notebook :)
>>
>> :)
>>
>> > So I've set up a temporary demonstration/test site of a
>> > toolfactory generated tool that does what I think I need - can some kind
>> > soul please test it and let me know how it goes ? If it's useful, it
>> needs
>> > to be adjusted to depend on whatever version of package_R you want to
>> work
>> > with - currently just uses the system R for demonstration purposes.
>> >
>> > I used the toolfactory2 (main toolshed) (which now allows any number of
>> > (optionally non editable) parameters!!!) to wrap the script shown at the
>> > bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are
>> > currently three parameters - the names of the R/BioC packages from
>> > sessionInfo(), the local directory where all the tarballs should be
>> stowed
>> > and the XML output prefix to prepend to each row of the generated XML
>> > stanza for tool_dependencies.xml
>> >
>> > The resulting toolshed tarball was uploaded to a local toolshed and then
>> > installed to produce a new tool in the "tool generators" section
>> > - r_bioc_depgen Generate dependencies for R/BioC packages
>> >
>> > If you import the history at
>> http://130.56.252.21/history/list_published
>> > you will see the toolfactory job (#1,#2,#3) - rerunning will show how
>> the
>> > parameters are defined - fugly but it does work.
>> > After generating/uploading/installing the new tool, outputs from a test
>> run
>> > are in #4 and #5 for DESeq
>>
>> This is cool!
>> I have to try this on a few packages!
>> This could also be of interest for our GSOC idea:
>>
>> https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations
>>
>> Thanks Ross for working on this!
>> Bjoern
>>
>> > Comments and suggestions welcomed!
>> >
>> > On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <
>> bjoern.gruen...@gmail.com>
>> > wrote:
>> >
>> >> Hi Ross,
>> >>
>> >> you are absolutely right.
>> >> My download_store repository is exactly for this purpose.
>> >>
>> >> https://github.com/bgruening/download_store
>> >>
>> >> If you are interested we could integrate your additional magic into the
>> >> notebook.
>> >>
>> >> Thanks,
>> >> Bjoern
>> >>
>> >> Am 11.01.2015 um 01:33 schrieb Ross:
>> >>> Hi, Björn,
>> >>> Looks pretty similar!
>> >>> Aren't the links your notebook generates transient? I think if you put
>> >> them
>> >>> into a tool_dependencies.xml, they will fail permanently immediately
>> >> after
>> >>> any of the package authors updates one of the relevant svn
>> repositories?
>> >>>
>> >>> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated
>> so a
>> >>> link that works today like
>> >>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when
>> Rcpp
>> >>> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
>> >>> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement
>> >> (eg)
>> >>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
>> >>> contrib directory?
>> >>>
>> >>> That's why my more complex script downloads all the latest archives
>> into
>> >> my

[galaxy-dev] Oct '14 and Jan'15 release, trouble installing tools from the toolshed, resetting metadata for installed repositories

2015-01-20 Thread Yves Gagnon

Hello fellow galaxy users :

I noticed that trying to install some tools from the main toolshed, 
sometimes it works, others it does not.  In a specific case, trying to 
install clustalw and its dependency, I get this error message :

URL: (redactedURL)/admin_toolshed/prepare_for_install
File '(redacted 
path)/galaxy_dist_dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py',
 
line 364 in respond
  app_iter = self.application(environ, detect_start_response)
File '(redacted 
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File '(redacted 
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', 
line 633 in __call__
  return self.application(environ, start_response)
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py', 
line 132 in __call__
  return self.handle_request( environ, start_response )
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py', 
line 190 in handle_request
  body = method( trans, **kwargs )
File '(redacted 
path)/galaxy_dist_dev/lib/galaxy/web/framework/decorators.py', line 87 in 
decorator
  return func( self, trans, *args, **kwargs )
File '(redacted 
path)/galaxy_dist_dev/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', 
line 1095 in prepare_for_install
  install_repository_manager.handle_tool_shed_repositories( 
installation_dict, using_api=False )
File '(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/install_manager.py', 
line 636 in handle_tool_shed_repositories
  new_tool_panel_section_label=new_tool_panel_section_label )
File '(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py',
 
line 200 in create_repository_dependency_objects
  self.app.installed_repository_manager.activate_repository( 
repository_db_record )
File '(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/installed_repository_manager.py',
 
line 90 in activate_repository
  repository_tools_tups = irmm.get_repository_tools_tups()
File '(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py',
 
line 69 in get_repository_tools_tups
  index, shed_conf_dict = self.tpm.get_shed_tool_conf_dict( 
self.metadata_dict.get( 'shed_config_filename' ) )
TypeError: 'NoneType' object is not iterable



Same goes for resetting all metadata on installed repositories after 
update, 23 out of ~200 tools work, for the rest I get this error in 
paster.log :



tool_shed.galaxy_install.metadata.installed_repository_metadata_manager 
ERROR 2015-01-20 15:18:35,357 Error attempting to reset metadata on 
repository tabular_to_fasta
Traceback (most recent call last):
  File "(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py",
 
line 114, in reset_metadata_on_selected_repositories
self.set_repository( repository )
  File "(redacted 
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py",
 
line 142, in set_repository
super( InstalledRepositoryMetadataManager, self ).set_repository( 
repository )
  File "(redacted 
path)/galaxy_dist_dev/lib/tool_shed/metadata/metadata_generator.py", line 
1066, in set_repository
self.metadata_dict = { 'shed_config_filename' : 
self.shed_config_dict.get( 'config_filename', None ) }
AttributeError: 'NoneType' object has no attribute 'get'


My galaxy contains this line for tool_conf files, and I moved all config 
files from the distrib root to the config folder:

tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml

Did I do something wrong while updating to the latest releases???

Thanks for your help!

Yves Gagnon
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)

2015-01-20 Thread Aysam Guerler

Hey Damion,

Thanks a lot for the description. We are currently switching the tool form
to a newer version but I will look into this fix soon and should be able
transfer it.

Thanks,
Sam

On Tue, Jan 20, 2015 at 2:45 PM, Dooley, Damion 
wrote:

> I was successful getting library dataset selection and form submission to
> happen within tool form tool form input  -
> Yay!  But it appears one last fix (besides the "url:
> "/visualization/list_libraries",' bug) is needed to enable history tool
> item to be RERUN.
>
> So if a galaxy dev can tackle this, great!  Otherwise I'll have to drop
> this approach :<(
>
> *
> For others who may need...
>
> tool form's param:
>
>  label="Reference Bin file" help="Select one reference bin at a time from
> the popup window's Reference Bin library."/>
>
>  tag contents
>
> #if $library_datasets
> -B "
> #for $i, $dataset in enumerate($library_datasets)
> #$dataset.get_file_name()
> $dataset.id
> #end for
> "
> #end if
>
> Successfully submits either the full file path of selected dataset, or its
> id.
>
> *
> However trying to rerun this history tool item gets us an error (not
> related to the above  contents):
>
> Error Traceback:
> View as:   Interactive  |  Text  |  XML (full)
> ⇝ AttributeError: 'int' object has no attribute 'id'
> 
> Module galaxy.tools.parameters.basic:146 in value_to_basic
>   view
> >>  return self.to_string( value, app )
> Module galaxy.tools.parameters.basic:2406 in to_string
> app 
> ldda331
> self object at 0x7fced01d2090>
> value   [331]
> >>  return [ldda.id for ldda in value]
> AttributeError: 'int' object has no attribute 'id'
>
> which is in /lib/galaxy/tools/parameters/basic.py:
>
> class LibraryDatasetToolParameter( ToolParameter ):
> """
> Parameter that lets users select a LDDA from a modal window,
> then use it within the wrapper.
> """
>
> ...
> def to_string( self, value, app ):
> if not value:
> return value
> return [ldda.id for ldda in value]
>
> "Value" is getting an array of dataset ids, whereas this method is
> expecting value to be a list of dataset objects.  I tried changing ldda.id
> to just "id" but found other parts of  LibraryDatasetTool code were broken
> as they too were expecting an object.  Maybe this is related to the tool
> job's history detail link showing "Reference Bin file" as a string label of
> an object?:
>
> Input Parameter Value   Note for rerun
> BLAST results as XML240: megablast Pasted Entry vs
> 16SMicrobial.fasta
> Reference Bin file
> []
> Tabular Report Column LabelsShort name
> HTML Report templatetemplates.html_report
>
>
>
> 
> The other bug:
>
> 2) Tool_form.mako has a hardcoded url that doesn't work for galaxy
> installs that have a prefix so that needs adjustment:
>
>   $(".add-librarydataset").click(function() {
> var link = $(this);
> $.ajax({
> url: "/visualization/list_libraries",
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)

2015-01-20 Thread Dooley, Damion

I was successful getting library dataset selection and form submission to 
happen within tool form tool form input  - Yay!  
But it appears one last fix (besides the "url: 
"/visualization/list_libraries",' bug) is needed to enable history tool item to 
be RERUN.

So if a galaxy dev can tackle this, great!  Otherwise I'll have to drop this 
approach :<( 

*
For others who may need...

tool form's param:



 tag contents

#if $library_datasets
-B "
#for $i, $dataset in enumerate($library_datasets)
#$dataset.get_file_name()
$dataset.id
#end for
"
#end if

Successfully submits either the full file path of selected dataset, or its id.

*
However trying to rerun this history tool item gets us an error (not related to 
the above  contents):

Error Traceback:
View as:   Interactive  |  Text  |  XML (full)
⇝ AttributeError: 'int' object has no attribute 'id'

Module galaxy.tools.parameters.basic:146 in value_to_basic  view
>>  return self.to_string( value, app )
Module galaxy.tools.parameters.basic:2406 in to_string 
app 
ldda331
self
value   [331]
>>  return [ldda.id for ldda in value]
AttributeError: 'int' object has no attribute 'id'

which is in /lib/galaxy/tools/parameters/basic.py:

class LibraryDatasetToolParameter( ToolParameter ):
"""
Parameter that lets users select a LDDA from a modal window, then 
use it within the wrapper.
"""

...
def to_string( self, value, app ):
if not value:
return value
return [ldda.id for ldda in value]

"Value" is getting an array of dataset ids, whereas this method is expecting 
value to be a list of dataset objects.  I tried changing ldda.id to just "id" 
but found other parts of  LibraryDatasetTool code were broken as they too were 
expecting an object.  Maybe this is related to the tool job's history detail 
link showing "Reference Bin file" as a string label of an object?:

Input Parameter Value   Note for rerun
BLAST results as XML240: megablast Pasted Entry vs 
16SMicrobial.fasta   
Reference Bin file  []  
Tabular Report Column LabelsShort name  
HTML Report templatetemplates.html_report




The other bug:

2) Tool_form.mako has a hardcoded url that doesn't work for galaxy installs 
that have a prefix so that needs adjustment:

  $(".add-librarydataset").click(function() {
var link = $(this);
$.ajax({
url: "/visualization/list_libraries",
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Tool development - Selecting a single item from input dataset.

2015-01-20 Thread Vimalkumar Velayudhan

Hi all,

I am trying to create a select box with the possibility of selecting only a
single item from the input dataset (figure 1). This works fine but the
option for selecting multiple files is still visible (figure 2). The
multiple="false" attribute has no effect.

Figure: http://i.imgur.com/oJVFCoF.png

I have the following in my XML.




Any suggestions?

galaxy-dist revision 5f4c13d622b8


Regards,
Vimalkumar Velayudhan
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

2015-01-20 Thread John Chilton

Are you using the local job runner (this will be the case if you
haven't explicitly configured something like pbs or DRMAA in your
job_conf.xml file)?

-John

On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier
 wrote:
> On 01/20/2015 06:20 PM, Wolfgang Maier wrote:
>>
>>
>> I have not seen this error before (I believe not with latest_2014.10.06)
>
>
> update: confirmed this now. It's enough to hg update to latest_2014.10.06
> and things are working again.
>
> The difference is that when building the dependency shell command the latest
> release seems to put the call to set_metadata.sh into that command, while
> before it seems it was run separately.
>
>
> Wolfgang
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

2015-01-20 Thread Wolfgang Maier


On 01/20/2015 06:20 PM, Wolfgang Maier wrote:


I have not seen this error before (I believe not with latest_2014.10.06)


update: confirmed this now. It's enough to hg update to 
latest_2014.10.06 and things are working again.


The difference is that when building the dependency shell command the 
latest release seems to put the call to set_metadata.sh into that 
command, while before it seems it was run separately.


Wolfgang

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Problem with Galaxy installation

2015-01-20 Thread FRITSCH Emilie - L. Int

Dear Martin,

Thanks for your answer. I found what the problem was, I had forgotten to delete 
the # in #port when I modified the galaxy.ini. It now works perfectly fine.

Thanks a lot again,
Emilie

De : Martin Čech mailto:mar...@bx.psu.edu>>
Date : mardi 20 janvier 2015 16:43
À : Emilie Fritsch mailto:e...@lesaffre.fr>>, 
"galaxy-dev@lists.galaxyproject.org" 
mailto:galaxy-dev@lists.galaxyproject.org>>
Objet : Re: [galaxy-dev] Problem with Galaxy installation

Hello Emilie,

this error often indicates that previous Galaxy process is still running and 
occupying the port. Try killing it.

If it is not the case could you please share your Galaxy configuration file 
with us (you can use e.g. https://gist.github.com/ ). Make sure you censor any 
login/password information e.g. DB access before sharing.

Thanks

Martin

On Tue Jan 20 2015 at 10:29:03 AM FRITSCH Emilie - L. Int 
mailto:e...@lesaffre.fr>> wrote:
Dear all,

I installed Galaxy last week on our server following the tutorial on 
Admin/GetGalaxy from the wiki. I had no problem and it was running fine. 
However, as one of my colleague required the localhost:8080,  I had to change 
the port number. I followed the tutorial on the wiki « Running galaxy in a 
production environment ». I copied the galaxy.ini.sample, modified the port and 
also the additional parameters as suggested. Since then, I get an error message 
stating that "socket.error: [Errno 98] Address already in use ».  I checked and 
the port is free.
I copy at the end of this message part of the standard and error output, hoping 
this could help figuring out what the problem is. Additionally I noticed that 
there was another warning, right at the beginning, when running the run.sh 
script:
/data/database/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1:
 RuntimeWarning: __builtin__.file size changed, may indicate binary 
incompatibility

I have checked on previous messages to see if someone had the same problem but 
usually it seemed that only changing the port in config/galaxy.ini worked.

Thanks a lot in advance, I am newbie and am a bit puzzled on that one,
Emilie

Starting server in PID 27030.
Traceback (most recent call last):
  File "./scripts/paster.py", line 33, in 
serve.run()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
1049, in run
invoke(command, command_name, options, args[1:])
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
1055, in invoke
exit_code = runner.run(args)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
220, in run
result = self.command()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
670, in command
serve()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
654, in serve
server(app)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", 
line 292, in server_wrapper
**context.local_conf)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", 
line 97, in fix_call
val = callable(*args, **kw)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1342, in server_runner
serve(wsgi_app, **kwargs)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1291, in serve
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1134, in __init__
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1113, in __init__
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 360, in __init__
HTTPServer.__init__(self, server_address, RequestHandlerClass)
  File "/usr/lib64/python2.6/SocketServer.py", line 412, in __init__
self.server_bind()
  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
  File "/usr/lib64/python2.6/SocketServer.py", line 423, in server_bind
self.socket.bind(self.server_address)
  File "", line 1, in bind
socket.error: [Errno 98] Address already in use
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  ht

[galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

2015-01-20 Thread Wolfgang Maier


Hi,

I am not sure whether I am diagnosing this problem approximately right 
or not, but it seems that with the latest release of Galaxy (stable 
branch, latest_2015.01.13) enviroment variables set in a 
tool-dependency's env.sh file and sourced before a job run are now 
leaking through to


./scripts/set_metadata.py called after the tool run finishes.

In my specific case, the sourced env.sh adds a python 3 virtualenv 
directory to PATH, which then causes set_metadata.py to fail with:


  File "./scripts/set_metadata.py", line 130
except Exception, e:
^
SyntaxError: invalid syntax

because that (amazingly enough :)) seems to be the first line not 
compatible with Python 3.


I have not seen this error before (I believe not with latest_2014.10.06) 
so I guess my question is: has the way that set_metadata.py is called 
been changed recently and would you consider the current behavior a bug 
or is this expected and ok ?


Best,
Wolfgang

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Web/Job scaling question

2015-01-20 Thread Nate Coraor

Hi Josephine,

The uWSGI setup is more robust, both in terms of scalability, and load
balancing. I'd suggest it for any multiuser setup at this point.

--nate

On Mon, Jan 5, 2015 at 3:10 PM, Josephine Palencia  wrote:

>
> Hello,
>
> Current setup:
>   : have 1-2 VMs w/ 4-core each, 4GB mem mounting remote lustre shared fs
>   : use remote job submission: cli/ssh
>   : proxied, use remote auth login/logout
>
>
> I'm prepping for a scenario where there will be 60-75 users/students
> who will  simultaneously submit several complete genomics  workflows
> to the single VM (I actually have 3 identical VMs). Full runs can go from
> 30min to an hour and a half each.
>
> I have  1  of the VMs set up with the standalone paste-based
> with 5 job handler and 5 web server processes. Would this be enough for
> this scenario? (or 10/10, 20/20, more?).
>
> I have the 2nd VM being set up using the uWSGI method.
>
> Would it be worth it to proceed with the standalone paste-based or just
> skip it and exclusively only concentrate on the uWSGI setup (or keep
> both?). I'd like to automate everything (remote job  submissions for full
> workflows) and just let it (60 test users  or so) run for  1 month to check
> reliability, anticipate problems.
>
> I'd appreciate feedback, thoughts from those who have already gone through
> a similar experience.
>
> TIA
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Problem with Galaxy installation

2015-01-20 Thread Martin Čech

Hello Emilie,

this error often indicates that previous Galaxy process is still running
and occupying the port. Try killing it.

If it is not the case could you please share your Galaxy configuration file
with us (you can use e.g. https://gist.github.com/ ). Make sure you censor
any login/password information e.g. DB access before sharing.

Thanks

Martin

On Tue Jan 20 2015 at 10:29:03 AM FRITSCH Emilie - L. Int 
wrote:

>  Dear all,
>
>  I installed Galaxy last week on our server following the tutorial on
> Admin/GetGalaxy from the wiki. I had no problem and it was running fine.
> However, as one of my colleague required the localhost:8080,  I had to
> change the port number. I followed the tutorial on the wiki « Running
> galaxy in a production environment ». I copied the galaxy.ini.sample,
> modified the port and also the additional parameters as suggested. Since
> then, I get an error message stating that "socket.error: [Errno 98] Address
> already in use ».  I checked and the port is free.
>  I copy at the end of this message part of the standard and error output,
> hoping this could help figuring out what the problem is. Additionally I
> noticed that there was another warning, right at the beginning, when
> running the run.sh script:
>  
> /data/database/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1:
> RuntimeWarning: __builtin__.file size changed, may indicate binary
> incompatibility
>
>  I have checked on previous messages to see if someone had the same
> problem but usually it seemed that only changing the port in
> config/galaxy.ini worked.
>
>  Thanks a lot in advance, I am newbie and am a bit puzzled on that one,
>  Emilie
>
>  Starting server in PID 27030.
> Traceback (most recent call last):
>   File "./scripts/paster.py", line 33, in 
> serve.run()
>   File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py",
> line 1049, in run
> invoke(command, command_name, options, args[1:])
>   File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py",
> line 1055, in invoke
> exit_code = runner.run(args)
>   File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py",
> line 220, in run
> result = self.command()
>   File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py",
> line 670, in command
> serve()
>   File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py",
> line 654, in serve
> server(app)
>   File
> "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line
> 292, in server_wrapper
> **context.local_conf)
>   File
> "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line
> 97, in fix_call
> val = callable(*args, **kw)
>   File
> "/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> line 1342, in server_runner
> serve(wsgi_app, **kwargs)
>   File
> "/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> line 1291, in serve
> request_queue_size=request_queue_size)
>   File
> "/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> line 1134, in __init__
> request_queue_size=request_queue_size)
>   File
> "/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> line 1113, in __init__
> request_queue_size=request_queue_size)
>   File
> "/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py",
> line 360, in __init__
> HTTPServer.__init__(self, server_address, RequestHandlerClass)
>   File "/usr/lib64/python2.6/SocketServer.py", line 412, in __init__
> self.server_bind()
>   File "/usr/lib64/python2.6/BaseHTTPServer.py", line 108, in server_bind
> SocketServer.TCPServer.server_bind(self)
>   File "/usr/lib64/python2.6/SocketServer.py", line 423, in server_bind
> self.socket.bind(self.server_address)
>   File "", line 1, in bind
> socket.error: [Errno 98] Address already in use
>  ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Problem with Galaxy installation

2015-01-20 Thread FRITSCH Emilie - L. Int

Dear all,

I installed Galaxy last week on our server following the tutorial on 
Admin/GetGalaxy from the wiki. I had no problem and it was running fine. 
However, as one of my colleague required the localhost:8080,  I had to change 
the port number. I followed the tutorial on the wiki « Running galaxy in a 
production environment ». I copied the galaxy.ini.sample, modified the port and 
also the additional parameters as suggested. Since then, I get an error message 
stating that "socket.error: [Errno 98] Address already in use ».  I checked and 
the port is free.
I copy at the end of this message part of the standard and error output, hoping 
this could help figuring out what the problem is. Additionally I noticed that 
there was another warning, right at the beginning, when running the run.sh 
script:
/data/database/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1:
 RuntimeWarning: __builtin__.file size changed, may indicate binary 
incompatibility

I have checked on previous messages to see if someone had the same problem but 
usually it seemed that only changing the port in config/galaxy.ini worked.

Thanks a lot in advance, I am newbie and am a bit puzzled on that one,
Emilie

Starting server in PID 27030.
Traceback (most recent call last):
  File "./scripts/paster.py", line 33, in 
serve.run()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
1049, in run
invoke(command, command_name, options, args[1:])
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
1055, in invoke
exit_code = runner.run(args)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
220, in run
result = self.command()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
670, in command
serve()
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 
654, in serve
server(app)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", 
line 292, in server_wrapper
**context.local_conf)
  File "/data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", 
line 97, in fix_call
val = callable(*args, **kw)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1342, in server_runner
serve(wsgi_app, **kwargs)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1291, in serve
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1134, in __init__
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 1113, in __init__
request_queue_size=request_queue_size)
  File 
"/data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", 
line 360, in __init__
HTTPServer.__init__(self, server_address, RequestHandlerClass)
  File "/usr/lib64/python2.6/SocketServer.py", line 412, in __init__
self.server_bind()
  File "/usr/lib64/python2.6/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
  File "/usr/lib64/python2.6/SocketServer.py", line 423, in server_bind
self.socket.bind(self.server_address)
  File "", line 1, in bind
socket.error: [Errno 98] Address already in use
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Partial automation for generating those twisty R dependency tool shed installation sequences

2015-01-20 Thread Ross

Hi Bjorn,
Thanks for caring :)

R/Bioc dependencies are determined by the target R package.
This realisation made me change tool factory 2 in the test toolshed so it
now allows R (or perl or python - not bash!) dependency package_r_
selection. It still allows the local system R but that's not recommended
for anything reproducible.
The generated tool will always use that R and so I can generate an RBioC
dependency generator using that script
https://wiki.galaxyproject.org/SetUpREnvironment for any available toolshed
package_r_
Feedback always appreciated...


On Sun, Jan 18, 2015 at 9:25 PM, Björn Grüning 
wrote:

> Hi Ross,
>
> Am 13.01.2015 um 00:13 schrieb Ross:
> > Hi  Björn,
> > I'm a bit old fashioned and think I prefer a proper Galaxy tool rather
> than
> > a notebook :)
>
> :)
>
> > So I've set up a temporary demonstration/test site of a
> > toolfactory generated tool that does what I think I need - can some kind
> > soul please test it and let me know how it goes ? If it's useful, it
> needs
> > to be adjusted to depend on whatever version of package_R you want to
> work
> > with - currently just uses the system R for demonstration purposes.
> >
> > I used the toolfactory2 (main toolshed) (which now allows any number of
> > (optionally non editable) parameters!!!) to wrap the script shown at the
> > bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are
> > currently three parameters - the names of the R/BioC packages from
> > sessionInfo(), the local directory where all the tarballs should be
> stowed
> > and the XML output prefix to prepend to each row of the generated XML
> > stanza for tool_dependencies.xml
> >
> > The resulting toolshed tarball was uploaded to a local toolshed and then
> > installed to produce a new tool in the "tool generators" section
> > - r_bioc_depgen Generate dependencies for R/BioC packages
> >
> > If you import the history at http://130.56.252.21/history/list_published
> > you will see the toolfactory job (#1,#2,#3) - rerunning will show how the
> > parameters are defined - fugly but it does work.
> > After generating/uploading/installing the new tool, outputs from a test
> run
> > are in #4 and #5 for DESeq
>
> This is cool!
> I have to try this on a few packages!
> This could also be of interest for our GSOC idea:
>
> https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations
>
> Thanks Ross for working on this!
> Bjoern
>
> > Comments and suggestions welcomed!
> >
> > On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <
> bjoern.gruen...@gmail.com>
> > wrote:
> >
> >> Hi Ross,
> >>
> >> you are absolutely right.
> >> My download_store repository is exactly for this purpose.
> >>
> >> https://github.com/bgruening/download_store
> >>
> >> If you are interested we could integrate your additional magic into the
> >> notebook.
> >>
> >> Thanks,
> >> Bjoern
> >>
> >> Am 11.01.2015 um 01:33 schrieb Ross:
> >>> Hi, Björn,
> >>> Looks pretty similar!
> >>> Aren't the links your notebook generates transient? I think if you put
> >> them
> >>> into a tool_dependencies.xml, they will fail permanently immediately
> >> after
> >>> any of the package authors updates one of the relevant svn
> repositories?
> >>>
> >>> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated
> so a
> >>> link that works today like
> >>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when
> Rcpp
> >>> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
> >>> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement
> >> (eg)
> >>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
> >>> contrib directory?
> >>>
> >>> That's why my more complex script downloads all the latest archives
> into
> >> my
> >>> local github archive repo and generates a permanent link to suit that
> >>> github repo.
> >>> We definitely need an automated solution as this is a really
> infuriating
> >>> aspect of trying to make code relying on R/BioC packages reproducible.
> >>>
> >>>
> >>> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <
> >> bjoern.gruen...@gmail.com>
> >>> wrote:
> >>>
>  Hi Ross,
> 
>  this is great!
>  Have you seen this notebook?
> 
> 
> 
> >>
> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
> 
>  It tries to do the same thing. Maybe it's also worth to mention? Maybe
>  we can enhance it?
> 
>  Thanks,
>  Bjoern
> 
>  Am 08.01.2015 um 08:09 schrieb Ross:
> > This may be helpful for anyone else struggling to get complex nested
> R
> > package dependency installation from the tool shed sorted out. That
> >> whole
> > can of worms. While we have setup_r_packages, the developer still has
> >> to
> > figure out the right magical incantation and make sure the tarballs
> are
> > available.
> >
> > https://wiki.galaxyproject.org/SetUpREnvironment has

Re: [galaxy-dev] Tool development - Selecting a single item from input dataset.

Re: [galaxy-dev] Partial automation for generating those twisty R dependency tool shed installation sequences

[galaxy-dev] Oct '14 and Jan'15 release, trouble installing tools from the toolshed, resetting metadata for installed repositories

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)

[galaxy-dev] Tool development - Selecting a single item from input dataset.

Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

Re: [galaxy-dev] Problem with Galaxy installation

[galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py

Re: [galaxy-dev] Web/Job scaling question

Re: [galaxy-dev] Problem with Galaxy installation

[galaxy-dev] Problem with Galaxy installation

Re: [galaxy-dev] Partial automation for generating those twisty R dependency tool shed installation sequences

14 matches

Site Navigation

Mail list logo

Footer information