Re: [galaxy-dev] Problem with Galaxy installation
Dear Martin, Thanks for your answer. I found what the problem was, I had forgotten to delete the # in #port when I modified the galaxy.ini. It now works perfectly fine. Thanks a lot again, Emilie De : Martin Čech mar...@bx.psu.edumailto:mar...@bx.psu.edu Date : mardi 20 janvier 2015 16:43 À : Emilie Fritsch e...@lesaffre.frmailto:e...@lesaffre.fr, galaxy-dev@lists.galaxyproject.orgmailto:galaxy-dev@lists.galaxyproject.org galaxy-dev@lists.galaxyproject.orgmailto:galaxy-dev@lists.galaxyproject.org Objet : Re: [galaxy-dev] Problem with Galaxy installation Hello Emilie, this error often indicates that previous Galaxy process is still running and occupying the port. Try killing it. If it is not the case could you please share your Galaxy configuration file with us (you can use e.g. https://gist.github.com/ ). Make sure you censor any login/password information e.g. DB access before sharing. Thanks Martin On Tue Jan 20 2015 at 10:29:03 AM FRITSCH Emilie - L. Int e...@lesaffre.frmailto:e...@lesaffre.fr wrote: Dear all, I installed Galaxy last week on our server following the tutorial on Admin/GetGalaxy from the wiki. I had no problem and it was running fine. However, as one of my colleague required the localhost:8080, I had to change the port number. I followed the tutorial on the wiki « Running galaxy in a production environment ». I copied the galaxy.ini.sample, modified the port and also the additional parameters as suggested. Since then, I get an error message stating that socket.error: [Errno 98] Address already in use ». I checked and the port is free. I copy at the end of this message part of the standard and error output, hoping this could help figuring out what the problem is. Additionally I noticed that there was another warning, right at the beginning, when running the run.sh script: /data/database/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility I have checked on previous messages to see if someone had the same problem but usually it seemed that only changing the port in config/galaxy.ini worked. Thanks a lot in advance, I am newbie and am a bit puzzled on that one, Emilie Starting server in PID 27030. Traceback (most recent call last): File ./scripts/paster.py, line 33, in module serve.run() File /data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1049, in run invoke(command, command_name, options, args[1:]) File /data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 1055, in invoke exit_code = runner.run(args) File /data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 220, in run result = self.command() File /data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 670, in command serve() File /data/database/galaxy-dist/lib/galaxy/util/pastescript/serve.py, line 654, in serve server(app) File /data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 292, in server_wrapper **context.local_conf) File /data/database/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py, line 97, in fix_call val = callable(*args, **kw) File /data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py, line 1342, in server_runner serve(wsgi_app, **kwargs) File /data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py, line 1291, in serve request_queue_size=request_queue_size) File /data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py, line 1134, in __init__ request_queue_size=request_queue_size) File /data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py, line 1113, in __init__ request_queue_size=request_queue_size) File /data/database/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py, line 360, in __init__ HTTPServer.__init__(self, server_address, RequestHandlerClass) File /usr/lib64/python2.6/SocketServer.py, line 412, in __init__ self.server_bind() File /usr/lib64/python2.6/BaseHTTPServer.py, line 108, in server_bind SocketServer.TCPServer.server_bind(self) File /usr/lib64/python2.6/SocketServer.py, line 423, in server_bind self.socket.bind(self.server_address) File string, line 1, in bind socket.error: [Errno 98] Address already in use ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other
Re: [galaxy-dev] Web/Job scaling question
Hi Josephine, The uWSGI setup is more robust, both in terms of scalability, and load balancing. I'd suggest it for any multiuser setup at this point. --nate On Mon, Jan 5, 2015 at 3:10 PM, Josephine Palencia josep...@psc.edu wrote: Hello, Current setup: : have 1-2 VMs w/ 4-core each, 4GB mem mounting remote lustre shared fs : use remote job submission: cli/ssh : proxied, use remote auth login/logout I'm prepping for a scenario where there will be 60-75 users/students who will simultaneously submit several complete genomics workflows to the single VM (I actually have 3 identical VMs). Full runs can go from 30min to an hour and a half each. I have 1 of the VMs set up with the standalone paste-based with 5 job handler and 5 web server processes. Would this be enough for this scenario? (or 10/10, 20/20, more?). I have the 2nd VM being set up using the uWSGI method. Would it be worth it to proceed with the standalone paste-based or just skip it and exclusively only concentrate on the uWSGI setup (or keep both?). I'd like to automate everything (remote job submissions for full workflows) and just let it (60 test users or so) run for 1 month to check reliability, anticipate problems. I'd appreciate feedback, thoughts from those who have already gone through a similar experience. TIA ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Tool development - Selecting a single item from input dataset.
Hi all, I am trying to create a select box with the possibility of selecting only a single item from the input dataset (figure 1). This works fine but the option for selecting multiple files is still visible (figure 2). The multiple=false attribute has no effect. Figure: http://i.imgur.com/oJVFCoF.png I have the following in my XML. param format=tabular name=ribo_files type=data label=Select Ribo-Seq alignment file multiple=false /param Any suggestions? galaxy-dist revision 5f4c13d622b8 Regards, Vimalkumar Velayudhan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
On 01/20/2015 06:20 PM, Wolfgang Maier wrote: I have not seen this error before (I believe not with latest_2014.10.06) update: confirmed this now. It's enough to hg update to latest_2014.10.06 and things are working again. The difference is that when building the dependency shell command the latest release seems to put the call to set_metadata.sh into that command, while before it seems it was run separately. Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool param type library_data (Dooley, Damion)
Hey Damion,
Thanks a lot for the description. We are currently switching the tool form
to a newer version but I will look into this fix soon and should be able
transfer it.
Thanks,
Sam
On Tue, Jan 20, 2015 at 2:45 PM, Dooley, Damion damion.doo...@bccdc.ca
wrote:
I was successful getting library dataset selection and form submission to
happen within tool form tool form input param type=library_data ... -
Yay! But it appears one last fix (besides the url:
/visualization/list_libraries,' bug) is needed to enable history tool
item to be RERUN.
So if a galaxy dev can tackle this, great! Otherwise I'll have to drop
this approach :(
*
For others who may need...
tool form's param:
param name=library_datasets type=library_data
label=Reference Bin file help=Select one reference bin at a time from
the popup window's Reference Bin library./
command tag contents
#if $library_datasets
-B
#for $i, $dataset in enumerate($library_datasets)
#$dataset.get_file_name()
$dataset.id
#end for
#end if
Successfully submits either the full file path of selected dataset, or its
id.
*
However trying to rerun this history tool item gets us an error (not
related to the above command contents):
Error Traceback:
View as: Interactive | Text | XML (full)
⇝ AttributeError: 'int' object has no attribute 'id'
Module galaxy.tools.parameters.basic:146 in value_to_basic
view
return self.to_string( value, app )
Module galaxy.tools.parameters.basic:2406 in to_string
app galaxy.app.UniverseApplication object at 0x59a0710
ldda331
selfgalaxy.tools.parameters.basic.LibraryDatasetToolParameter
object at 0x7fced01d2090
value [331]
return [ldda.id for ldda in value]
AttributeError: 'int' object has no attribute 'id'
which is in /lib/galaxy/tools/parameters/basic.py:
class LibraryDatasetToolParameter( ToolParameter ):
Parameter that lets users select a LDDA from a modal window,
then use it within the wrapper.
...
def to_string( self, value, app ):
if not value:
return value
return [ldda.id for ldda in value]
Value is getting an array of dataset ids, whereas this method is
expecting value to be a list of dataset objects. I tried changing ldda.id
to just id but found other parts of LibraryDatasetTool code were broken
as they too were expecting an object. Maybe this is related to the tool
job's history detail link showing Reference Bin file as a string label of
an object?:
Input Parameter Value Note for rerun
BLAST results as XML240: megablast Pasted Entry vs
16SMicrobial.fasta
Reference Bin file
[galaxy.model.LibraryDatasetDatasetAssociation object at 0x7fcee04d4350]
Tabular Report Column LabelsShort name
HTML Report templatetemplates.html_report
The other bug:
2) Tool_form.mako has a hardcoded url that doesn't work for galaxy
installs that have a prefix so that needs adjustment:
$(.add-librarydataset).click(function() {
var link = $(this);
$.ajax({
url: /visualization/list_libraries,
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Oct '14 and Jan'15 release, trouble installing tools from the toolshed, resetting metadata for installed repositories
Hello fellow galaxy users :
I noticed that trying to install some tools from the main toolshed,
sometimes it works, others it does not. In a specific case, trying to
install clustalw and its dependency, I get this error message :
URL: (redactedURL)/admin_toolshed/prepare_for_install
File '(redacted
path)/galaxy_dist_dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py',
line 364 in respond
app_iter = self.application(environ, detect_start_response)
File '(redacted
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py',
line 84 in __call__
return self.application(environ, start_response)
File '(redacted
path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
line 633 in __call__
return self.application(environ, start_response)
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py',
line 132 in __call__
return self.handle_request( environ, start_response )
File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py',
line 190 in handle_request
body = method( trans, **kwargs )
File '(redacted
path)/galaxy_dist_dev/lib/galaxy/web/framework/decorators.py', line 87 in
decorator
return func( self, trans, *args, **kwargs )
File '(redacted
path)/galaxy_dist_dev/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py',
line 1095 in prepare_for_install
install_repository_manager.handle_tool_shed_repositories(
installation_dict, using_api=False )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/install_manager.py',
line 636 in handle_tool_shed_repositories
new_tool_panel_section_label=new_tool_panel_section_label )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py',
line 200 in create_repository_dependency_objects
self.app.installed_repository_manager.activate_repository(
repository_db_record )
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/installed_repository_manager.py',
line 90 in activate_repository
repository_tools_tups = irmm.get_repository_tools_tups()
File '(redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py',
line 69 in get_repository_tools_tups
index, shed_conf_dict = self.tpm.get_shed_tool_conf_dict(
self.metadata_dict.get( 'shed_config_filename' ) )
TypeError: 'NoneType' object is not iterable
Same goes for resetting all metadata on installed repositories after
update, 23 out of ~200 tools work, for the rest I get this error in
paster.log :
tool_shed.galaxy_install.metadata.installed_repository_metadata_manager
ERROR 2015-01-20 15:18:35,357 Error attempting to reset metadata on
repository tabular_to_fasta
Traceback (most recent call last):
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py,
line 114, in reset_metadata_on_selected_repositories
self.set_repository( repository )
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py,
line 142, in set_repository
super( InstalledRepositoryMetadataManager, self ).set_repository(
repository )
File (redacted
path)/galaxy_dist_dev/lib/tool_shed/metadata/metadata_generator.py, line
1066, in set_repository
self.metadata_dict = { 'shed_config_filename' :
self.shed_config_dict.get( 'config_filename', None ) }
AttributeError: 'NoneType' object has no attribute 'get'
My galaxy contains this line for tool_conf files, and I moved all config
files from the distrib root to the config folder:
tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml
Did I do something wrong while updating to the latest releases???
Thanks for your help!
Yves Gagnon
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Partial automation for generating those twisty R dependency tool shed installation sequences
In case anyone needs to prepare a complete set of twisty R/BioC dependencies in the right order for automated toolshed installation, https://testtoolshed.g2.bx.psu.edu/view/fubar/biocdepgen_r_3_1_2 contains a dependency package generator for the iuc package_r_3_1_2 - eg for DESeq2, you end up with 46 dependent tarballs in addition to DESeq2 itself !!! Please, DO NOT install this on a public instance - like the tool factory 2 it's intended only for tool development on throw away instances. Not much documentation - set the tardir to where you want the tarballs saved, the XML and path prefix needed before the tarball name in tool_dependencies.xml to point to where the toolshed should download those packages from (I use a github repo eg) and the names of all the R/BioC packages showing in sessionInfo() as other attached packages after your R script has run successfully. I hope this helps minimise the pain for other tool developers. Let me know how you go, or if anyone wants specific versions for other r base packages please. On Tue, Jan 20, 2015 at 8:06 PM, Ross ross.laza...@gmail.com wrote: Hi Bjorn, Thanks for caring :) R/Bioc dependencies are determined by the target R package. This realisation made me change tool factory 2 in the test toolshed so it now allows R (or perl or python - not bash!) dependency package_r_ selection. It still allows the local system R but that's not recommended for anything reproducible. The generated tool will always use that R and so I can generate an RBioC dependency generator using that script https://wiki.galaxyproject.org/SetUpREnvironment for any available toolshed package_r_ Feedback always appreciated... On Sun, Jan 18, 2015 at 9:25 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ross, Am 13.01.2015 um 00:13 schrieb Ross: Hi Björn, I'm a bit old fashioned and think I prefer a proper Galaxy tool rather than a notebook :) :) So I've set up a temporary demonstration/test site of a toolfactory generated tool that does what I think I need - can some kind soul please test it and let me know how it goes ? If it's useful, it needs to be adjusted to depend on whatever version of package_R you want to work with - currently just uses the system R for demonstration purposes. I used the toolfactory2 (main toolshed) (which now allows any number of (optionally non editable) parameters!!!) to wrap the script shown at the bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are currently three parameters - the names of the R/BioC packages from sessionInfo(), the local directory where all the tarballs should be stowed and the XML output prefix to prepend to each row of the generated XML stanza for tool_dependencies.xml The resulting toolshed tarball was uploaded to a local toolshed and then installed to produce a new tool in the tool generators section - r_bioc_depgen Generate dependencies for R/BioC packages If you import the history at http://130.56.252.21/history/list_published you will see the toolfactory job (#1,#2,#3) - rerunning will show how the parameters are defined - fugly but it does work. After generating/uploading/installing the new tool, outputs from a test run are in #4 and #5 for DESeq This is cool! I have to try this on a few packages! This could also be of interest for our GSOC idea: https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations Thanks Ross for working on this! Bjoern Comments and suggestions welcomed! On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ross, you are absolutely right. My download_store repository is exactly for this purpose. https://github.com/bgruening/download_store If you are interested we could integrate your additional magic into the notebook. Thanks, Bjoern Am 11.01.2015 um 01:33 schrieb Ross: Hi, Björn, Looks pretty similar! Aren't the links your notebook generates transient? I think if you put them into a tool_dependencies.xml, they will fail permanently immediately after any of the package authors updates one of the relevant svn repositories? AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a link that works today like http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp next gets updated and Rcpp_0.11.3.tar.gz is migrated to http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement (eg) http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the contrib directory? That's why my more complex script downloads all the latest archives into my local github archive repo and generates a permanent link to suit that github repo. We definitely need an automated solution as this is a really infuriating aspect of trying to make code relying on R/BioC packages reproducible. On Thu, Jan 8, 2015 at 11:28
Re: [galaxy-dev] Tool development - Selecting a single item from input dataset.
I think this is the (relatively new) Galaxy ability to automatically run N copies of your tool given N input files, making N outputs and is related to the collections work. (This is possible if your tool takes a single input file) Peter On Tue, Jan 20, 2015 at 6:17 PM, Vimalkumar Velayudhan vi...@biotechcoder.com wrote: Hi all, I am trying to create a select box with the possibility of selecting only a single item from the input dataset (figure 1). This works fine but the option for selecting multiple files is still visible (figure 2). The multiple=false attribute has no effect. Figure: http://i.imgur.com/oJVFCoF.png I have the following in my XML. param format=tabular name=ribo_files type=data label=Select Ribo-Seq alignment file multiple=false /param Any suggestions? galaxy-dist revision 5f4c13d622b8 Regards, Vimalkumar Velayudhan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool param type library_data (Dooley, Damion)
I was successful getting library dataset selection and form submission to
happen within tool form tool form input param type=library_data ... - Yay!
But it appears one last fix (besides the url:
/visualization/list_libraries,' bug) is needed to enable history tool item to
be RERUN.
So if a galaxy dev can tackle this, great! Otherwise I'll have to drop this
approach :(
*
For others who may need...
tool form's param:
param name=library_datasets type=library_data label=Reference Bin
file help=Select one reference bin at a time from the popup window's
Reference Bin library./
command tag contents
#if $library_datasets
-B
#for $i, $dataset in enumerate($library_datasets)
#$dataset.get_file_name()
$dataset.id
#end for
#end if
Successfully submits either the full file path of selected dataset, or its id.
*
However trying to rerun this history tool item gets us an error (not related to
the above command contents):
Error Traceback:
View as: Interactive | Text | XML (full)
⇝ AttributeError: 'int' object has no attribute 'id'
Module galaxy.tools.parameters.basic:146 in value_to_basic view
return self.to_string( value, app )
Module galaxy.tools.parameters.basic:2406 in to_string
app galaxy.app.UniverseApplication object at 0x59a0710
ldda331
selfgalaxy.tools.parameters.basic.LibraryDatasetToolParameter
object at 0x7fced01d2090
value [331]
return [ldda.id for ldda in value]
AttributeError: 'int' object has no attribute 'id'
which is in /lib/galaxy/tools/parameters/basic.py:
class LibraryDatasetToolParameter( ToolParameter ):
Parameter that lets users select a LDDA from a modal window, then
use it within the wrapper.
...
def to_string( self, value, app ):
if not value:
return value
return [ldda.id for ldda in value]
Value is getting an array of dataset ids, whereas this method is expecting
value to be a list of dataset objects. I tried changing ldda.id to just id
but found other parts of LibraryDatasetTool code were broken as they too were
expecting an object. Maybe this is related to the tool job's history detail
link showing Reference Bin file as a string label of an object?:
Input Parameter Value Note for rerun
BLAST results as XML240: megablast Pasted Entry vs
16SMicrobial.fasta
Reference Bin file [galaxy.model.LibraryDatasetDatasetAssociation
object at 0x7fcee04d4350]
Tabular Report Column LabelsShort name
HTML Report templatetemplates.html_report
The other bug:
2) Tool_form.mako has a hardcoded url that doesn't work for galaxy installs
that have a prefix so that needs adjustment:
$(.add-librarydataset).click(function() {
var link = $(this);
$.ajax({
url: /visualization/list_libraries,
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
