Thanks for the additional details - I am responding on list for people who may encounter similar problems in the future.
The job running code will provide both core tools and tool shed tools with the same PYTHONPATH - but some tools might setup a Python virtualenv for instance which would replace the PYTHONPATH completely I believe. Since gsummary is not in the tool shed - it will not be able to find any of the tool shed dependencies. These rpy issues are tough ones. It should be possible to hack something together if you change the requirements in gsummary to be package requirements for R and Rpy and then find working env.sh files in tool_dependency_dir and copy them to tool_dependency_dir/R/2.11/env.sh and tool_dependency_dir/Rpy/1.0.3/env.sh respectively. The tool shed is suppose to solve all of these pesky dependency problems - but not all the tools have been migrated and fixed yet. Thanks for your patience and for using Galaxy! -John On Wed, Oct 1, 2014 at 1:36 PM, David Bernick <dbern...@soe.ucsc.edu> wrote: > Hi John - > > Yes, we are using MySQL. This event happened as a result of attempting to > add/delete tools from our instance (Rpy_1_0_3 and R_2_11). > We ended up reloading the database from our nightly backup, and that got us > going again. > > The issue I was trying to solve when I "broke" the database was > Summary_statistics not being able to find its Rpy module. I now have R and > Rpy installed, such that tool-shed tools are happy with it, but gsummary.py > is still unable to find Rpy. We are attempting to solve this, but are as yet > without a fix. In the meantime, I have installed datamash, and that is > working, It seems that tool-shed tools have a different PYTHONPATH than > galaxy-dist tools? > > As it happens, we also had a few cases with multiple instances of tool-shed > tools being installed. Since this server is brand new, it was fine to delete > them. I suppose that maybe we are now fitting in MySQL BLOBs now?? > > thanks again - > David > > > > On Oct 1, 2014, at 9:44 AM, John Chilton <jmchil...@gmail.com> wrote: > >> Hello David, >> >> We don't have a workaround yet - but we (Dan did most of the >> investigating) understand the problem I think. You are using a MySQL >> database I assume? (Ignore the rest of this e-mail if not). I think >> MySQL has lower limits on the size of BLOBs that can be stored in the >> database than postgres - and we think the tool shed is tried to >> storing a really large blob of metadata in the database and it got >> truncated and now Galaxy is failing because it cannot load that entry >> as JSON. >> >> The work around would be something like trying to find the truncated >> column in the tool_repositories table - deleting it and references - >> changing that column to a LONGBLOB and reinstalling that tool. >> >> I have created a Trello card for this bug report. >> >> https://trello.com/c/RbW6pOd2 >> >> It is said a lot but we strongly encourage not using MySQL. >> >> It might be worth trying this patch >> (https://bitbucket.org/galaxy/galaxy-central/pull-request/511/allow-model-objects-to-be-loaded-when-they/diff#comment-3288189) >> from Dan as well. I am not sure what the tool shed would do if it >> cannot find the metadata though. >> >> -John >> >> On Mon, Sep 29, 2014 at 11:41 AM, David Bernick <dbern...@soe.ucsc.edu> >> wrote: >>> Good morning - >>> >>> I have acquired an Internal server error on our local galaxy install. It >>> seems to be an unterminated string. How do I resolve this? >>> thanks in advance - >>> David Bernick >>> UCSC >>> >>> Here is the relevant portion of the log (I hope). >>> >>> Error - <type 'exceptions.ValueError'>: Unterminated string starting at: >>> line 1 column 65528 (char 65528) >>> URL: http://campusrocks.cse.ucsc.edu:8080/admin/index >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', >>> line 149 in __call__ >>> app_iter = self.application(environ, sr_checker) >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', >>> line 84 in __call__ >>> return self.application(environ, start_response) >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', >>> line 633 in __call__ >>> return self.application(environ, start_response) >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', >>> line 132 in __call__ >>> return self.handle_request( environ, start_response ) >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', >>> line 190 in handle_request >>> body = method( trans, **kwargs ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/__init__.py', >>> line 383 in decorator >>> return func( self, trans, *args, **kwargs ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/base/controllers/admin.py', >>> line 34 in index >>> installing_repository_ids = >>> suc.get_ids_of_tool_shed_repositories_being_installed( trans.app, >>> as_string=True ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/tool_shed/util/shed_util_common.py', >>> line 277 in get_ids_of_tool_shed_repositories_being_installed >>> app.install_model.ToolShedRepository.status == loading_datatypes_status ) >>> ): >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', >>> line 2348 in instances >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', >>> line 2120 in _instance >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', >>> line 1974 in populate_state >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', >>> line 150 in fetch_col >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/types.py', >>> line 658 in process >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/model/custom_types.py', >>> line 52 in process_result_value >>> return json_decoder.decode( str( _sniffnfix_pg9_hex(value) ) ) >>> File '/usr/lib64/python2.6/json/decoder.py', line 319 in decode >>> obj, end = self.raw_decode(s, idx=_w(s, 0).end()) >>> File '/usr/lib64/python2.6/json/decoder.py', line 336 in raw_decode >>> obj, end = self._scanner.iterscan(s, **kw).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 155 in JSONString >>> return scanstring(match.string, match.end(), encoding, strict) >>> ValueError: Unterminated string starting at: line 1 column 65528 (char >>> 65528) >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
