Dear John,

thank you very much your help and the warnings!

I'm testing it with a second database and so far it seems to work :-).

I was also suspecting the encoding to be the issue, that's why I did it through 
pg_dump and specified the encoding of the new server for the dump. The error 
stayed the same and so far I have just seen it in the context of toolshed 
operations. I don't know what else I could do to ensure proper encoding.

Best regards,

----- Original Message -----
From: "John Chilton" <>
To: "Sarah Diehl" <>
Cc: " List" <>
Sent: Thursday, October 16, 2014 3:17:39 PM
Subject: Re: [galaxy-dev] Help with Galaxy server migration

Hmm.... hopefully someone more knowledgeable about the tool shed than
me responds also but I had a couple quick thoughts.

The first is a warning - workflows may break. At very least workflows
that depend on the previous instance having gone through tool
migrations instead of tool install. My understanding is tool
migrations cause links to the older versions of the tools to be
created so workflows continue to operate - just installing the same
version of the tool again doesn't set this up. Also - in theory if you
resinstall the same versions of tools tool re-runs and stuff should
still work - but I am not confident and I have not tested it myself -
so if that functionality is important to you, you test it out before
throwing out the old configuration.

Next - I am a little bit concerned about this error. It would seem to
point to some sort of encoding problem with the way the database was
transferred - different table encodings or something? I don't really
know anything about encoding and postgres though.

That said I am not surprised there are problems with the tool shed.
There are all sorts of implicit dependencies between what is on disk
and what is in the database. If you really do want to start fresh I
would clear out all of the tool shed tables before continuing. These
tables including -
repository_dependency, tool_dependency, tool_version,
tool_version_association, migrate_tools.

Rather than truncating the existing tables - you could also just have
Galaxy target a completely separate database for tool shed install
stuff by creating a new postgres database, setting
install_database_connection in your galaxy ini file (either
universe_wsgi.ini or galaxy.ini depending on what version of Bjoern's
docker image you are targetting) - or in the newest version of that
Dockerfile you can just make sure your docker run includes a -e

You will also want to make sure that database gets populated - I think
either of the following would work
./ install
./ install upgrade

Sorry I have not definitive answers - but hopefully this is helpful.
Good luck with the migration. You should let the list know how the
Docker container thing goes for a production server.


On Thu, Oct 16, 2014 at 8:36 AM, Sarah Diehl <> wrote:
> Dear all,
> I'm trying to migrate our local Galaxy server from a standalone server 
> running on CentOS 5 to a setup that runs Galaxy in a Docker container 
> (bgruening/galaxy-stable). Due to Docker and the setup of the container, 
> almost all data, tools, configs etc. have to move to a different place. Since 
> our old server is pretty cluttered up anyway, I would like to move just user 
> data (logins, histories, datasets ...), but none of the tool installations, 
> tool data or configs.
> My plan was to start with a fresh and clean new Galaxy installation, transfer 
> the datasets and the database and then reinstall all tools through the 
> toolshed (of course there are many more small odds and ends that need fixing).
> The problem is that after transfering the database, I always get an error 
> when going to "Manage installed tool shed repositories" (see end of the 
> mail). I tried just overwriting the pg_data directory (PostgreSQL versions 
> are identical) and also exporting to sql (pg_dump) and importing in the new 
> DB. Both times I made sure to run upgrade afterwards. The error 
> stays the same.
> I can't make much sense of the error message, my only guess is that it looks 
> for something that's not there. However, I really don't want to transfer the 
> tool directories and tool configurations, because that will get messy and 
> difficult, since many paths need to be adjusted. There's a high chance for 
> error and this way I also carry over the clutter.
> So is there a way to remove all tool installation information from the 
> database, before I transfer it? Can this even be disentangled from the 
> histories and datasets? Any help or other ideas on how to do this migration 
> are highly appreciated!
> Thanks,
> Sarah
> - - [16/Oct/2014:09:54:35 +0000] "GET 
> /admin_toolshed/browse_repositories HTTP/1.1" 500 - 
> "http://deep2:8080/admin/index"; "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; 
> rv:32.0) Gecko/20100101 Firefox/32.0"
> Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode 
> byte 0xe2 in position 393: ordinal not in range(128)
> URL: http://deep2:8080/admin_toolshed/browse_repositories
> File 'lib/galaxy/web/framework/middleware/', line 149 in __call__
>   app_iter = self.application(environ, sr_checker)
> File '/galaxy-central/eggs/Paste-', line 
> 84 in __call__
>   return self.application(environ, start_response)
> File '/galaxy-central/eggs/Paste-', 
> line 633 in __call__
>   return self.application(environ, start_response)
> File 'lib/galaxy/web/framework/', line 132 in __call__
>   return self.handle_request( environ, start_response )
> File 'lib/galaxy/web/framework/', line 190 in handle_request
>   body = method( trans, **kwargs )
> File 'lib/galaxy/web/framework/', line 87 in decorator
>   return func( self, trans, *args, **kwargs )
> File 'lib/galaxy/webapps/galaxy/controllers/', line 167 in 
> browse_repositories
>   return self.installed_repository_grid( trans, **kwd )
> File 'lib/galaxy/web/framework/helpers/', line 306 in __call__
>   kwargs=kwargs )
> File 'lib/galaxy/web/framework/', line 771 in fill_template
>   return self.fill_template_mako( filename, **kwargs )
> File 'lib/galaxy/web/framework/', line 786 in fill_template_mako
>   return template.render( **data )
> File '/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/', line 296 
> in render
>   return runtime._render(self, self.callable_, args, data)
> File '/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/', line 660 in 
> _render
>   **_kwargs_for_callable(callable_, data))
> File '/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/', line 692 in 
> _render_context
>   _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
> File '/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/', line 718 in 
> _exec_template
>   callable_(context, *args, **kwargs)
> File '/galaxy-central/database/compiled_templates/', line 66 in 
> render_body
>   __M_writer(unicode(next.body()))
> File '/galaxy-central/database/compiled_templates/', line 91 
> in render_body
>   __M_writer(unicode(self.load()))
> File '/galaxy-central/database/compiled_templates/', line 
> 120 in render_load
>   __M_writer(unicode( h.dumps( self.get_grid_config( embedded=embedded, 
> insert=insert ) ) ))
> File '/galaxy-central/database/compiled_templates/', line 
> 303 in render_get_grid_config
>   value = column.get_value( trans, grid, item )
> File 'lib/tool_shed/galaxy_install/grids/', line 106 
> in get_value
>   return suc.get_tool_shed_repository_status_label(, 
> tool_shed_repository )
> File 'lib/tool_shed/util/', line 829 in 
> get_tool_shed_repository_status_label
>   elif tool_shed_repository.tool_dependencies_being_installed:
> File 'lib/galaxy/model/tool_shed_install/', line 408 in 
> tool_dependencies_being_installed
>   for tool_dependency in self.tool_dependencies:
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 168 in __get__
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 453 in get
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 508 in _load_for_state
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 574 in _emit_lazyload
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 2115 in all
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/',
>  line 2341 in instances
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/',
>  line 3204 in fetchall
> File 
> '/galaxy-central/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/',
>  line 3171 in _fetchall_impl
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 393: 
> ordinal not in range(128)
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> To search Galaxy mailing lists use the unified search at:
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to