When installing Galaxy Tool Shed repositories into a local Galaxy
instance, tool shed dependecies are exchanged as an encoded python
dictionary via the HTTP GET method.

If a tool shed repository contains enough dependencies, the length
of this encoded dictionary can become extremely long.  In my case,
it became long enough for my Apache server to choke on it:

[Wed Sep 11 15:34:53 2013] [error] [client XXX.XXX.XXX.XXX] request failed: URI too long (longer than 8190)

The HTTP standard does not place a limit on the length of a URI,
but the de facto limit is 2000 characters.  See the following
stackoverflow answer:


I would recommend either:
  1) communicating toolshed dependencies via an HTTP POST
  2) redesigning tool shed <-> instance communications

As a stopgap measure, I've modified lib/tool_shed/util/encoding_util.py
to use a more compact encoding scheme.  I'm attaching a copy in
case someone else runs into the same problem.

import base64
import zlib
from galaxy import eggs
from galaxy.util.hash_util import hmac_new
from galaxy.util.json import json_fix

import pkg_resources

pkg_resources.require( "simplejson" )
import simplejson

encoding_sep = '__esep__'
encoding_sep2 = '__esepii__'

def tool_shed_decode( val ):
    mac, encoded_val = val.split( ':', 1 )
    assert mac == hmac_new( 'ToolShedAndGalaxyMustHaveThisSameKey', encoded_val )
    compressed_val = base64.urlsafe_b64decode( encoded_val.encode( 'utf8' ) )
    json_val = zlib.decompress( compressed_val )
    value = simplejson.loads( json_val )
    return json_fix( value )

def tool_shed_encode( val ):
    json_val = simplejson.dumps( val, separators=( ',', ':' ) )
    compressed_val = zlib.compress( json_val, 9 )
    encoded_val = base64.urlsafe_b64encode( compressed_val )
    mac = hmac_new( 'ToolShedAndGalaxyMustHaveThisSameKey', encoded_val )
    return "%s:%s" % ( mac, encoded_val )
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to