Just to follow up: we've implemented a workaround for uploading large files into libraries using upload_file_from_url(): https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#bioblend .galaxy.libraries.LibraryClient.upload_file_from_url
Also, there is a corresponding function for uploading files into histories: https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#bioblend .galaxy.tools.ToolClient.put_url Ilya -----Original Message----- From: galaxy-dev <galaxy-dev-boun...@lists.galaxyproject.org> on behalf of Ilya Sytchev <isytc...@hsph.harvard.edu> Date: Wednesday, April 27, 2016 at 15:53 To: "galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyproject.org> Subject: [galaxy-dev] Problem uploading multigigabyte files via API >Hi, > >I'm running Galaxy 16.01 behind Nginx 1.4.6 on Ubuntu 14.04 and I came >across an issue where uploading files larger than ~3GB via the API into >histories and libraries fails. > >Steps to reproduce (using BioBlend 0.7.0): > >from bioblend.galaxy.objects import GalaxyInstance >gi = GalaxyInstance("galaxy-dev", "api_key") >h = gi.histories.create() >h.upload_dataset('/path/to/multigigabytefile') > >Observed results: > >ConnectionError: Unexpected response from galaxy: 504: <html> > ><head><title>504 Gateway Time-out</title></head> ><body bgcolor="white"> ><center><h1>504 Gateway Time-out</h1></center> ><hr><center>nginx/1.4.6 (Ubuntu)</center> ></body> ></html> > >It looks like Nginx is accepting the full incoming data transfer over the >network and saving it to disk successfully, and the error occurs during >the subsequent forwarding of the request to Galaxy. There doesn't seem to >be a particular threshold for file size (for example, 2GB and 2.7GB file >can be uploaded successfully). Also, large file uploads do work when >Galaxy is not running behind Nginx. > > >Relevant log entries and config details are attached. I'd appreciate any >help. > >Thanks, >Ilya > > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
