On Nov 16, 2012, at 9:14 AM, greg wrote: > I'm confused if files are uploaded through the Galaxy server or through > Apache? > > I already set up the Apache proxy explained here: > http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy > (Does that affect uploads?) > > My goal is to let users upload as large of files as possible through HTTP.
The file is uploaded "through" Apache, but Apache doesn't do anything special with it other than stream the bits through to Galaxy. With nginx it's possible to have the proxy server actually handle the upload itself. > So far I just tested and upload of a 140MB file and I got this error > after a couple of minutes: > > 1: dsim-all-chromosome-r1.3.fasta.msg.updated.fasta > 0 bytes > An error occurred running this job:Traceback (most recent call last): > File "/misc/local/galaxy/galaxy-dist/tools/data_source/upload.py", > line 384, in <module> > __main__() > File "/misc/local/galaxy/galaxy-dist/tools/data_source/upload.py", > line 359, in __main__ > registry.load_datat > > > I see this information in the bug report: > > Traceback (most recent call last): > File "/misc/local/galaxy/galaxy-dist/tools/data_source/upload.py", > line 384, in <module> > __main__() > File "/misc/local/galaxy/galaxy-dist/tools/data_source/upload.py", > line 359, in __main__ > registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] ) > File "/misc/local/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", > line 65, in load_datatypes > tree = galaxy.util.parse_xml( config ) > File "/misc/local/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", > line 143, in parse_xml > tree = ElementTree.parse(fname) > File "build/bdist.linux-x86_64/egg/elementtree/ElementTree.py", line > 859, in parse > File "build/bdist.linux-x86_64/egg/elementtree/ElementTree.py", line > 576, in parse > IOError: [Errno 2] No such file or directory: '/scratch/galaxy/tmptzuSYJ' > Traceback (most recent call last): > File "./scripts/set_metadata.py", line 118, in <module> > __main__() > File "./scripts/set_metadata.py", line 68, in __main__ > datatypes_registry.load_datatypes( root_dir=config_root, > config=datatypes_config ) > File "/misc/local/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", > line 65, in load_datatypes > tree = galaxy.util.parse_xml( config ) > File "/misc/local/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", > line 143, in parse_xml > tree = ElementTree.parse(fname) > File "build/bdist.linux-x86_64/egg/elementtree/ElementTree.py", line > 859, in parse > File "build/bdist.linux-x86_64/egg/elementtree/ElementTree.py", line > 576, in parse > IOError: [Errno 2] No such file or directory: '/scratch/galaxy/tmptzuSYJ' > > > Also here is what /scratch/galaxy looks like. It seems like the file > it mentions does exist?? > >> ls -lh /scratch/galaxy/ > total 134M > -rw-r--r-- 1 galaxy scicomp 20K Nov 13 13:35 tmpfCaiGX > -rw-r--r-- 1 galaxy scicomp 20K Nov 14 13:36 tmpM2Wclq > -rw-r--r-- 1 galaxy scicomp 20K Nov 14 13:35 tmpsIyPur > -rw-r--r-- 1 galaxy scicomp 20K Nov 16 08:48 tmptY5RaM > -rw-r--r-- 1 galaxy scicomp 20K Nov 16 08:54 tmptzuSYJ > -rw-r--r-- 1 galaxy scicomp 20K Nov 15 15:47 tmpu77PEK > -rw------- 1 galaxy scicomp 286 Nov 16 08:58 tmpykoBIq > -rw------- 1 galaxy scicomp 134M Nov 16 08:58 upload_file_data_G3eZEg Are you running the upload tool on a cluster? --nate > > Thanks, > > Greg > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
