Hi Demian, My guess at it is that MarkLogic would try to maintain the binary file as request param within memory, which is limited to some size for good reason. With 10+ concurrent requests, all uploading 2 Gb, memory would build up very rapidly. I'm guessing the limits are related to the limits within xdmp:document-get, which does not allow reading files larger than 16Mb or 64Mb (depending on 32/64-bit install).
I think I wouldn't advice to use form upload to send large files across. Other protocols like (S)FTP are better suited for that. You could allow such files to be uploaded to file locations through a separate (S)FTP server on the same host. You can let MarkLogic monitor that directory using a Scheduled Task, and perhaps leveraging Information Studio flows. In these processes you can use different function to load such large files, though I am not sure they are able to handle 2+Gb files.. Kind regards, Geert > -----Oorspronkelijk bericht----- > Van: [email protected] [mailto:general- > [email protected]] Namens Demian Hess > Verzonden: dinsdag 7 februari 2012 19:45 > Aan: [email protected] > Onderwerp: [MarkLogic Dev General] Can app server handle file uploads >2GB? > > We would like to store large binary files (many EPS and high > resolution PDFs) in MarkLogic. It would be easiest for users to create > zips with all their files and then upload via an HTML form > (enctype=multipart/form-data), but the size of the zip archives would > be more than 2GB. Indeed, some individual files can get to 2GB by > themselves. > > I tried POSTing a 2GB file to an app server and got a "413 Request > Entity Too Large" error. I don't see any settings in MarkLogic to > control max size of uploads for HTTP app servers. > > How efficient would ML be at handling large posts of 100MB and larger? > Would it buffer everything in memory or would it stream to a temporary > file on disk? Would ML 5.0 be more efficient than 4.0 at handling > large file uploads? > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
