That's great, thanks. On Tue, Feb 7, 2012 at 4:06 PM, David Erickson <[email protected]> wrote: > I'll echo Blakeley's comment about the loading binary documents chapter. > > Working with the server's local filesystem can be done with > xdmp:filesystem-directory, xdmp:filesystem-file, etc … > > --Dave > > On Feb 7, 2012, at 3:58 PM, Demian Hess wrote: > >> I think FTP is the best choice for this. >> >> In regard to monitoring a directory: is there an xdmp function for >> getting a listing of files and directories from the file system? I >> didn't see that in the API documents. >> >> Thanks, Demian >> >> On Tue, Feb 7, 2012 at 2:04 PM, Geert Josten <[email protected]> wrote: >>> Hi Demian, >>> >>> My guess at it is that MarkLogic would try to maintain the binary file as >>> request param within memory, which is limited to some size for good >>> reason. With 10+ concurrent requests, all uploading 2 Gb, memory would >>> build up very rapidly. I'm guessing the limits are related to the limits >>> within xdmp:document-get, which does not allow reading files larger than >>> 16Mb or 64Mb (depending on 32/64-bit install). >>> >>> I think I wouldn't advice to use form upload to send large files across. >>> Other protocols like (S)FTP are better suited for that. You could allow >>> such files to be uploaded to file locations through a separate (S)FTP >>> server on the same host. You can let MarkLogic monitor that directory >>> using a Scheduled Task, and perhaps leveraging Information Studio flows. >>> In these processes you can use different function to load such large >>> files, though I am not sure they are able to handle 2+Gb files.. >>> >>> Kind regards, >>> Geert >>> >>>> -----Oorspronkelijk bericht----- >>>> Van: [email protected] [mailto:general- >>>> [email protected]] Namens Demian Hess >>>> Verzonden: dinsdag 7 februari 2012 19:45 >>>> Aan: [email protected] >>>> Onderwerp: [MarkLogic Dev General] Can app server handle file uploads >>>> 2GB? >>>> >>>> We would like to store large binary files (many EPS and high >>>> resolution PDFs) in MarkLogic. It would be easiest for users to create >>>> zips with all their files and then upload via an HTML form >>>> (enctype=multipart/form-data), but the size of the zip archives would >>>> be more than 2GB. Indeed, some individual files can get to 2GB by >>>> themselves. >>>> >>>> I tried POSTing a 2GB file to an app server and got a "413 Request >>>> Entity Too Large" error. I don't see any settings in MarkLogic to >>>> control max size of uploads for HTTP app servers. >>>> >>>> How efficient would ML be at handling large posts of 100MB and larger? >>>> Would it buffer everything in memory or would it stream to a temporary >>>> file on disk? Would ML 5.0 be more efficient than 4.0 at handling >>>> large file uploads? >>>> _______________________________________________ >>>> General mailing list >>>> [email protected] >>>> http://developer.marklogic.com/mailman/listinfo/general >>> _______________________________________________ >>> General mailing list >>> [email protected] >>> http://developer.marklogic.com/mailman/listinfo/general >> >> >> >> -- >> Demian Hess >> >> Avalon Consulting, LLC >> 527 Maple Avenue East, Suite 200, Vienna, VA 22180 >> >> Mobile: 301-943-8307 >> Fax: 845-367-5496 >> [email protected] >> _______________________________________________ >> General mailing list >> [email protected] >> http://developer.marklogic.com/mailman/listinfo/general > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general
-- Demian Hess Avalon Consulting, LLC 527 Maple Avenue East, Suite 200, Vienna, VA 22180 Mobile: 301-943-8307 Fax: 845-367-5496 [email protected] _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
