There's no hard limits that I'm aware of, just factors that affect the
reliability at the extremes.
For instance, I can upload and download 9GB+ files to a machine in our local
development subnet without problem. There's no other processes or traffic
affecting the machines, the network connection is perfect, and I used a
modern browser (chrome). I'm sure I could upload much larger files without
problem (other than user patience).
If you've got someone uploading a large file in IE6, from a 10 year old XP
machine over dial-up or a slow DSL, to a server that is full of traffic then
you're destined to run into a hiccup. One potential problem that could be
looked into would be that a database connection remains open for the entire
duration of the upload/download. Also, for uploading, if it would take too
long, keep in mind that sometimes users are logged out after too long of
inactivity.
The uploaded files themselves go into the assetstore, which would only be
limited by your filesystem:
ext3/ext4: 16GB - 2TB per file.
-------------
For batch importing, there's a number of resources available. There has also
been past discussion on the list for how people prefer to handle it.
I use:
https://wiki.duraspace.org/display/DSPACE/Simple+Archive+Format+Packager
Which is a Java command line tool to package up your content (files, and a
spreadsheet full of metadata) into a format that can be imported into
DSpace.
Once you've got a package ready, you use the DSpace [dspace]/bin/dspace
import command to import the package.
https://wiki.duraspace.org/display/DSDOC/System+Administration#SystemAdministration-ImportingItems
Peter Dietz
On Tue, May 24, 2011 at 10:16 AM, George S Kozak <[email protected]> wrote:
> Dear Shivraj Thorat:
>
>
>
> The batch import process is defined in the DSpace Manual, but here is what
> I do:
>
>
>
> 1. For every item that you wish to upload, you need to create a
> directory on your server. In my case, I use /dspace/import/ as my default
> directory. If I have 25 items to upload, I create directories
> /dspace/import/001 - /dspace/import/025
>
> 2. In each of these directories I place the file(s) that I want to
> upload for the item.
>
> 3. I create a file called “contents” that defines the files that
> makeup the item.
>
> 4. I then create a Dublin core file that defines the metadata for
> the file (see attachment for a sample that I use locally)
>
> 5. Then as the dspace user I execute: /dspace/bin/dspace import
> --add --eperson=(your DSpace userid) --collection=(collection handle)
> --source=/dspace/import --mapfile=mapfile
>
>
>
> Does anyone else have anything useful to add to importing items?
>
>
>
> *From:* Shivraj Thorat [mailto:[email protected]]
> *Sent:* Tuesday, May 24, 2011 12:50 AM
> *To:* George S Kozak
> *Subject:* Re: [Dspace-tech] How large files can one ingest using the web
> user interface?
>
>
>
> It is good but let me know detail procedure for batch ingest. it is looking
> helpful for me.
>
> my dspace server is on ubuntu and i dont have command on the same.
>
> i have imported metadata using batch importer with browser. it around 2000
> items.
>
> now i have to upload bitstreams for each item using browser or any other
> tool with windows envinment.
>
> right now i have a csv file exported from dspace having id, collection, and
> other concern metadata fields, on of the field contain filename for
> bitstream to be upload.
>
> need detail help in to same
>
> Thanks
>
> On Mon, May 23, 2011 at 7:07 PM, George S Kozak <[email protected]> wrote:
>
> Magnus:
>
>
>
> I usually tell my users that the larger the files the less chance there is
> of successfully uploading them in the web user interface. The problem isn’t
> DSpace but involves the browser timing out.
>
> I have had users upload 1-2GB files sucessfully, but our local guidelines
> state that we prefer files that are 50MB or less. If a file is greater
> than 500MB, I ask people to give me the files to load using the batch
> loader.
>
>
>
> As for downloading, people downloading (depending on the speed of their
> internet providers) could take a long time getting the item downloaded, and
> their browser could timeout in the process.
>
>
>
> I know that @MIRE had some streaming options for certain files that would
> probably help with this.
>
>
>
> George Kozak
>
> Digital Library Specialist
>
> Cornell University Library Information Technologies (CUL-IT)
>
> 501 Olin Library
>
> Cornell University
>
> Ithaca, NY 14853
>
> 607-255-8924
>
>
>
> *From:* Magnus Norberg [mailto:[email protected]]
> *Sent:* Monday, May 23, 2011 5:44 AM
> *To:* DSpace techlist techlist
> *Subject:* [Dspace-tech] How large files can one ingest using the web user
> interface?
>
>
>
> Hello everyone,
>
> I got two questions:
>
> 1) How large files (GB) can one ingest into DSpace using the web user
> interface?
>
> 2) How large files (GB) can one download from DSpace using the web user
> interface?
>
> Thanks!
>
> /Magnus
>
>
>
>
> ------------------------------------------------------------------------------
> What Every C/C++ and Fortran developer Should Know!
> Read this article and learn how Intel has extended the reach of its
> next-generation tools to help Windows* and Linux* C/C++ and Fortran
> developers boost performance applications - including clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> DSpace-tech mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
>
>
> --
> Regards
>
>
> Mr. Shivraj Thorat
> Librarian
> BVDU, Institute of Management & Entrepreneurship Development
> Paud Rd, Kothrud,
> Pune-38
>
>
> ------------------------------------------------------------------------------
> vRanger cuts backup time in half-while increasing security.
> With the market-leading solution for virtual backup and recovery,
> you get blazing-fast, flexible, and affordable data protection.
> Download your free trial now.
> http://p.sf.net/sfu/quest-d2dcopy1
> _______________________________________________
> DSpace-tech mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
>
------------------------------------------------------------------------------
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery,
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now.
http://p.sf.net/sfu/quest-d2dcopy1
_______________________________________________
DSpace-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-tech
