evelopment also forms part of the DataFlow project
> http://www.dataflow.ox.ac.uk/
>
> Sally
>
> --
> Sally Rumsey
> Digital Collections Development Manager
> Bodleian Digital Library Systems and Services (BDLSS)
>
> sally.rum...@bodleian.ox.ac.uk
> Tel: 01865 2
r amount on this, one of his project manager’s
> Mark
> >> Macgillvray has been thinking on this re ‘Open Scholarship’ a fair
> amount –
> >> wish we could get a University to start playing around with this, of
> course
> >> the Tardis folk down in Oz have been doing go
gt; , Kathi Fletcher
> Subject: Re: [sword-app-tech] How to send large fiels
>
> FWIW we've got a couple of students under New Zealand's "Summer of eResearch"
> scheme looking at implementing a university data catalogue, with CKAN being
> one of the candidate systems.
Brilliant, thank you Pete for posting your replies! It is obvious that
you've already considered this area in depth and so I am sure that many
people will find your responses very helpful indeed. My questions were
intended to provoke thought on the list, as I know from my own experience
and failing
Hi Ben,
In response to your questions (and apologies if I'm teaching anyone on this
list to suck eggs):-
- What do you feel you might gain by placing 500Gb+ files into a repository,
compared with having them in an addressable filestore?
My understanding is that many funding bodies are now requi
er; < sword-app-tech@lists.sourceforge.net>
> oda...@gmail.com; Rufus
> Subject: Re: [sword-app-tech] How to send large fiels
>
> FWIW we've got a couple of students under New Zealand's "Summer of
> eResearch" scheme looking at implementing a university data catalog
com]
> Sent: 05 December 2011 14:50
> To: David FLANDERS
> Cc: < sword-app-tech@lists.sourceforge.net> Rufus; Leggett, Pete
> Subject: Re: [sword-app-tech] How to send large fiels
>
> Hi,
>
> I have CC'd Rufus Pollack of CKAN in case he has ideas about some so
t; From: Kathi Fletcher [mailto:kathi.fletc...@gmail.com]
>> Sent: 05 December 2011 14:50
>> To: David FLANDERS
>> Cc: < sword-app-tech@lists.sourceforge.net> Rufus; Leggett, Pete
>>
>>
>> Subject: Re: [sword-app-tech] How to send large fiels
>>
>>
f
>
>
>
> From: Kathi Fletcher [mailto:kathi.fletc...@gmail.com]
> Sent: 05 December 2011 14:50
> To: David FLANDERS
> Cc: < sword-app-tech@lists.sourceforge.net> Rufus; Leggett, Pete
>
>
> Subject: Re: [sword-app-tech] How to send large fiels
>
>
>
> Hi,
>
Steve
Androulakis. /dff
From: Kathi Fletcher [mailto:kathi.fletc...@gmail.com]
Sent: 05 December 2011 14:50
To: David FLANDERS
Cc: < sword-app-tech@lists.sourceforge.net> Rufus; Leggett, Pete
Subject: Re: [sword-app-tech] How to send large fiels
Hi,
I have CC'd Rufus Pollack of CKAN
t: Re: [sword-app-tech] How to send large fiels
To: Ben O'Steen , Stuart Lewis
Cc: "< sword-app-tech@lists.sourceforge.net>" <
sword-app-tech@lists.sourceforge.net>, "Leggett, Pete" <
p.f.legg...@exeter.ac.uk>
+1
** **
Why not use systems *built
RS
Cc: < sword-app-tech@lists.sourceforge.net> Leggett, Pete
Subject: Re: [sword-app-tech] How to send large fiels
I believe I know what you are getting at re: nosql stores, but a better choice
might be a communial HBase, R server or Hadoop deployment - nothing for hard
use, but something
s and then
> interoperate with repositories rather than build the same functionality
> into repositories? /dff
>
> ** **
>
> *From:* Ben O'Steen [mailto:bost...@gmail.com]
> *Sent:* 05 December 2011 08:00
> *To:* Stuart Lewis
> *Cc:* < sword-app-tech@lists.sour
functionality into repositories?
/dff
From: Ben O'Steen [mailto:bost...@gmail.com]
Sent: 05 December 2011 08:00
To: Stuart Lewis
Cc: < sword-app-tech@lists.sourceforge.net> Leggett, Pete
Subject: Re: [sword-app-tech] How to send large fiels
While I think I understand the drive to put these fil
While I think I understand the drive to put these files within a
repository, I would suggest caution. Just because it might be possible to
put a file into the care of a repository doesn't make it a practical or
useful thing to do.
- What do you feel you might gain by placing 500Gb+ files into a
re
he Dspace server.
>>> Could item be uploaded directly into asset store maybe ?
>>> The other problem is how anyone could download the item once it¹s in
>>> Dspace ?
>>>
>>> Anyone else doing this sort of very large item ( i.e. TB¹s ) injest ?
>>>
anyone could download the item once it¹s in
>>Dspace ?
>>
>> Anyone else doing this sort of very large item ( i.e. TB¹s ) injest ?
>>
>> Thank you,
>>
>> Pete
>>
>>
>> From: David FLANDERS [mailto:d.fland...@jisc.ac.uk]
>>
November 2011 18:10
> To: Ben O'Steen; Stuart Lewis
> Cc: <, sword-app-tech@lists.sourceforge.net>,
> Subject: Re: [sword-app-tech] How to send large fiels
>
> To second that, some amazing things being done down here in Australia as they
> take *large* data off of scientif
* **
>
> *From:* David FLANDERS [mailto:d.fland...@jisc.ac.uk]
> *Sent:* 21 November 2011 18:10
> *To:* Ben O'Steen; Stuart Lewis
>
> *Cc:* <, sword-app-tech@lists.sourceforge.net>,
> *Subject:* Re: [sword-app-tech] How to send large fiels
>
> ** **
>
&
Thank you,
Pete
From: David FLANDERS [mailto:d.fland...@jisc.ac.uk]
Sent: 21 November 2011 18:10
To: Ben O'Steen; Stuart Lewis
Cc: <, sword-app-tech@lists.sourceforge.net>,
Subject: Re: [sword-app-tech] How to send large fiels
To second that, some amazing things being done down
[sword-app-tech] How to send large fiels
+1 for deposit by reference. It is almost like giving a metadata receipt for a
deposit not happening via the http route.
I would also highly recommend looking at High-Performance SSH or HPN-SSH. On
comparable hardware, I have been shown that it outpaces ev
Hi all,
2011/11/14 Stuart Lewis
> The method that seems to work in this setting is to use 'deposit by
> reference'. That is, you deposit a description of the item, including
> details of where the item can be found. It is then up to ingesting system
> to pull the data - perhaps via an offline
+1 for deposit by reference. It is almost like giving a metadata receipt
for a deposit not happening via the http route.
I would also highly recommend looking at High-Performance SSH or HPN-SSH.
On comparable hardware, I have been shown that it outpaces even grid-ftp
for file transfer speeds, but
Hi Jesús,
The method that seems to work in this setting is to use 'deposit by reference'.
That is, you deposit a description of the item, including details of where the
item can be found. It is then up to ingesting system to pull the data -
perhaps via an offline queue process, using some oth
Hi everyone,
We are using SWORD to deposit DIPs in ICA-AtoM from Archivematica, but we
have encountered some problems with large files (8GB). HTTP requests can be
very resource intensive and unmanageable when the contents are very
big. Does anyone have any recommendations, for depositing large
fil
25 matches
Mail list logo
