Re: [Dspace-tech] small correction for dspace_migrate
Collection-based export is working as expected when using the -m
(migrate) flag in DSpace 1.5.2 with one exception.
/bin/export ItemExport --type=COLLECTION --id=collID --dest=dest_dir
--number=seq_num -m
This will correctly remove the handle file, as well as the following
metadata fields from dublin_core.xml:
dcvalue element=date qualifier=accessioned/dcvalue
dcvalue element=date qualifier=available/dcvalue
dcvalue element=date qualifier=issued/dcvalue*
dcvalue element=description qualifier=provenance/dcvalue
dcvalue element=format qualifier=extent/dcvalue
dcvalue element=format qualifier=mimetype/dcvalue
(*Date.issued is retained if the item has been published before.)
It also removes dc.identifier.uri in all cases. The dspace_migrate
script uses a regular expression to only remove identifier.uri fields
that start with http://hdl (retaining non-handle based identifier.uri
elements), while ItemExport.java is eliminating all identifier.uri fields.
Line 474:
if ((!migrate) ||
(migrate !(
(dcv.element.equals(date)
qualifier.equals(issued)) ||
(dcv.element.equals(date)
qualifier.equals(accessioned)) ||
(dcv.element.equals(date)
qualifier.equals(available)) ||
(dcv.element.equals(identifier)
qualifier.equals(uri)) ||
(dcv.element.equals(description)
qualifier.equals(provenance)) ||
(dcv.element.equals(format)
qualifier.equals(extent)) ||
(dcv.element.equals(format)
qualifier.equals(mimetype)
{
out.write(utf8, 0, utf8.length);
-John
Stuart Lewis wrote:
Hi John,
I was transferring a few collections from a DSpace 1.5.2 development
server to a production instance this morning and encountered a small
issue with the dspace_migrate bash script.
Since the item exporter in 1.6 has a migrate option (-m flag) this
script might get removed, so I'm posting this simple fix just in case
anyone else encounters the same problem.
Thanks for reporting this problem. The -m 'migrate' flag was
introduced in 1.5.2, so you could try running that instead. It would
be good to know if it works for you to give us more confidence in
removing the dspace_migrate script. We also have an open issue to make
the -m 'migrate' flag more configurable
(http://jira.dspace.org/jira/browse/DS-287
)
I've noted the bug in the JIRA issue to remove the dspace_migrate
script, so if we decide not to remove it, then we can make sure it
gets fixed before we release 1.6.
Thanks,
Stuart Lewis
IT Innovations Analyst and Developer
Te Tumu Herenga The University of Auckland Library
Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand
Ph: 64 9 373-7599 x81928
http://www.library.auckland.ac.nz/
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] small correction for dspace_migrate
I was transferring a few collections from a DSpace 1.5.2 development server to a production instance this morning and encountered a small issue with the dspace_migrate bash script. Since the item exporter in 1.6 has a migrate option (-m flag) this script might get removed, so I'm posting this simple fix just in case anyone else encounters the same problem. Transferring Items Between DSpace Instances: http://www.dspace.org/1_5_2Documentation/ch09.html#N1382D DSpace System Documentation lists the auto-generated fields the dspace_migrate script removes from dublin_core.xml prior to moving a collection (date.accessioned, date.available, description.provenance, etc.). The script also removes the handle file. When I attempted to import the collection into the production instance, I ended up with two handles-- one newly created and one left over from the dev server because it was still listed in the dublin_core.xml file as a dc.identifier.uri value. Here's the fix: Lines 101 and 104 perform the deletion of several element/qualifier pairs from the dublin_core.xml metadata, but a small typo occurs twice on the lines that should remove the handle. Removing one character from both lines fixes the problem. This: | $SED /element=\identifier\ qualifier=\uri\\http:\/\/hdl/d Should be this: | $SED /element=\identifier\ qualifier=\uri\http:\/\/hdl/d Note: removing the extra escape character (backslash \) before the greater than () symbol causes the script to correctly remove any dc.identifier.uri that starts with http://hdl without removing any non-handle URIs nor removing any local accession numbers stored in dc.indentifer.other. I hope this helps, John Davison OhioLINK DRC -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Bulk-import-data
Baseer-- If you're interested in the DSpace batch submission command line function detailed here, Item Importer and Exporter: http://www.dspace.org/1_5_2Documentation/ch09.html#N13795 Then let me suggest an Excel tool that makes the process of creating submission packages very easy. Detailed directions available on our wiki, with a download link at the bottom of the page: http://sites.google.com/a/ohiolink.edu/drmc/bulk-submission If however you're interested in non-Dublin Core descriptive metadata, you'll want to read the DSpace documentation, Metadata Schema Registry: http://www.dspace.org/1_5_2Documentation/ch05.html#N11B71 And the good folks at the Texas Digital Library have an excellent set of instructions here: What if I need to import metadata in a schema other than Dublin Core? http://www.tdl.org/wp-content/uploads/2009/04/DSpaceBatchImportFormat.pdf I hope this helps, John Davison OhioLINK DRC Khan, Baseer wrote: Hello , I need to bulk import data into Dspace , Is there any possible ways of doing it other than manually doing it one by one . I read about the data import in Dspace documentation but it does for a individual entry in collection and the input format is in Dublin code . My question is , is there a way that we can bulk import data in any format in Dspace with any script ? Baseer. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] multi-million record stress test
Hello All, I'm writing to share the initial results of a DSpace stress test we are performing, and to ask for your thoughts and suggestions as we begin ramping up for the next round of tests. http://sites.google.com/a/ohiolink.edu/drmc/Home/stretch-armstrong Background: At OhioLINK we've built a federation of DSpace instances across the state called the Digital Resource Commons (http://drc.ohiolink.edu), and we hope to expand our offerings beyond the academic library community. As part of that research, we are building a multi-million record test of DSpace using Amazon's Elastic Compute Cloud (EC2). Item Import: Like many of you, we noticed longer batch submission times as our repository grew beyond 200,000 items and knew we needed to find a solution. Our first goal was to confirm the results of the ROAD Project (http://www.jisc.ac.uk/whatwedo/programmes/reppres/tools/road.aspx) test mentioned by Stewart Lewis. In Stewart's scenario, the entire 300,000 record submission took place at one time. We wanted to see if the problem exists even when the submission is broken up into several smaller blocks and takes place over a period of days. Our initial data confirm and extend the results Stewart posted, 'DSpace at a Third of a Million Items.' http://blog.stuartlewis.com/wp-content/uploads/2009/01/dspace-banding.png While he was interested in the performance of the SWORD client, his experiment shows a steady increase in the time-to-ingest for a single batch submission of 300,000 records. We've confirmed the problem rests with the submission process itself, and is not just an issue with extremely large 'one shot' batches. Special thanks to Stewart (Auckland University), Tom De Mulder and Simon Brown (Cambridge University) for their early comments and assistance. Please feel free to send along your own insights and suggestions, John Davison Assistant Director Digital Resource Development -- Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Zooming software for Dspace
Alas, our JPEG2000 experiments were much less successful than this blog post implies. Our primary goal was to create Pan Zoom functionality that mimics the ease of use in Google Maps and other 'click-drag' tools. This shim simply was unable to scale to production level functionality, and we shifted our focus to the enterprise version of Zoomify and it's single file format. However, the work being done by aDORe with the Kakadu library and Djatoka image server may hold promise for a JPEG2K-specific solution. At some point in the future we might take another look at JPEG2000 (I jokingly refer to JPEG2010). For now, the Zoomify tool is fulfilling our needs. Djatoka demos: http://african.lanl.gov/aDORe/projects/djatoka/demo.html The Kakadu-to-Zoomify code exists here if anyone is interested in looking at it: https://dev.ohiolink.edu/svn/j2kTileRenderer/trunk/ I hope this helps, John Davison OhioLINK Stuart Lewis wrote: Looks like it has already been done (for DSpace) by the OhioLINK developers using on-the-fly tile creation: http://dltj.org/article/introducing-j2ktilerenderer/ Thanks, Stuart Lewis Digital Services Programmer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 _http://www.library.auckland.ac.nz http://www.library.auckland.ac.nz//_ *From:* Williams, Steven D [mailto:s.willi...@austin.utexas.edu] *Sent:* Saturday, 2 May 2009 7:59 a.m. *To:* dspace-tech@lists.sourceforge.net *Subject:* Re: [Dspace-tech] Zooming software for Dspace I have also been looking for a flash/JavaScript image loader that allows zoom. We use Zoomify on our other sites, but it needs to create a bunch of tiled images in order to work. I am not sure that would be the best fit for dspace. Steve Williams Webmaster University of Texas Libraries University of Texas at Austin PCL 1.128G 512-495-4384 *From:* Stuart Lewis [mailto:s.le...@auckland.ac.nz] *Sent:* Friday, May 01, 2009 2:42 PM *To:* Khan, Baseer; dspace-tech@lists.sourceforge.net *Subject:* Re: [Dspace-tech] Zooming software for Dspace Hi, I've not examined it in full detail to see how good a fit it would be, but Zoomify may suit what you are looking for. http://www.zoomify.com/compare.htm It comes with free and paid-for options depending upon your budget and requirements. Thanks, Stuart Lewis Digital Services Programmer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 _http://www.library.auckland.ac.nz https://libex1.lbr.auckland.ac.nz/exchweb/bin/redir.asp?URL=http://www.library.auckland.ac.nz/__/_ *From:* Khan, Baseer [mailto:baseer.k...@wichita.edu] *Sent:* Sat 02/05/2009 04:16 *To:* dspace-tech@lists.sourceforge.net *Subject:* [Dspace-tech] Zooming software for Dspace Hello , Is there any software which I can embed in Dspace for image zooming . I found @mire solutions provide this , but is there any free ware available and anyone is using it in their repository . Thank you , Baseer. -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.0.238 / Virus Database: 270.12.16/2094 - Release Date: 05/03/09 16:51:00 -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Streaming video assets from DSpace
Hi Jonathan, For what it's worth we've had pretty good luck at the Ohio Digital Resource Commons with Flash Video (.FLV) saved in progressive download format. Right now we are in the process of converting about 2,000 RealMedia (.RM) files, each more than an hour long. We embedded a simple Flash player (.SWF) in the DSpace XMLUI, and pass it the location of the video file so it plays automatically on the 'View Item' page. The video is also available for download in the 'Files in this Item' section of the page. All of our currently available video is behind Films for the Humanities Sciences copyright, but if you're interested send me a PM and I can let you take a closer look. John Davison Assistant Director Digital Resource Development OhioLINK -Original Message- From: Dorothea Salo [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 3:06 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Streaming video assets from DSpace On Mon, Nov 10, 2008 at 12:43 PM, Jonathan Roby [EMAIL PROTECTED] wrote: At the University of Manitoba, we're still running version 1.3.2 and I've been tasked to see if DSpace can play video files straight from the View/Open link, so no apparent downloading. The file formats we're interested in are mostly MPEG-2. In a word, no. You'll have to interpose a streaming server in there somewhere. Search the dspace-tech archives for details. Dorothea -- Dorothea Salo[EMAIL PROTECTED] Digital Repository Librarian AIM: mindsatuw University of Wisconsin Rm 218, Memorial Library (608) 262-5493 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] ItemImport
Jose, You are correct about both file descriptions and access restrictions for the batch ItemImporter. In the example below, you will find: 1)The bitstream being imported, 2)The bundle it is placed into, 3)Permissions restricted to Administrator only, and 4)A description of the file. AI010.jpg '\t' bundle:ORIGINAL '\t' permissions:-r Administrator '\t' description:Archival Master As Richard noted '\t' is a tab-delimiter. Make sure 'bundle' is lower case, and if you are adding an item to a preexisting bundle (e.g. THUMBNAIL) make sure it is in upper case. It's also worth noting the 'description' information is displayed under the Edit This Item--Item Bitstream tab, not in the DC metadata. Hope this helps, John Davison Ohio Digital Resource Commons -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Blanco, Jose Sent: Friday, August 08, 2008 11:14 AM To: Richard Rodgers Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] ItemImport Thanks for verifying this. Jose -Original Message- From: Richard Rodgers [mailto:[EMAIL PROTECTED] Sent: Friday, August 08, 2008 11:23 AM To: Blanco, Jose Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] ItemImport Hi Jose: Looks like the doc is a little behind the code - you might have noticed the thread where we are trying to rationalize the documentation process. For now, the ItemImporter code is your best bet. But yes, the Bitstream description can be added as you suggest, but note that the '\t' really refers to a tab separation in the import file, not the literal token '\t'. Hope this helps, Richard On Fri, 2008-08-08 at 10:54 -0400, Blanco, Jose wrote: I remember seeing that in 1.5 when using the item importer you can pass in a file description, and perhaps even permission info, but I can't find the documentation. From looking at the code, it seems that to put in a file description, the following must be added to the line listing the file: \tdescription: Your description. Is this right? Is there documentation on this? Thanks! Jose -- --- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] ItemImport
Dorothea- Props to devs indeed. I think Mark Diggory and Christine Moulen pointed this out to me late last year. We've been using it ever since. Regarding line parsing, yes you can use this to simply add descriptions. Your 'contents' manifest would look like this: AI010.jpg '\t' description:Archival Master -John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dorothea Salo Sent: Friday, August 08, 2008 1:11 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] ItemImport First, this is awesome, props to the devs! I've been wanting bitstream descriptions in batch imports for a long time. Second, I have a possibly-stupid question... On Fri, Aug 8, 2008 at 11:54 AM, John Davison [EMAIL PROTECTED] wrote: Jose, You are correct about both file descriptions and access restrictions for the batch ItemImporter. In the example below, you will find: 1)The bitstream being imported, 2)The bundle it is placed into, 3)Permissions restricted to Administrator only, and 4)A description of the file. AI010.jpg '\t' bundle:ORIGINAL '\t' permissions:-r Administrator '\t' description:Archival Master Is this line parsed such that bits of it can be omitted? For example, if I'm fine with whatever permissions the collection would normally assign, but I have a description to add, can I leave the permissions: segment out? Alternately, is there a keyword for default permissions? Dorothea -- Dorothea Salo [EMAIL PROTECTED] Digital Repository Librarian AIM: mindsatuw University of Wisconsin Rm 218, Memorial Library (608) 262-5493 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
[Dspace-tech] stand-alone Handle Server
Hello All, I'm investigating the use of a stand-alone remote Handle Server instead of the one bundled with DSpace. Robert Tansley and the China Digital Museum Project released a patch on SourceForge (Enable Use Of Remote Handle Server: https://sourceforge.net/tracker/index.php?func=detailaid=1272731group_id=1 9984atid=319984) but the file was last updated on 9-28-2005. Its summary lists several outstanding issues, including batch handle generation for bulk import and suffix sequencing. I'm curious if this is the currently accepted solution, or if there are other, more recent options available. Thanks in advance, John Davison Ohio Library Information Network - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
