Hi Charlene, My immediate thought is whether using SWORD (v1 or v2) would help with this automated ingest. Creating SWORD packages may take a bit more work, but SWORD is built for this sort of ingest process (i.e. ingesting automatically from another system or program)
http://swordapp.org/ DSpace supports SWORD v1 and v2. Your suggested process also sounds reasonable, but I did want to make sure you've considered SWORD. As for the DSpace import command -- it doesn't currently support versioning. There is an "itemupdate" command that lets you alter metadata/files, but it's still also not hooked into versioning. The Item Versioning feature in DSpace is currently only UI enabled. However, SWORDv2 does support some basic form of "file versioning" (keeps a copy of old files, if configured) and also allows you to update existing metadata, etc. More in the config documentation for SWORDv2 at: https://wiki.duraspace.org/display/DSDOC4x/SWORDv2+Server Hope that helps! - Tim On 7/3/2014 1:51 PM, Charlene Chinda Barina wrote: > Hi, > > I have been tasked with figuring out a way to automatically add items to > our repository from a variety of web-based resources, and I wanted to > see if you all, with more experience in this, would possibly have a > better/simpler approach. > > Many of our data sources come from third-party applications, via APIs or > other downloads. For example, one site has a csv report that is updated > daily. We want to download that and add it to our repository, which will > serve as the reference point for "official" data dumps or reports. We > want it to be completely automated, as limited staffing is often a > bottleneck in these kinds of routine data management processes. > > My idea was, via a script with cron, to: > > 1. Use wget or cURL (or an API) to download the files to the server > where DSpace is being hosted > 2. Generate a SAF CSV with the expected metadata (most fields remain > the same, other than perhaps filename/date). > 3. Run SAFBuilder to create the SAF packages > 4. Run a Dspace import command to add these items to the repository > > Has anyone done something like this before, and is there a better way to > do this? Also, I wasn't sure if import would allow for new item version > creation; I don't see it in the command-line reference, but does anyone > know? > > Many thanks, > Charlene > > -- > Charlene Barina, MPH > Research Coordinator, US IMPACT Study > The Information School > 303-359-6347 | Skype: cbarina > facebook.com/ImpactSurvey <http://facebook.com/ImpactSurvey> | > twitter.com/impactsurvey <http://twitter.com/impactsurvey> > > > ------------------------------------------------------------------------------ > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > > > > _______________________________________________ > DSpace-tech mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dspace-tech > List Etiquette: > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > ------------------------------------------------------------------------------ Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft _______________________________________________ DSpace-tech mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
