---------- Forwarded message ----------
From: Ondrej Kosarko <ko...@centrum.cz>
Date: 2015-05-27 12:05 GMT+02:00
Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent?
To: Christian Scheible <christian.schei...@uni-konstanz.de>
Cc: "dspace-tech@lists.sourceforge.net" <dspace-tech@lists.sourceforge.net>
Hi Christian,
I'm not sure I remember all the setup, but here goes...
in dspace.cfg there is
event.dispatcher.default.consumers = versioning, discovery, eperson,
harvester, eudatreplication, *xoai*
and
# consumer to maintain the oai index
event.consumer.xoai.class = cz.cuni.mff.ufal.event.OAIIndexEventConsumer
event.consumer.xoai.filters =
Community|Collection|Item|Bundle|Bitstream+Add|Create|Modify|Modify_Metadata|Delete|Remove
and you need to make sure the class is found by the event dispatcher, we
solve it by adding dspace-oai(where the consumer lives) as a dependency to
dspace-xmlui
diff --git a/dspace-xmlui/pom.xml b/dspace-xmlui/pom.xml
index 17d75c0..f19257f 100644
--- a/dspace-xmlui/pom.xml
+++ b/dspace-xmlui/pom.xml
@@ -93,6 +93,22 @@
</profiles>
<dependencies>
+ <!-- because of event listener that updates oai -->
+ <dependency>
+ <groupId>org.dspace</groupId>
+ <artifactId>dspace-oai</artifactId>
+ <classifier>classes</classifier>
+ </dependency>
I think that's about it...
Regards,
OK
2015-05-27 9:05 GMT+02:00 Christian Scheible <
christian.schei...@uni-konstanz.de>:
> Hi Ondřej,
>
> I have seen that you are using your own OAIIndexEventConsumer. We did the
> same but I added It in the module additions (which leads to code
> duplication).
> How do you register your Consumer? Is it the same way as for other
> consumers? Because I couldn't solve that problem.
>
> Best Regards,
> Christian
>
>
> Am 27.05.2015 um 08:41 schrieb Ondřej Košarko:
>
> In short no...
>
> As I've suggested in the thread the indexAll should include also the
> withdrawn, that's what we do
> https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L193
> We use our Event consumer to trigger the update (it currently doesn't
> handle caching and changes in discoverable flag)
> https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/cz/cuni/mff/ufal/event/OAIIndexEventConsumer.java
> And by the way dspace changed the deletion mode to transient in
> https://jira.duraspace.org/browse/DS-2491 so maybe the right way for
> dspace is updating the documentation...
>
>
> OK
>
>
>
> 2015-05-26 23:19 GMT+02:00 Jozef Misutka <misu...@ufal.mff.cuni.cz>:
>
>> Ondrej, is this happening to us as well?
>>
>> jm
>>
>> ------------------------------
>> *From: *"Tim Donohue" <tdono...@duraspace.org>
>> *To: *dspace-tech@lists.sourceforge.net
>> *Sent: *Tuesday, 26 May, 2015 23:09:20
>>
>> *Subject: *Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent?
>>
>> Hi All,
>>
>> I'm only just now "re-discovering" this thread, as I've found this same
>> behavior in DSpace OAI-PMH servers. Here's how to see this behavior (and
>> I've double checked it on http://demo.dspace.org, running 5.2):
>>
>> 1. Create an Item
>> 2. Run the OAI-PMH import (./dspace oai import).
>> 3. The item now appears in OAI-PMH
>> 4. Withdraw that Item
>> 5. Re-Run the OAI-PMH import (./dspace oai import)
>> 6. The Item still appears in OAI-PMH and is NEVER flagged as withdrawn.
>>
>> So, you can still access all its metadata, etc. The ONLY way to trigger
>> an update to the OAI-PMH record (of the withdrawn item) is to re-import
>> EVERYTHING (./dspace oai import -c). However, as noted, since withdrawn
>> items are not included in the XOAI "indexAll()" command, the item will
>> now disappear from OAI-PMH entirely.
>>
>> Here's that indexAll() command:
>>
>> https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191
>>
>> This behavior seems to be counter to what is documented at:
>>
>> https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions
>>
>> The documentation specifically states that "DSpace keeps a permanent
>> record of withdrawn items". It also states that a request for a
>> withdrawn item "will yield the 'record deleted' header".
>>
>> This sounds like a bug to me. The documentation definitely does not
>> match with the behavior.
>>
>> I've created a new bug ticket for this. It will need a volunteer to
>> resolve.
>>
>> https://jira.duraspace.org/browse/DS-2593
>>
>> - Tim
>>
>> On 3/6/2015 5:21 AM, helix84 wrote:
>> > Yes, that makes sense. Still, it needs a brief verification because
>> > sometimes our understanding of a concept and its implementation may
>> > differ.
>> >
>> >
>> > On Fri, Mar 6, 2015 at 10:12 AM, Kristian Roberto Salcedo
>> > <k.r.salc...@ub.uio.no> wrote:
>> >> Hi Ivan,
>> >>
>> >> I might have overlooked something, but from your comments on
>> https://jira.duraspace.org/browse/DS-2491 wouldnt it be meaningful
>> >> to do both your declaration change and Ondřej's suggestion in order to
>> match both what is declared and what the documentation says about OAI and
>> deleted items?
>> >>
>> >> regards,
>> >> Kristian
>> >>
>> >>> -----Original Message-----
>> >>> From: ivan.ma...@gmail.com [mailto:ivan.ma...@gmail.com] On Behalf Of
>> >>> helix84
>> >>> Sent: Thursday, March 05, 2015 3:46 PM
>> >>> To: Kristian Roberto Salcedo
>> >>> Cc: dspace-tech@lists.sourceforge.net; João Melo
>> >>> Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent?
>> >>>
>> >>> Hi Kristian,
>> >>>
>> >>> I think you're right that the declared
>> >>> <deletedRecord>persistent</deletedRecord> doesn't match how DSpace
>> >>> behaves. I filed a Jira issue and created a pull request to change
>> the declared
>> >>> status to <deletedRecord>transient</deletedRecord>.
>> >>>
>> >>> https://jira.duraspace.org/browse/DS-2491
>> >>>
>> >>>
>> >>> Regards,
>> >>> ~~helix84
>> >>>
>> >>> Compulsory reading: DSpace Mailing List Etiquette
>> >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>> >>>
>> >>>
>> >>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian Roberto Salcedo
>> >>> <k.r.salc...@ub.uio.no> wrote:
>> >>>> Hi all,
>> >>>>
>> >>>> In Dspace 4.2 we are currently seeing that our OAI feeds are not
>> being
>> >>> persistent as per these definitions when it comes to deleted
>> (withdrawn)
>> >>> items:
>> >>>>
>> >>>> http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion
>> >>>>
>> https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI-
>> >>> P
>> >>>> MHDataProvider2.0(Internals)-Deletions
>> >>>>
>> >>>> Withdrawn items do not get the <header status=deleted> as they
>> should,
>> >>> like in this example:
>> >>>>
>> >>>>
>> >>>
>> http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re
>> >>>> cord_Example_1
>> >>>>
>> >>>> Withdrawing an item is not reflected in the OAI entry in any way.
>> >>>>
>> >>>> ----------------------------------------------
>> >>>>
>> >>>> This is our Identify page at the moment:
>> >>>>
>> >>>> https://www.duo.uio.no/oai/request?verb=Identify
>> >>>>
>> >>>> ----------------------------------------------
>> >>>>
>> >>>> I believe we're doing everything right regarding config parameters
>> and
>> >>> maintenance of the solr oai index.
>> >>>>
>> >>>> The only place I can find a "persistent"-parameter is in this config
>> file:
>> >>>> /www/var/data/dspace/config/oaicat.properties
>> >>>> which I thought was deprecated, but we still set it just to be sure:
>> >>>> Identify.deletedRecord=persistent
>> >>>>
>> >>>> All other relevant config parameters are set in these two files as
>> far as I can
>> >>> tell:
>> >>>>
>> >>>> /www/var/data/dspace/config/modules/oai.cfg
>> >>>> /www/var/data/dspace/config/crosswalks/oai/description.xml
>> >>>>
>> >>>> We are running the oai import -o command nightly.
>> >>>>
>> >>>> ----------------------------------------------
>> >>>>
>> >>>> The only way we've found to remove withdrawn items from the OAI feed
>> is
>> >>> by running a complete re-indexing of the oai solr index:
>> >>>>
>> >>>> /www/var/data/dspace/bin/dspace oai import -c
>> >>>>
>> >>>> with a subsequent
>> >>>>
>> >>>> /www/var/data/dspace/bin/dspace oai clean-cache
>> >>>>
>> >>>> This removes a withdrawn item:
>> >>>>
>> >>>> https://www.duo.uio.no/handle/10852/42670
>> >>>>
>> >>>> completely from the feed:
>> >>>>
>> >>>>
>> >>> https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai
>> >>> _d
>> >>>> c&identifier=oai:localhost:10852/42670
>> >>>>
>> >>>> which is not what we want...
>> >>>>
>> >>>> ----------------------------------------------
>> >>>>
>> >>>>
>> >>>> Is anyone else experiencing the same problem?
>> >>>>
>> >>>> Maybe I've missed something - If this actually works differently
>> than we
>> >>> expect or has been addressed in OAI 2.1, please let me know.
>> >>>>
>> >>>>
>> >>>> regards,
>> >>>> Kristian Salcedo
>> >>>> Universitetet of Oslo Library
>> >>>> Department of digital services
>> >>>>
>> >>>>
>> >>>>
>> ----------------------------------------------------------------------
>> >>>> -------- Dive into the World of Parallel Programming The Go Parallel
>> >>>> Website, sponsored by Intel and developed in partnership with
>> Slashdot
>> >>>> Media, is your hub for all things parallel software development, from
>> >>>> weekly thought leadership blogs to news, videos, case studies,
>> >>>> tutorials and more. Take a look and join the conversation now.
>> >>>> http://goparallel.sourceforge.net/
>> >>>> _______________________________________________
>> >>>> DSpace-tech mailing list
>> >>>> DSpace-tech@lists.sourceforge.net
>> >>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>> >>>> List Etiquette:
>> >>>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> > by Intel and developed in partnership with Slashdot Media, is your hub
>> for all
>> > things parallel software development, from weekly thought leadership
>> blogs to
>> > news, videos, case studies, tutorials and more. Take a look and join the
>> > conversation now. http://goparallel.sourceforge.net/
>> > _______________________________________________
>> > DSpace-tech mailing list
>> > DSpace-tech@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/dspace-tech
>> > List Etiquette:
>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> DSpace-tech mailing list
>> DSpace-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>> List Etiquette:
>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>>
>
>
>
> ------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> DSpace-tech mailing
> listDSpace-tech@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
>
>
> --
> Christian Scheible
> Softwareentwickler / Abt. Content-basierte Dienste
> Kommunikations-, Informations- und Medienzentrum (KIM)
> Universität Konstanz
> 78457 Konstanz
> +49 (0)7531 / 88-2857
> Raum B 703
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
