In short no... As I've suggested in the thread the indexAll should include also the withdrawn, that's what we do https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L193 We use our Event consumer to trigger the update (it currently doesn't handle caching and changes in discoverable flag) https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/cz/cuni/mff/ufal/event/OAIIndexEventConsumer.java And by the way dspace changed the deletion mode to transient in https://jira.duraspace.org/browse/DS-2491 so maybe the right way for dspace is updating the documentation...
OK 2015-05-26 23:19 GMT+02:00 Jozef Misutka <misu...@ufal.mff.cuni.cz>: > Ondrej, is this happening to us as well? > > jm > > ------------------------------ > *From: *"Tim Donohue" <tdono...@duraspace.org> > *To: *dspace-tech@lists.sourceforge.net > *Sent: *Tuesday, 26 May, 2015 23:09:20 > > *Subject: *Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent? > > Hi All, > > I'm only just now "re-discovering" this thread, as I've found this same > behavior in DSpace OAI-PMH servers. Here's how to see this behavior (and > I've double checked it on http://demo.dspace.org, running 5.2): > > 1. Create an Item > 2. Run the OAI-PMH import (./dspace oai import). > 3. The item now appears in OAI-PMH > 4. Withdraw that Item > 5. Re-Run the OAI-PMH import (./dspace oai import) > 6. The Item still appears in OAI-PMH and is NEVER flagged as withdrawn. > > So, you can still access all its metadata, etc. The ONLY way to trigger > an update to the OAI-PMH record (of the withdrawn item) is to re-import > EVERYTHING (./dspace oai import -c). However, as noted, since withdrawn > items are not included in the XOAI "indexAll()" command, the item will > now disappear from OAI-PMH entirely. > > Here's that indexAll() command: > > https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191 > > This behavior seems to be counter to what is documented at: > > https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions > > The documentation specifically states that "DSpace keeps a permanent > record of withdrawn items". It also states that a request for a > withdrawn item "will yield the 'record deleted' header". > > This sounds like a bug to me. The documentation definitely does not > match with the behavior. > > I've created a new bug ticket for this. It will need a volunteer to > resolve. > > https://jira.duraspace.org/browse/DS-2593 > > - Tim > > On 3/6/2015 5:21 AM, helix84 wrote: > > Yes, that makes sense. Still, it needs a brief verification because > > sometimes our understanding of a concept and its implementation may > > differ. > > > > > > On Fri, Mar 6, 2015 at 10:12 AM, Kristian Roberto Salcedo > > <k.r.salc...@ub.uio.no> wrote: > >> Hi Ivan, > >> > >> I might have overlooked something, but from your comments on > https://jira.duraspace.org/browse/DS-2491 wouldnt it be meaningful > >> to do both your declaration change and Ondřej's suggestion in order to > match both what is declared and what the documentation says about OAI and > deleted items? > >> > >> regards, > >> Kristian > >> > >>> -----Original Message----- > >>> From: ivan.ma...@gmail.com [mailto:ivan.ma...@gmail.com] On Behalf Of > >>> helix84 > >>> Sent: Thursday, March 05, 2015 3:46 PM > >>> To: Kristian Roberto Salcedo > >>> Cc: dspace-tech@lists.sourceforge.net; João Melo > >>> Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not persistent? > >>> > >>> Hi Kristian, > >>> > >>> I think you're right that the declared > >>> <deletedRecord>persistent</deletedRecord> doesn't match how DSpace > >>> behaves. I filed a Jira issue and created a pull request to change the > declared > >>> status to <deletedRecord>transient</deletedRecord>. > >>> > >>> https://jira.duraspace.org/browse/DS-2491 > >>> > >>> > >>> Regards, > >>> ~~helix84 > >>> > >>> Compulsory reading: DSpace Mailing List Etiquette > >>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > >>> > >>> > >>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian Roberto Salcedo > >>> <k.r.salc...@ub.uio.no> wrote: > >>>> Hi all, > >>>> > >>>> In Dspace 4.2 we are currently seeing that our OAI feeds are not being > >>> persistent as per these definitions when it comes to deleted > (withdrawn) > >>> items: > >>>> > >>>> http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion > >>>> https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI- > >>> P > >>>> MHDataProvider2.0(Internals)-Deletions > >>>> > >>>> Withdrawn items do not get the <header status=deleted> as they should, > >>> like in this example: > >>>> > >>>> > >>> http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re > >>>> cord_Example_1 > >>>> > >>>> Withdrawing an item is not reflected in the OAI entry in any way. > >>>> > >>>> ---------------------------------------------- > >>>> > >>>> This is our Identify page at the moment: > >>>> > >>>> https://www.duo.uio.no/oai/request?verb=Identify > >>>> > >>>> ---------------------------------------------- > >>>> > >>>> I believe we're doing everything right regarding config parameters and > >>> maintenance of the solr oai index. > >>>> > >>>> The only place I can find a "persistent"-parameter is in this config > file: > >>>> /www/var/data/dspace/config/oaicat.properties > >>>> which I thought was deprecated, but we still set it just to be sure: > >>>> Identify.deletedRecord=persistent > >>>> > >>>> All other relevant config parameters are set in these two files as > far as I can > >>> tell: > >>>> > >>>> /www/var/data/dspace/config/modules/oai.cfg > >>>> /www/var/data/dspace/config/crosswalks/oai/description.xml > >>>> > >>>> We are running the oai import -o command nightly. > >>>> > >>>> ---------------------------------------------- > >>>> > >>>> The only way we've found to remove withdrawn items from the OAI feed > is > >>> by running a complete re-indexing of the oai solr index: > >>>> > >>>> /www/var/data/dspace/bin/dspace oai import -c > >>>> > >>>> with a subsequent > >>>> > >>>> /www/var/data/dspace/bin/dspace oai clean-cache > >>>> > >>>> This removes a withdrawn item: > >>>> > >>>> https://www.duo.uio.no/handle/10852/42670 > >>>> > >>>> completely from the feed: > >>>> > >>>> > >>> https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai > >>> _d > >>>> c&identifier=oai:localhost:10852/42670 > >>>> > >>>> which is not what we want... > >>>> > >>>> ---------------------------------------------- > >>>> > >>>> > >>>> Is anyone else experiencing the same problem? > >>>> > >>>> Maybe I've missed something - If this actually works differently than > we > >>> expect or has been addressed in OAI 2.1, please let me know. > >>>> > >>>> > >>>> regards, > >>>> Kristian Salcedo > >>>> Universitetet of Oslo Library > >>>> Department of digital services > >>>> > >>>> > >>>> ---------------------------------------------------------------------- > >>>> -------- Dive into the World of Parallel Programming The Go Parallel > >>>> Website, sponsored by Intel and developed in partnership with Slashdot > >>>> Media, is your hub for all things parallel software development, from > >>>> weekly thought leadership blogs to news, videos, case studies, > >>>> tutorials and more. Take a look and join the conversation now. > >>>> http://goparallel.sourceforge.net/ > >>>> _______________________________________________ > >>>> DSpace-tech mailing list > >>>> DSpace-tech@lists.sourceforge.net > >>>> https://lists.sourceforge.net/lists/listinfo/dspace-tech > >>>> List Etiquette: > >>>> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > > > > > ------------------------------------------------------------------------------ > > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > > by Intel and developed in partnership with Slashdot Media, is your hub > for all > > things parallel software development, from weekly thought leadership > blogs to > > news, videos, case studies, tutorials and more. Take a look and join the > > conversation now. http://goparallel.sourceforge.net/ > > _______________________________________________ > > DSpace-tech mailing list > > DSpace-tech@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/dspace-tech > > List Etiquette: > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > DSpace-tech mailing list > DSpace-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dspace-tech > List Etiquette: > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >
------------------------------------------------------------------------------
_______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
