Hi Ondřej, hi Tim,
It would be great to see this as a pull request. I had a solution for
that problem (the OAIIndexEventConsumer was part of dspace-api) but to
make it good enough for a pull request it would be needed to move the
xoai dependency from dspace-oai to dspace-api and additionally add at
least the caching part, org.dspace.xoai.app.XOAI and
org.dspace.xoai.util.ItemUtils to dspace-api.
Or as Ondřej suggested add a dependency to dspace-oai to xmlui, jspui,
rest and wherever the event has to be triggered.
Best Regards,
Christian
Am 28.05.2015 um 14:08 schrieb Ondřej Košarko:
Hey Tim,
No reason for not submitting a pull request and I for sure can move
the code around a bit. However, I don't see a way around adding
dspace-oai classes as a dependency. At the end of the day the consumer
just calls org.dspace.xoai.app.XOAI.index.
xmlui webapp does not have dspace-oai jar/classes in it's WEB-INF dir,
the same is probably true for jspui webapp.
Perhaps there is another approach?
Regards,
OK
2015-05-27 16:23 GMT+02:00 Tim Donohue <tdono...@duraspace.org
<mailto:tdono...@duraspace.org>>:
Hi Ondřej,
I wonder if you'd be willing to submit a Pull Request of these
changes (the new OAIIndexEventConsumer, etc)? These seem like
potentially useful changes overall, and would be something we'd
consider adding to DSpace.
My only (small) recommendation would be to simply put the
OAIIndexEventConsumer into the "dspace-api", so that it can be
easily enabled/disabled from the
"event.dispatcher.default.consumers" setting. Plus, if it's in the
dspace-api, that ensures it'd also be triggered by any updates
from other interface (XMLUI, JSPUI, REST, etc)
- Tim
On 5/27/2015 5:08 AM, Ondřej Košarko wrote:
---------- Forwarded message ----------
From: *Ondrej Kosarko* <ko...@centrum.cz
<mailto:ko...@centrum.cz> <mailto:ko...@centrum.cz
<mailto:ko...@centrum.cz>>>
Date: 2015-05-27 12:05 GMT+02:00
Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not
persistent?
To: Christian Scheible <christian.schei...@uni-konstanz.de
<mailto:christian.schei...@uni-konstanz.de>
<mailto:christian.schei...@uni-konstanz.de
<mailto:christian.schei...@uni-konstanz.de>>>
Cc: "dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>
<mailto:dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>>"
<dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>
<mailto:dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>>>
Hi Christian,
I'm not sure I remember all the setup, but here goes...
in dspace.cfg there is
event.dispatcher.default.consumers = versioning, discovery,
eperson,
harvester, eudatreplication, *xoai*
and
# consumer to maintain the oai index
event.consumer.xoai.class =
cz.cuni.mff.ufal.event.OAIIndexEventConsumer
event.consumer.xoai.filters =
Community|Collection|Item|Bundle|Bitstream+Add|Create|Modify|Modify_Metadata|Delete|Remove
and you need to make sure the class is found by the event
dispatcher, we
solve it by adding dspace-oai(where the consumer lives) as a
dependency
to dspace-xmlui
diff --git a/dspace-xmlui/pom.xml b/dspace-xmlui/pom.xml
index 17d75c0..f19257f 100644
--- a/dspace-xmlui/pom.xml
+++ b/dspace-xmlui/pom.xml
@@ -93,6 +93,22 @@
</profiles>
<dependencies>
+ <!-- because of event listener that updates oai -->
+ <dependency>
+ <groupId>org.dspace</groupId>
+ <artifactId>dspace-oai</artifactId>
+ <classifier>classes</classifier>
+ </dependency>
I think that's about it...
Regards,
OK
2015-05-27 9:05 GMT+02:00 Christian Scheible
<christian.schei...@uni-konstanz.de
<mailto:christian.schei...@uni-konstanz.de>
<mailto:christian.schei...@uni-konstanz.de
<mailto:christian.schei...@uni-konstanz.de>>>:
Hi Ondřej,
I have seen that you are using your own
OAIIndexEventConsumer. We
did the same but I added It in the module additions (which
leads to
code duplication).
How do you register your Consumer? Is it the same way as
for other
consumers? Because I couldn't solve that problem.
Best Regards,
Christian
Am 27.05.2015 um 08:41 schrieb Ondřej Košarko:
In short no...
As I've suggested in the thread the indexAll should
include also
the withdrawn, that's what we do
https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L193
We use our Event consumer to trigger the update (it
currently
doesn't handle caching and changes in discoverable flag)
https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/cz/cuni/mff/ufal/event/OAIIndexEventConsumer.java
And by the way dspace changed the deletion mode to
transient in
https://jira.duraspace.org/browse/DS-2491 so maybe the
right way
for dspace is updating the documentation...
OK
2015-05-26 23:19 GMT+02:00 Jozef Misutka
<misu...@ufal.mff.cuni.cz <mailto:misu...@ufal.mff.cuni.cz>
<mailto:misu...@ufal.mff.cuni.cz
<mailto:misu...@ufal.mff.cuni.cz>>>:
Ondrej, is this happening to us as well?
jm
------------------------------------------------------------------------
*From: *"Tim Donohue" <tdono...@duraspace.org
<mailto:tdono...@duraspace.org>
<mailto:tdono...@duraspace.org
<mailto:tdono...@duraspace.org>>>
*To: *dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>
<mailto:dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>>
*Sent: *Tuesday, 26 May, 2015 23:09:20
*Subject: *Re: [Dspace-tech] OAI-PMH data provider
2.0 not
persistent?
Hi All,
I'm only just now "re-discovering" this thread, as
I've found
this same
behavior in DSpace OAI-PMH servers. Here's how to
see this
behavior (and
I've double checked it on http://demo.dspace.org,
running 5.2):
1. Create an Item
2. Run the OAI-PMH import (./dspace oai import).
3. The item now appears in OAI-PMH
4. Withdraw that Item
5. Re-Run the OAI-PMH import (./dspace oai import)
6. The Item still appears in OAI-PMH and is NEVER
flagged as
withdrawn.
So, you can still access all its metadata, etc.
The ONLY way
to trigger
an update to the OAI-PMH record (of the withdrawn
item) is to
re-import
EVERYTHING (./dspace oai import -c). However, as
noted, since
withdrawn
items are not included in the XOAI "indexAll()"
command, the
item will
now disappear from OAI-PMH entirely.
Here's that indexAll() command:
https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191
This behavior seems to be counter to what is
documented at:
https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions
The documentation specifically states that "DSpace
keeps a
permanent
record of withdrawn items". It also states that a
request for a
withdrawn item "will yield the 'record deleted'
header".
This sounds like a bug to me. The documentation
definitely
does not
match with the behavior.
I've created a new bug ticket for this. It will need a
volunteer to resolve.
https://jira.duraspace.org/browse/DS-2593
- Tim
On 3/6/2015 5:21 AM, helix84 wrote:
> Yes, that makes sense. Still, it needs a brief
verification
because
> sometimes our understanding of a concept and its
implementation may
> differ.
>
>
> On Fri, Mar 6, 2015 at 10:12 AM, Kristian
Roberto Salcedo
> <k.r.salc...@ub.uio.no
<mailto:k.r.salc...@ub.uio.no>
<mailto:k.r.salc...@ub.uio.no
<mailto:k.r.salc...@ub.uio.no>>> wrote:
>> Hi Ivan,
>>
>> I might have overlooked something, but from
your comments
on https://jira.duraspace.org/browse/DS-2491
wouldnt it be
meaningful
>> to do both your declaration change and Ondřej's
suggestion
in order to match both what is declared and what the
documentation says about OAI and deleted items?
>>
>> regards,
>> Kristian
>>
>>> -----Original Message-----
>>> From: ivan.ma...@gmail.com
<mailto:ivan.ma...@gmail.com> <mailto:ivan.ma...@gmail.com
<mailto:ivan.ma...@gmail.com>>
[mailto:ivan.ma...@gmail.com
<mailto:ivan.ma...@gmail.com> <mailto:ivan.ma...@gmail.com
<mailto:ivan.ma...@gmail.com>>] On
Behalf Of
>>> helix84
>>> Sent: Thursday, March 05, 2015 3:46 PM
>>> To: Kristian Roberto Salcedo
>>> Cc: dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>
<mailto:dspace-tech@lists.sourceforge.net
<mailto:dspace-tech@lists.sourceforge.net>>; João Melo
>>> Subject: Re: [Dspace-tech] OAI-PMH data
provider 2.0 not
persistent?
>>>
>>> Hi Kristian,
>>>
>>> I think you're right that the declared
>>> <deletedRecord>persistent</deletedRecord>
doesn't match
how DSpace
>>> behaves. I filed a Jira issue and created a
pull request
to change the declared
>>> status to
<deletedRecord>transient</deletedRecord>.
>>>
>>> https://jira.duraspace.org/browse/DS-2491
>>>
>>>
>>> Regards,
>>> ~~helix84
>>>
>>> Compulsory reading: DSpace Mailing List Etiquette
>>>
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>>>
>>>
>>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian
Roberto Salcedo
>>> <k.r.salc...@ub.uio.no
<mailto:k.r.salc...@ub.uio.no>
<mailto:k.r.salc...@ub.uio.no
<mailto:k.r.salc...@ub.uio.no>>> wrote:
>>>> Hi all,
>>>>
>>>> In Dspace 4.2 we are currently seeing that
our OAI feeds
are not being
>>> persistent as per these definitions when it
comes to
deleted (withdrawn)
>>> items:
>>>>
>>>>
http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion
>>>>
https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI-
>>> P
>>>> MHDataProvider2.0(Internals)-Deletions
>>>>
>>>> Withdrawn items do not get the <header
status=deleted> as
they should,
>>> like in this example:
>>>>
>>>>
>>>
http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re
>>>> cord_Example_1
>>>>
>>>> Withdrawing an item is not reflected in the
OAI entry in
any way.
>>>>
>>>> ----------------------------------------------
>>>>
>>>> This is our Identify page at the moment:
>>>>
>>>> https://www.duo.uio.no/oai/request?verb=Identify
>>>>
>>>> ----------------------------------------------
>>>>
>>>> I believe we're doing everything right
regarding config
parameters and
>>> maintenance of the solr oai index.
>>>>
>>>> The only place I can find a
"persistent"-parameter is in
this config file:
>>>> /www/var/data/dspace/config/oaicat.properties
>>>> which I thought was deprecated, but we still
set it just
to be sure:
>>>> Identify.deletedRecord=persistent
>>>>
>>>> All other relevant config parameters are set
in these two
files as far as I can
>>> tell:
>>>>
>>>> /www/var/data/dspace/config/modules/oai.cfg
>>>>
/www/var/data/dspace/config/crosswalks/oai/description.xml
>>>>
>>>> We are running the oai import -o command nightly.
>>>>
>>>> ----------------------------------------------
>>>>
>>>> The only way we've found to remove withdrawn
items from
the OAI feed is
>>> by running a complete re-indexing of the oai
solr index:
>>>>
>>>> /www/var/data/dspace/bin/dspace oai import -c
>>>>
>>>> with a subsequent
>>>>
>>>> /www/var/data/dspace/bin/dspace oai clean-cache
>>>>
>>>> This removes a withdrawn item:
>>>>
>>>> https://www.duo.uio.no/handle/10852/42670
>>>>
>>>> completely from the feed:
>>>>
>>>>
>>>
https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai
>>> _d
>>>> c&identifier=oai:localhost:10852/42670
>>>>
>>>> which is not what we want...
>>>>
>>>> ----------------------------------------------
>>>>
>>>>
>>>> Is anyone else experiencing the same problem?
>>>>
>>>> Maybe I've missed something - If this
actually works
differently than we
>>> expect or has been addressed in OAI 2.1,
please let me know.
>>>>
>>>>
>>>> regards,
>>>> Kristian Salcedo
>>>> Universitetet of Oslo Library
>>>> Department of digital services
>>>>
>>>>
>>>>
----------------------------------------------------------------------
>>>> -------- Dive into the World of Parallel
Programming The
Go Parallel
>>>> Website, sponsored by Intel and developed in
partnership
with Slashdot
>>>> Media, is your hub for all things parallel
software
development, from
>>>> weekly thought leadership blogs to news,
videos, case
studies,
>>>> tutorials and more. Take a look and join the
conversation
now.
>>>> http://goparallel.sourceforge.net/
>>>> _______________________________________________
>>>> DSpace-tech mailing list
>>>> DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
<mailto:DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>>
>>>>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>>> List Etiquette:
>>>>
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
>
------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The
Go Parallel
Website, sponsored
> by Intel and developed in partnership with
Slashdot Media,
is your hub for all
> things parallel software development, from
weekly thought
leadership blogs to
> news, videos, case studies, tutorials and more.
Take a look
and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
<mailto:DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>>
>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
> List Etiquette:
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
<mailto:DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette:
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
<mailto:DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List
Etiquette:https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
--
Christian Scheible
Softwareentwickler / Abt. Content-basierte Dienste
Kommunikations-, Informations- und Medienzentrum (KIM)
Universität Konstanz
78457 Konstanz
+49 (0)7531 / 88-2857
Raum B 703
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
<mailto:DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette:
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
<mailto:DSpace-tech@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette:
https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
--
Christian Scheible
Softwareentwickler / Abt. Content-basierte Dienste
Kommunikations-, Informations- und Medienzentrum (KIM)
Universität Konstanz
78457 Konstanz
+49 (0)7531 / 88-2857
Raum B 703
------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette