Hi Ondřej, hi Tim,

It would be great to see this as a pull request. I had a solution for that problem (the OAIIndexEventConsumer was part of dspace-api) but to make it good enough for a pull request it would be needed to move the xoai dependency from dspace-oai to dspace-api and additionally add at least the caching part, org.dspace.xoai.app.XOAI and org.dspace.xoai.util.ItemUtils to dspace-api. Or as Ondřej suggested add a dependency to dspace-oai to xmlui, jspui, rest and wherever the event has to be triggered.

Best Regards,
Christian

Am 28.05.2015 um 14:08 schrieb Ondřej Košarko:
Hey Tim,

No reason for not submitting a pull request and I for sure can move the code around a bit. However, I don't see a way around adding dspace-oai classes as a dependency. At the end of the day the consumer just calls org.dspace.xoai.app.XOAI.index.

xmlui webapp does not have dspace-oai jar/classes in it's WEB-INF dir, the same is probably true for jspui webapp.

Perhaps there is another approach?

Regards,
OK

2015-05-27 16:23 GMT+02:00 Tim Donohue <tdono...@duraspace.org <mailto:tdono...@duraspace.org>>:

    Hi Ondřej,

    I wonder if you'd be willing to submit a Pull Request of these
    changes (the new OAIIndexEventConsumer, etc)?  These seem like
    potentially useful changes overall, and would be something we'd
    consider adding to DSpace.

    My only (small) recommendation would be to simply put the
    OAIIndexEventConsumer into the "dspace-api", so that it can be
    easily enabled/disabled from the
    "event.dispatcher.default.consumers" setting. Plus, if it's in the
    dspace-api, that ensures it'd also be triggered by any updates
    from other interface (XMLUI, JSPUI, REST, etc)

    - Tim

    On 5/27/2015 5:08 AM, Ondřej Košarko wrote:


        ---------- Forwarded message ----------
        From: *Ondrej Kosarko* <ko...@centrum.cz
        <mailto:ko...@centrum.cz> <mailto:ko...@centrum.cz
        <mailto:ko...@centrum.cz>>>
        Date: 2015-05-27 12:05 GMT+02:00
        Subject: Re: [Dspace-tech] OAI-PMH data provider 2.0 not
        persistent?
        To: Christian Scheible <christian.schei...@uni-konstanz.de
        <mailto:christian.schei...@uni-konstanz.de>
        <mailto:christian.schei...@uni-konstanz.de
        <mailto:christian.schei...@uni-konstanz.de>>>
        Cc: "dspace-tech@lists.sourceforge.net
        <mailto:dspace-tech@lists.sourceforge.net>
        <mailto:dspace-tech@lists.sourceforge.net
        <mailto:dspace-tech@lists.sourceforge.net>>"
        <dspace-tech@lists.sourceforge.net
        <mailto:dspace-tech@lists.sourceforge.net>
        <mailto:dspace-tech@lists.sourceforge.net
        <mailto:dspace-tech@lists.sourceforge.net>>>


        Hi Christian,

        I'm not sure I remember all the setup, but here goes...

        in dspace.cfg there is

           event.dispatcher.default.consumers = versioning, discovery,
        eperson,
        harvester, eudatreplication, *xoai*


        and

        # consumer to maintain the oai index
        event.consumer.xoai.class =
        cz.cuni.mff.ufal.event.OAIIndexEventConsumer
        event.consumer.xoai.filters =
        
Community|Collection|Item|Bundle|Bitstream+Add|Create|Modify|Modify_Metadata|Delete|Remove


        and you need to make sure the class is found by the event
        dispatcher, we
        solve it by adding dspace-oai(where the consumer lives) as a
        dependency
        to dspace-xmlui

        diff --git a/dspace-xmlui/pom.xml b/dspace-xmlui/pom.xml
        index 17d75c0..f19257f 100644
        --- a/dspace-xmlui/pom.xml
        +++ b/dspace-xmlui/pom.xml
        @@ -93,6 +93,22 @@
              </profiles>
              <dependencies>
        +     <!-- because of event listener that updates oai -->
        +        <dependency>
        +  <groupId>org.dspace</groupId>
        +  <artifactId>dspace-oai</artifactId>
        +        <classifier>classes</classifier>
        +        </dependency>

        I think that's about it...

        Regards,
        OK


        2015-05-27 9:05 GMT+02:00 Christian Scheible
        <christian.schei...@uni-konstanz.de
        <mailto:christian.schei...@uni-konstanz.de>
        <mailto:christian.schei...@uni-konstanz.de
        <mailto:christian.schei...@uni-konstanz.de>>>:

            Hi Ondřej,

            I have seen that you are using your own
        OAIIndexEventConsumer. We
            did the same but I added It in the module additions (which
        leads to
            code duplication).
            How do you register your Consumer? Is it the same way as
        for other
            consumers? Because I couldn't solve that problem.

            Best Regards,
            Christian


            Am 27.05.2015 um 08:41 schrieb Ondřej Košarko:

                In short no...

                As I've suggested in the thread the indexAll should
            include also
                the withdrawn, that's what we do
            
https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L193
                We use our Event consumer to trigger the update (it
            currently
                doesn't handle caching and changes in discoverable flag)
            
https://github.com/ufal/lindat-dspace/blob/lindat/dspace-oai/src/main/java/cz/cuni/mff/ufal/event/OAIIndexEventConsumer.java
                And by the way dspace changed the deletion mode to
            transient in
            https://jira.duraspace.org/browse/DS-2491 so maybe the
            right way
                for dspace is updating the documentation...


                OK



                2015-05-26 23:19 GMT+02:00 Jozef Misutka
            <misu...@ufal.mff.cuni.cz <mailto:misu...@ufal.mff.cuni.cz>
                <mailto:misu...@ufal.mff.cuni.cz
            <mailto:misu...@ufal.mff.cuni.cz>>>:

                    Ondrej, is this happening to us as well?

                    jm

            
------------------------------------------------------------------------
                    *From: *"Tim Donohue" <tdono...@duraspace.org
            <mailto:tdono...@duraspace.org>
                    <mailto:tdono...@duraspace.org
            <mailto:tdono...@duraspace.org>>>
                    *To: *dspace-tech@lists.sourceforge.net
            <mailto:dspace-tech@lists.sourceforge.net>
                    <mailto:dspace-tech@lists.sourceforge.net
            <mailto:dspace-tech@lists.sourceforge.net>>
                    *Sent: *Tuesday, 26 May, 2015 23:09:20

                    *Subject: *Re: [Dspace-tech] OAI-PMH data provider
            2.0 not

                    persistent?

                    Hi All,

                    I'm only just now "re-discovering" this thread, as
            I've found
                    this same
                    behavior in DSpace OAI-PMH servers. Here's how to
            see this
                    behavior (and
                    I've double checked it on http://demo.dspace.org,
            running 5.2):

                    1. Create an Item
                    2. Run the OAI-PMH import (./dspace oai import).
                    3. The item now appears in OAI-PMH
                    4. Withdraw that Item
                    5. Re-Run the OAI-PMH import (./dspace oai import)
                    6. The Item still appears in OAI-PMH and is NEVER
            flagged as
                    withdrawn.

                    So, you can still access all its metadata, etc.
            The ONLY way
                    to trigger
                    an update to the OAI-PMH record (of the withdrawn
            item) is to
                    re-import
                    EVERYTHING (./dspace oai import -c). However, as
            noted, since
                    withdrawn
                    items are not included in the XOAI "indexAll()"
            command, the
                    item will
                    now disappear from OAI-PMH entirely.

                    Here's that indexAll() command:
            
https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java#L191

                    This behavior seems to be counter to what is
            documented at:
            
https://wiki.duraspace.org/pages/viewpage.action?pageId=45548245#OAI-PMHDataProvider2.0%28Internals%29-Deletions

                    The documentation specifically states that "DSpace
            keeps a
                    permanent
                    record of withdrawn items". It also states that a
            request for a
                    withdrawn item "will yield the 'record deleted'
            header".

                    This sounds like a bug to me. The documentation
            definitely
                    does not
                    match with the behavior.

                    I've created a new bug ticket for this. It will need a
                    volunteer to resolve.

            https://jira.duraspace.org/browse/DS-2593

                    - Tim

                    On 3/6/2015 5:21 AM, helix84 wrote:
                    > Yes, that makes sense. Still, it needs a brief
            verification
                    because
                    > sometimes our understanding of a concept and its
                    implementation may
                    > differ.
                    >
                    >
                    > On Fri, Mar 6, 2015 at 10:12 AM, Kristian
            Roberto Salcedo
                    > <k.r.salc...@ub.uio.no
            <mailto:k.r.salc...@ub.uio.no>
            <mailto:k.r.salc...@ub.uio.no
            <mailto:k.r.salc...@ub.uio.no>>> wrote:
                    >> Hi Ivan,
                    >>
                    >> I might have overlooked something, but from
            your comments
                    on https://jira.duraspace.org/browse/DS-2491
            wouldnt it be
                    meaningful
                    >> to do both your declaration change and Ondřej's
            suggestion
                    in order to match both what is declared and what the
                    documentation says about OAI and deleted items?
                    >>
                    >> regards,
                    >> Kristian
                    >>
                    >>> -----Original Message-----
                    >>> From: ivan.ma...@gmail.com
            <mailto:ivan.ma...@gmail.com> <mailto:ivan.ma...@gmail.com
            <mailto:ivan.ma...@gmail.com>>
                    [mailto:ivan.ma...@gmail.com
            <mailto:ivan.ma...@gmail.com> <mailto:ivan.ma...@gmail.com
            <mailto:ivan.ma...@gmail.com>>] On
                    Behalf Of
                    >>> helix84
                    >>> Sent: Thursday, March 05, 2015 3:46 PM
                    >>> To: Kristian Roberto Salcedo
                    >>> Cc: dspace-tech@lists.sourceforge.net
            <mailto:dspace-tech@lists.sourceforge.net>
                    <mailto:dspace-tech@lists.sourceforge.net
            <mailto:dspace-tech@lists.sourceforge.net>>; João Melo
                    >>> Subject: Re: [Dspace-tech] OAI-PMH data
            provider 2.0 not
                    persistent?
                    >>>
                    >>> Hi Kristian,
                    >>>
                    >>> I think you're right that the declared
                    >>> <deletedRecord>persistent</deletedRecord>
            doesn't match
                    how DSpace
                    >>> behaves. I filed a Jira issue and created a
            pull request
                    to change the declared
                    >>> status to
            <deletedRecord>transient</deletedRecord>.
                    >>>
                    >>> https://jira.duraspace.org/browse/DS-2491
                    >>>
                    >>>
                    >>> Regards,
                    >>> ~~helix84
                    >>>
                    >>> Compulsory reading: DSpace Mailing List Etiquette
                    >>>
            https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
                    >>>
                    >>>
                    >>> On Thu, Mar 5, 2015 at 2:52 PM, Kristian
            Roberto Salcedo
                    >>> <k.r.salc...@ub.uio.no
            <mailto:k.r.salc...@ub.uio.no>
            <mailto:k.r.salc...@ub.uio.no
            <mailto:k.r.salc...@ub.uio.no>>> wrote:
                    >>>> Hi all,
                    >>>>
                    >>>> In Dspace 4.2 we are currently seeing that
            our OAI feeds
                    are not being
                    >>> persistent as per these definitions when it
            comes to
                    deleted (withdrawn)
                    >>> items:
                    >>>>
                    >>>>
            http://www.openarchives.org/OAI/openarchivesprotocol.html#deletion
                    >>>>
            
https://wiki.duraspace.org/pages/viewpage.action?pageId=34640887#OAI-
                    >>> P
                    >>>> MHDataProvider2.0(Internals)-Deletions
                    >>>>
                    >>>> Withdrawn items do not get the <header
            status=deleted> as
                    they should,
                    >>> like in this example:
                    >>>>
                    >>>>
                    >>>
            
http://webservices.itcs.umich.edu/mediawiki/oaibp/index.php/Deleted_Re
                    >>>> cord_Example_1
                    >>>>
                    >>>> Withdrawing an item is not reflected in the
            OAI entry in
                    any way.
                    >>>>
                    >>>> ----------------------------------------------
                    >>>>
                    >>>> This is our Identify page at the moment:
                    >>>>
                    >>>> https://www.duo.uio.no/oai/request?verb=Identify
                    >>>>
                    >>>> ----------------------------------------------
                    >>>>
                    >>>> I believe we're doing everything right
            regarding config
                    parameters and
                    >>> maintenance of the solr oai index.
                    >>>>
                    >>>> The only place I can find a
            "persistent"-parameter is in
                    this config file:
                    >>>> /www/var/data/dspace/config/oaicat.properties
                    >>>> which I thought was deprecated, but we still
            set it just
                    to be sure:
                    >>>> Identify.deletedRecord=persistent
                    >>>>
                    >>>> All other relevant config parameters are set
            in these two
                    files as far as I can
                    >>> tell:
                    >>>>
                    >>>> /www/var/data/dspace/config/modules/oai.cfg
                    >>>>
            /www/var/data/dspace/config/crosswalks/oai/description.xml
                    >>>>
                    >>>> We are running the oai import -o command nightly.
                    >>>>
                    >>>> ----------------------------------------------
                    >>>>
                    >>>> The only way we've found to remove withdrawn
            items from
                    the OAI feed is
                    >>> by running a complete re-indexing of the oai
            solr index:
                    >>>>
                    >>>> /www/var/data/dspace/bin/dspace oai import -c
                    >>>>
                    >>>> with a subsequent
                    >>>>
                    >>>> /www/var/data/dspace/bin/dspace oai clean-cache
                    >>>>
                    >>>> This removes a withdrawn item:
                    >>>>
                    >>>> https://www.duo.uio.no/handle/10852/42670
                    >>>>
                    >>>> completely from the feed:
                    >>>>
                    >>>>
                    >>>
            https://www.duo.uio.no/oai/request?verb=GetRecord&metadataPrefix=oai
                    >>> _d
                    >>>> c&identifier=oai:localhost:10852/42670
                    >>>>
                    >>>> which is not what we want...
                    >>>>
                    >>>> ----------------------------------------------
                    >>>>
                    >>>>
                    >>>> Is anyone else experiencing the same problem?
                    >>>>
                    >>>> Maybe I've missed something - If this
            actually works
                    differently than we
                    >>> expect or has been addressed in OAI 2.1,
            please let me know.
                    >>>>
                    >>>>
                    >>>> regards,
                    >>>> Kristian Salcedo
                    >>>> Universitetet of Oslo Library
                    >>>> Department of digital services
                    >>>>
                    >>>>
                    >>>>
            
----------------------------------------------------------------------
                    >>>> -------- Dive into the World of Parallel
            Programming The
                    Go Parallel
                    >>>> Website, sponsored by Intel and developed in
            partnership
                    with Slashdot
                    >>>> Media, is your hub for all things parallel
            software
                    development, from
                    >>>> weekly thought leadership blogs to news,
            videos, case
                    studies,
                    >>>> tutorials and more. Take a look and join the
            conversation
                    now.
                    >>>> http://goparallel.sourceforge.net/
                    >>>> _______________________________________________
                    >>>> DSpace-tech mailing list
                    >>>> DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>
                    <mailto:DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>>
                    >>>>
            https://lists.sourceforge.net/lists/listinfo/dspace-tech
                    >>>> List Etiquette:
                    >>>>
            https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
                    >
                    >
            
------------------------------------------------------------------------------
                    > Dive into the World of Parallel Programming The
            Go Parallel
                    Website, sponsored
                    > by Intel and developed in partnership with
            Slashdot Media,
                    is your hub for all
                    > things parallel software development, from
            weekly thought
                    leadership blogs to
                    > news, videos, case studies, tutorials and more.
            Take a look
                    and join the
                    > conversation now. http://goparallel.sourceforge.net/
                    > _______________________________________________
                    > DSpace-tech mailing list
                    > DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>
                    <mailto:DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>>
                    >
            https://lists.sourceforge.net/lists/listinfo/dspace-tech
                    > List Etiquette:
            https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
                    >

            
------------------------------------------------------------------------------
            _______________________________________________
                    DSpace-tech mailing list
            DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>
                    <mailto:DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>>
            https://lists.sourceforge.net/lists/listinfo/dspace-tech
                    List Etiquette:
            https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette




            
------------------------------------------------------------------------------


                _______________________________________________
                DSpace-tech mailing list
            DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>
            <mailto:DSpace-tech@lists.sourceforge.net
            <mailto:DSpace-tech@lists.sourceforge.net>>
            https://lists.sourceforge.net/lists/listinfo/dspace-tech
                List
            
Etiquette:https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette



            --
            Christian Scheible
            Softwareentwickler / Abt. Content-basierte Dienste
            Kommunikations-, Informations- und Medienzentrum (KIM)
            Universität Konstanz
            78457 Konstanz
            +49 (0)7531 / 88-2857
            Raum B 703


        
------------------------------------------------------------------------------

            _______________________________________________
            DSpace-tech mailing list
        DSpace-tech@lists.sourceforge.net
        <mailto:DSpace-tech@lists.sourceforge.net>
            <mailto:DSpace-tech@lists.sourceforge.net
        <mailto:DSpace-tech@lists.sourceforge.net>>
        https://lists.sourceforge.net/lists/listinfo/dspace-tech
            List Etiquette:
        https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette





        
------------------------------------------------------------------------------



        _______________________________________________
        DSpace-tech mailing list
        DSpace-tech@lists.sourceforge.net
        <mailto:DSpace-tech@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/dspace-tech
        List Etiquette:
        https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette




------------------------------------------------------------------------------


_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette


--
Christian Scheible
Softwareentwickler / Abt. Content-basierte Dienste
Kommunikations-, Informations- und Medienzentrum (KIM)
Universität Konstanz
78457 Konstanz
+49 (0)7531 / 88-2857
Raum B 703

------------------------------------------------------------------------------
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

Reply via email to