Kia ora Kim, Thanks very much for your help! Cleaning the cache resolves much of our confusion at least (and not having to restart the server will save us a great deal of time!) so now I can see clearly what’s going on and what we want to do.
So now we’re at the point where bitstreamaccessFilter is filtering out both the collections based on their restrictive bundle policies. What we’d like is a filter that acts on *item* policies. So where the item is anon read it would be included in the OAI feed for the appropriate context, but where the item is restricted it wouldn’t. Is there an existing filter that would do this, or does this require editing the Java source code (not in our skillset but we could look at contracting out)? Deborah From: Kim Shepherd [mailto:kim.sheph...@gmail.com] Sent: Monday, 1 June 2015 11:28 p.m. To: Fitchett, Deborah Cc: dspace-tech (dspace-tech@lists.sourceforge.net) Subject: Re: [Dspace-tech] XOAI and filters based on access permissions Kia ora Deborah, I've done a bit of testing in 5.2, and will do the same test in 3.4 tomorrow -- though I think it'll be the same, looking at the DSpaceAuthorizationFilter in 3.x vs 5.x, and my findings/notes are that this does work nicely, in that people won't be able to do a GetRecord for your restricted items in *any* context (including default) that uses the bitstreamaccessFilter filter (now called "bitstreamAccessCondition" in DSpace 5.x for any others reading this), BUT the trick is that the authorization filter doesn't actually look at bitstream policies! It looks at bundle policies. I'm not really sure why.* Here's an example you can try in my test repository: Record page: (i'm keeping item public so we can check it) http://dspace.shepherd.nz/handle/or2015/256 OAI "GetRecord" request for that handle: http://dspace.shepherd.nz/oai/request?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:dspace.shepherd.nz:or2015/256 You should see an error response with "The given id does not exist" So here is my suggestion: 1. Make sure that your default context references the "bitstreamaccessFilter", and any other contexts in which you don't wish these items to appear (there is the potential that you could have contexts in which you generally don't want this filter, but do want to exclude Symplectic items - in these cases, you may need to rethink - in DSpace 5 there are some clever things that can be done with ANDing and ORing various filters together, but prior to that you might have to think about filtering based on metadata instead. It sounds like as long as you keep your Restricted Theses as they are and keep the bundle policies in place, this will work OK for you) 2. Find a few items and makes sure there is no Anonymous:READ policy for the "ORIGINAL" bundle, and then rebuild your OAI catalog with ${dspace}/bin/dspace oai import -c -v ${dspace}/bin/dspace oai import clean-cache Some good news: you don't need to restart Tomcat to check your changes! The DSpace kernel reads configuration whenever you run that "dspace oai" command, and is independent of Tomcat. For the future, and for others wondering about issues like this, there are also ways to use logical filter definitions to "NOT" a MetadataValueExists filter, so you could check for the existence of a metadata value like pubs.elements-id (Symplectic Elements adds this to every repository item) and *exclude* the item from the context. (and for these specific items, which you've mentioned are also non-public at the ITEM level, I believe the fix https://jira.duraspace.org/browse/DS-1856 in DSpace 5 would fix this without any other intervention). Anyway, I am hoping my suggestions above work for 3.x - I'll try to find time to test them - and sorry to keep harping on about DSpace 5, I have it on the brain a bit at the moment ;) I realise "upgrade!" is not a very good answer to specific problems like this! Cheers Kim * If you look at https://github.com/DSpace/DSpace/blob/master/dspace-oai/src/main/java/org/dspace/xoai/filter/DSpaceAuthorizationFilter.java, you will see the check: Item dspaceItem = (Item) HandleManager.resolveToObject(context, handle); AuthorizeManager.authorizeAction(context, dspaceItem, Constants.READ); for (Bundle b : dspaceItem.getBundles()) AuthorizeManager.authorizeAction(context, b, Constants.READ); return true; It's the same in 3.x classes. This might be intentional or it might just be an oversight while iterating item bundles, I'll check doc and might log a JIRA issue M: k...@shepherd.nz<mailto:k...@shepherd.nz> T: @kimshepherd P: +6421883635 0CCB D957 0C35 F5C1 497E CDCF FC4B ABA3 2A1A FAEC https://keybase.io/kshepherd On 29 May 2015 at 16:26, Fitchett, Deborah <deborah.fitch...@lincoln.ac.nz<mailto:deborah.fitch...@lincoln.ac.nz>> wrote: Hi there, We’re on DSpace 3.1 and have just switched over to XMLUI. All we know about DSpace we learned from a 2-day workshop and from working through this XLMUI switch, so please bear with us! Currently we have our XOAI feed set up such that it effectively excludes items from two collections: • “Restricted thesis” collection – theses accessible to our own staff and students only. Items have anonymous READ access, but bitstreams are restricted • “Symplectic” collection – intended as a dark archive of copyrighted material kept for internal purposes. Items and bitstreams are all access-restricted. Firstly, we’ve noticed that while this excludes the items from ListIdentifiers and from ListRecords, the metadata for the “Symplectic” collection is still returned in a GetRecord request if you know the handle. (It isn’t returned in the “Restricted thesis” collection.) This is unexpected and very undesirable for us. Can anyone explain it and/or suggest a way to ensure this doesn’t happen? Secondly, we’d like to move to a situation where: • “Restricted thesis” collection is included (at least in one OAI context – this is so we can harvest it into our library’s discovery layer Primo) • “Symplectic” collection remains excluded from OAI Our current settings: 1. We have dspace.cfg to: harvest.include.restricted.oai = false We understood this to mean that only items with anon READ would be exposed to OAI, however it’s since been suggested that this has nothing to do with XOAI, and indeed our experience showed no result to switching this between true and false. 2. We also have <Filter refid=”bitstreamaccessFilter” /> on our OAI context (Before I continue, whenever we make a change in dspace.cfg or xoai.xml and related files we are then stopping/restarting dspace and then running the “dspace oai import -cv” job. If this isn’t sufficient to pick up our changes and refresh the cache and we should be doing something else, or doing things in a different order, please let us know, as this may explain everything!) After some changing settings back and forth in our Dev environment we’ve managed to get a situation precisely opposite to what we want, but not in a way that’s very enlightening, and also with no idea how we got there as our settings are now the same as when we started but our results aren’t. Here is a chart I’ve entitled “OAI matrix of woes”: Context Verb # Items in: All # Items in: Restricted theses (items have anon READ; bundles/bitstreams are restricted) # Items in: Symplectic (items and bundles/bitstreams are restricted) request (bitstreamaccessFilter) ListIdentifiers 3705 1888 105 ListRecords 1863 8 105 GetRecord returns record returns record if bundle TEXT anon READ (and other bundles/bitstreams restricted); otherwise does not exist does not exist custom (no filter) ListIdentifiers 5712 1888 106 ListRecords 5712 1888 105 GetRecord returns record returns record returns record How can we: • (As in our first question) Make ListIdentifiers, ListRecords, and GetRecord all behave the same in terms of access? • Include items in the feed where the item has anonymous READ, even if some or all of the bundles and bitstreams are restricted? • While excluding items where the item is restricted? Thanks very much for any clues! Nāku noa, nā Deborah Fitchett Senior Advisor, Digital Access Library, Teaching and Learning p +64 3 423 0358<tel:%2B64%203%20423%200358> e deborah.fitch...@lincoln.ac.nz<mailto:deborah.fitch...@lincoln.ac.nz> | w library.lincoln.ac.nz<http://library.lincoln.ac.nz/> Lincoln University, Te Whare Wānaka o Aoraki New Zealand's specialist land-based university ________________________________ P Please consider the environment before you print this email. "The contents of this e-mail (including any attachments) may be confidential and/or subject to copyright. Any unauthorised use, distribution, or copying of the contents is expressly prohibited. If you have received this e-mail in error, please advise the sender by return e-mail or telephone and then delete this e-mail together with all attachments from your system." ------------------------------------------------------------------------------ _______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net<mailto:DSpace-tech@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
------------------------------------------------------------------------------
_______________________________________________ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
