http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662
--- Comment #60 from Andreas Hedström Mace <[email protected]> --- (In reply to David Cook from comment #52) > (In reply to Leif Andersson from comment #51) > > (In reply to David Cook from comment #50) > > > Honestly though, I would ask that people think further about the frequency > > > of harvests. Is every 2-3 seconds really necessary? Do we really need it > > > to > > > be able to perform that quickly? > > > > > > > Well, the use case envisioned by Stockholm UL would in practice ideally > > involve fetching ONE record every 10 minutes or so! > > The cataloger will be creating/modifying a bib record and a mfhd record in > > our union catalog. > > Next, the cataloger will turn to our local catalog, Koha, expecting to find > > this record already imported. > > If there is a way for the harvester to decide which ONE record to > > get...maybe even with some intervention by the cataloger...? > > So when this ONE record is asked for, we want it to be a quick process > > getting it from the source and into Koha. > > To confuse things a little, I will have to contradict my colleague at Stockholm Univ. Library a little buy saying that I don't see why we would want to replicate functionality already offered by LIBRIS (the Swedish union catalogue, for those lurking this thread) - where you can download records individually and then run batch exports at night - rather than creating something better/faster. For me it is preferable to have the catalogue as up-to-date as possible, since LIBRIS will be the "master" (or source of truth as David calls it) for our data. I would rather want the harvester to run every ten seconds or so (or however fast we can get it), to get all updates made to "our" records. The only drawback I can see from such an approach would be an increased load on the servers, which is not a trivial thing of course, but something that should be manageable. (LIBRIS might have a bigger problem if a lot of Swedish libraries start using OAI-PMH harvesting with this approach, but they have themselves recommended this use and will have to handle it accordingly.) Also, I would prefer if the process of harvesting records can be as automated as possible, not involving any extra steps on the catalogers part. We want to make their cataloging easier - not more complex! > So you're saying that the union catalogue will only have updates about once > every 10 minutes? Or that the cataloguer will only be accessing a record in > the union catalogue and Koha once every 10 minutes? Records that we handle, i.e. adding/updating either the bibliographic record or the holdings record (or both), is probably only done around every 5-10 min as Leif say. But changes to the bibliographic records, for which we have holdings attached, made by other Swedish libraries is probably much more often. I will try to look at this in the upcoming days, manually harvesting at close intervals (I'm thinking about both trying a 3 second and 10 second approach) to see how many records are downloaded with each harvest. > I am including a mechanism for fetching "one" record, so long as the user > knows the OAI-PMH identifier they're after. They'll be able to add a task > for that. Perhaps a future development could be done to provide an interface > in the cataloguing module for adding/updating a single record. In place of > that interface, they'll be able to add a task in the same area as the other > tasks in order to get the one record... > > > Then nightly more massive harvests could be performed to catch up with other > > modifications to the union catalog. > > > > Those nightly harvests would certainly be possible with the current design. As I mentioned above, ideally I would want the harvester to run repeatedly, on short intervals. But other libraries whom are interested in using the harvester might have other ideas on which set-up would be best for them. So having the flexibility of running either way (individual harvest+night more massive harvestes or repeated harvest every 10 seconds or so) would be wonderful! I think Davids idea for a plug-in approach, together with the harvest tasks, will work well here! -- You are receiving this mail because: You are watching all bug changes. _______________________________________________ Koha-bugs mailing list [email protected] http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
