Re: [CODE4LIB] New OAI-PMH harvester project

[Demian Katz]

Thanks, Roy! Glad to hear you found this useful. I've also revised the README 
slightly to indicate the possibility of composer.phar-based installation.

In a different conversation today, Chad Nelson brought up ResourceSync, which 
in some ways may prove to be OAI-PMH's successor (though it isn't exactly the 
same and had a broader range of use cases). I should note that I've named this 
project vufindharvest rather than vufindoaipmh because I envision the 
possibility of supporting other protocols in the future, and I hope that some 
logic and patterns can be shared across multiple protocol implementations. If 
anybody has use cases, code or ideas, I'm happy to discuss future possibilities!

- Demian

From: Code for Libraries [CODE4LIB@LISTSERV.ND.EDU] on behalf of Roy Tennant 
[roytenn...@gmail.com]
Sent: Thursday, July 14, 2016 6:42 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] New OAI-PMH harvester project

This is really cool, Demian, thanks a lot! I found it extremely easy to set
up and get going, but with one minor change to the instructions. On Ubuntu
Unix I needed to call Composer like so: php composer.phar install . But
that wasn't much of a problem, as when I installed composer it told me how
to invoke it.

When I fired it up and pointed it at my photos server it downloaded over
13,000 OAI records in what seemed like a flash. I didn't realize my server
(written from scratch in Perl) would respond so quickly (I set my
resumption tokens at 100 recs).

This is now my go-to OAI harvester as it is so easy to use and works (at
least so far) without any trouble at all. Just what I like in a basic tool
like this. Thanks for spinning this off from the VuFind code.
Roy

On Thu, Jul 14, 2016 at 10:05 AM, Demian Katz <demian.k...@villanova.edu>
wrote:

> Hello,
>
> VuFind has been bundled with an OAI-PMH harvest tool for a long time. It
> occurred to me that this tool might be of more general interest, so I have
> broken it out into a stand-alone project here:
>
> https://github.com/vufind-org/vufindharvest
>
> This provides a simple command-line interface for downloading records from
> an OAI-PMH repository into a directory, including support for incremental
> updates. All of the details are documented in the README on GitHub.
>
> The project can also be pulled in as a Composer dependency so that
> harvests can be controlled by other applications; this is how VuFind will
> continue to utilize the functionality in future releases.
>
> If anyone has questions about this project, please let me know!
>
> - Demian
>

Re: [CODE4LIB] New OAI-PMH harvester project

[Roy Tennant]

This is really cool, Demian, thanks a lot! I found it extremely easy to set
up and get going, but with one minor change to the instructions. On Ubuntu
Unix I needed to call Composer like so: php composer.phar install . But
that wasn't much of a problem, as when I installed composer it told me how
to invoke it.

When I fired it up and pointed it at my photos server it downloaded over
13,000 OAI records in what seemed like a flash. I didn't realize my server
(written from scratch in Perl) would respond so quickly (I set my
resumption tokens at 100 recs).

This is now my go-to OAI harvester as it is so easy to use and works (at
least so far) without any trouble at all. Just what I like in a basic tool
like this. Thanks for spinning this off from the VuFind code.
Roy

On Thu, Jul 14, 2016 at 10:05 AM, Demian Katz 
wrote:

> Hello,
>
> VuFind has been bundled with an OAI-PMH harvest tool for a long time. It
> occurred to me that this tool might be of more general interest, so I have
> broken it out into a stand-alone project here:
>
> https://github.com/vufind-org/vufindharvest
>
> This provides a simple command-line interface for downloading records from
> an OAI-PMH repository into a directory, including support for incremental
> updates. All of the details are documented in the README on GitHub.
>
> The project can also be pulled in as a Composer dependency so that
> harvests can be controlled by other applications; this is how VuFind will
> continue to utilize the functionality in future releases.
>
> If anyone has questions about this project, please let me know!
>
> - Demian
>

[CODE4LIB] New OAI-PMH harvester project

[Demian Katz]

Hello,

VuFind has been bundled with an OAI-PMH harvest tool for a long time. It 
occurred to me that this tool might be of more general interest, so I have 
broken it out into a stand-alone project here:

https://github.com/vufind-org/vufindharvest

This provides a simple command-line interface for downloading records from an 
OAI-PMH repository into a directory, including support for incremental updates. 
All of the details are documented in the README on GitHub.

The project can also be pulled in as a Composer dependency so that harvests can 
be controlled by other applications; this is how VuFind will continue to 
utilize the functionality in future releases.

If anyone has questions about this project, please let me know!

- Demian