Hi, I see the problem now. Unfortunately, with the changes in 2.0 not all parts of the legacy admin UI was fixed. The in browser display of harvests is one of those. Instead, you can use the `oaiharvest` command-line tool to schedule a harvest in `bibsched`, our legacy task scheduler.
E.g. $ oaiharvest -r desy —dates=yyyy-mm-dd:yyyy-mm-dd # to avoid harvesting the entire repo Then start bibched: $ bibsched start You can monitor the process by accessing bibsched: $ bibsched Press ‘h’ for a list of help commands. To be able to fully run the harvest you also need to make sure a Celery worker instance is running as described here: http://invenio.readthedocs.org/en/latest/getting-started/first-steps.html?highlight=celery#development <http://invenio.readthedocs.org/en/latest/getting-started/first-steps.html?highlight=celery#development> This will execute a process which will process the harvested data and then finally create bibupload jobs that you can run in bibsched (if you tick off upload step in post-process). Word of warning, though, this version of Invenio is a developer preview release and not fully stable. We are working hard towards the next release to address these and other issues. Hope this helps. Cheers, Jan > On 07 Aug 2015, at 15:25, Werner Greßhoff <werner.gressh...@uni-muenster.de> > wrote: > > Hello! > > First things first: changing the workflow to > 'oaiharvest_harvest_repositories' doesn't change the result! > So here is now the log: > > 2015-08-07 15:18:05,935 ERROR: /js/jquery.min.js [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 137, in page_not_found > response = legacy_application(request.environ, g.start_response) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 485, in application > raise NotFound() > NotFound: 404: Not Found > 2015-08-07 15:18:05,987 ERROR: /js/ui.core.js [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 137, in page_not_found > response = legacy_application(request.environ, g.start_response) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 485, in application > raise NotFound() > NotFound: 404: Not Found > 2015-08-07 15:18:31,388 ERROR: /js/jquery.min.js [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 137, in page_not_found > response = legacy_application(request.environ, g.start_response) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 485, in application > raise NotFound() > NotFound: 404: Not Found > 2015-08-07 15:18:31,444 ERROR: /js/ui.core.js [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 137, in page_not_found > response = legacy_application(request.environ, g.start_response) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 485, in application > raise NotFound() > NotFound: 404: Not Found > 2015-08-07 15:19:18,285 ERROR: [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/logging/wrappers.py:310] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 119, in __call__ > response = self.app.full_dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", > line 270, in error_router > return original_handler(e) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", > line 133, in handle_user_exception > return super(Flask, self).handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1475, in full_dispatch_request > rv = self.dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 164, in web_admin > handler=legacy_publisher) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 472, in application > handler(req) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 162, in legacy_publisher > possible_handler) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 630, in mp_legacy_publisher > return _check_result(req, module_globals[possible_handler](req, **form)) > File "<string>", line 551, in preview_original_xml > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1146, in perform_request_preview_original_xml > from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id > ImportError: cannot import name get_oai_src_by_id > 2015-08-07 15:19:18,326 ERROR: Exception on > /admin/oaiharvest/oaiharvestadmin.py/preview_original_xml [GET] [in > /home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py:1423] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 119, in __call__ > response = self.app.full_dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", > line 270, in error_router > return original_handler(e) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", > line 133, in handle_user_exception > return super(Flask, self).handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1475, in full_dispatch_request > rv = self.dispatch_request() > > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 164, in web_admin > handler=legacy_publisher) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 472, in application > handler(req) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 162, in legacy_publisher > possible_handler) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 630, in mp_legacy_publisher > return _check_result(req, module_globals[possible_handler](req, **form)) > File "<string>", line 551, in preview_original_xml > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1146, in perform_request_preview_original_xml > from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id > ImportError: cannot import name get_oai_src_by_id > 2015-08-07 15:19:18,344 ERROR: [in > /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/logging/wrappers.py:310] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 119, in __call__ > response = self.app.full_dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", > line 270, in error_router > return original_handler(e) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", > line 133, in handle_user_exception > return super(Flask, self).handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1475, in full_dispatch_request > rv = self.dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 164, in web_admin > handler=legacy_publisher) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 472, in application > handler(req) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 162, in legacy_publisher > possible_handler) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 630, in mp_legacy_publisher > return _check_result(req, module_globals[possible_handler](req, **form)) > File "<string>", line 574, in preview_harvested_xml > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1154, in perform_request_preview_harvested_xml > return harvest_postprocess_record(oai_src_id, record_id, treat_new=True) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1075, in harvest_postprocess_record > from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id > ImportError: cannot import name get_oai_src_by_id > 2015-08-07 15:19:18,385 ERROR: Exception on > /admin/oaiharvest/oaiharvestadmin.py/preview_harvested_xml [GET] [in > /home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py:1423] > Traceback (most recent call last): > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 119, in __call__ > response = self.app.full_dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", > line 270, in error_router > return original_handler(e) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", > line 133, in handle_user_exception > return super(Flask, self).handle_user_exception(e) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1475, in full_dispatch_request > rv = self.dispatch_request() > File > "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", > line 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 164, in web_admin > handler=legacy_publisher) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 472, in application > handler(req) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", > line 162, in legacy_publisher > possible_handler) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", > line 630, in mp_legacy_publisher > return _check_result(req, module_globals[possible_handler](req, **form)) > File "<string>", line 574, in preview_harvested_xml > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1154, in perform_request_preview_harvested_xml > > return harvest_postprocess_record(oai_src_id, record_id, treat_new=True) > File > "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", > line 1075, in harvest_postprocess_record > from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id > ImportError: cannot import name get_oai_src_by_id > > > Best regards, > Werner > > > > > > Am 07.08.2015 um 15:12 schrieb Jan Åge Lavik: >> Hello Werner, >> >> So the Internal Server Error happens when you save the repository in the OAI >> Harvest Admin UI? Can you dig our the logs from the server which has more >> details? Say, check in cdvirtualenv var/log/invenio.base.log file. >> >> Regarding the workflow to choose, it is a bit confusing right now due to the >> state of the migration from legacy to new modules, but you can try selecting >> the workflows: 'oaiharvest_harvest_repositories' (which reads post-process >> arguments) or 'oaiharvest_repository_with_approval' (which uses an approval >> step in the "Holding Pen" available under /admin/holdingpen). >> >> Cheers, >> Jan >> >> --- >> Jan Age Lavik >> System Developer >> INSPIRE-HEP <http://inspirehep.net <http://inspirehep.net/>> >> >> Github: @jalavik <https://github.com/jalavik> >> Work phone: +41 22 76 78682 >> >> On Fri, Aug 7, 2015 at 2:40 PM, Werner Greßhoff >> <werner.gressh...@uni-muenster.de <mailto:werner.gressh...@uni-muenster.de>> >> wrote: >> Hi! >> >> I tried the first URL leading to the same Internal Server Error as >> with our own repository. In the OAI Harvest Admin UI I entered a >> name (Desy), the URL (https://bib-pubdb1.desy.de/oai2d >> <https://bib-pubdb1.desy.de/oai2d>) and chose >> marcxml as Metadata-Prefix and the insert Workflow (which I replaced >> with marcxml_workflow after first Internal Server Error). The rest >> of the page (deprecated postprocessing options) I left blank. Then >> I searched for an identifier and tested the harvesting process. >> >> Where am I going wrong? >> >> Best regards, >> Werner >> >> Am 07.08.2015 um 12:28 schrieb Wagner, Alexander: >> Hi! >> >> The URL I tried was http://repositorium-dev.uni-muenster.de/oai/miami >> <http://repositorium-dev.uni-muenster.de/oai/miami>. >> This is our staging system, which is so far the only one to produce >> marcxml output. >> >> If it's only about to get Marc records, you may try one of >> the join2 repos, e.g. >> >> https://bib-pubdb1.desy.de/oai2d <https://bib-pubdb1.desy.de/oai2d> >> https://impulse.mlz-garching.de/oai2d <https://impulse.mlz-garching.de/oai2d> >> https://juser.fz-juelich.de/oai2d <https://juser.fz-juelich.de/oai2d> >> https://publications.rwth-aachen.de/oai2d >> <https://publications.rwth-aachen.de/oai2d> >> https://repository.gsi.de/oai2d <https://repository.gsi.de/oai2d> >> >> However, I'd suggest to try the openaire or vdb sets. If you >> harvest blindly you'll get our authority records. They are >> nice as well, but probably not what you're striving for. >> >> >> >> -- >> Mit freundlichen Grüßen >> >> Werner Greßhoff >> Dezernat 2 - Digitale Dienste >> Universitäts- und Landesbibliothek Münster >> > > > -- > Mit freundlichen Grüßen > > Werner Greßhoff > Dezernat 2 - Digitale Dienste > Universitäts- und Landesbibliothek Münster > <Werner_Gresshoff.vcf>
