Re: TypeError when calling oaiharvest from CLI

Fri, 07 Aug 2015 06:27:01 -0700 

Hello!
First things first: changing the workflow to 'oaiharvest_harvest_repositories' doesn't change the result! 
So here is now the log:

Konsole output
2015-08-07 15:18:05,935 ERROR: /js/jquery.min.js [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 137, in page_not_found 
   response = legacy_application(request.environ, g.start_response)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 485, in application 
   raise NotFound()
NotFound: 404: Not Found
2015-08-07 15:18:05,987 ERROR: /js/ui.core.js [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 137, in page_not_found 
   response = legacy_application(request.environ, g.start_response)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 485, in application 
   raise NotFound()
NotFound: 404: Not Found
2015-08-07 15:18:31,388 ERROR: /js/jquery.min.js [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 137, in page_not_found 
   response = legacy_application(request.environ, g.start_response)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 485, in application 
   raise NotFound()
NotFound: 404: Not Found
2015-08-07 15:18:31,444 ERROR: /js/ui.core.js [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py:142] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 137, in page_not_found 
   response = legacy_application(request.environ, g.start_response)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 485, in application 
   raise NotFound()
NotFound: 404: Not Found
2015-08-07 15:19:18,285 ERROR: [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/logging/wrappers.py:310] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 119, in __call__ 
   response = self.app.full_dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request 
   rv = self.handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", line 270, in error_router 
   return original_handler(e)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", line 133, in handle_user_exception 
   return super(Flask, self).handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception 
   reraise(exc_type, exc_value, tb)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request 
   rv = self.dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request 
   return self.view_functions[rule.endpoint](**req.view_args)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 164, in web_admin 
   handler=legacy_publisher)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 472, in application 
   handler(req)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 162, in legacy_publisher 
   possible_handler)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 630, in mp_legacy_publisher return _check_result(req, module_globals[possible_handler](req, **form)) 
 File "<string>", line 551, in preview_original_xml
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1146, in perform_request_preview_original_xml 
   from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id
ImportError: cannot import name get_oai_src_by_id
2015-08-07 15:19:18,326 ERROR: Exception on /admin/oaiharvest/oaiharvestadmin.py/preview_original_xml [GET] [in /home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py:1423] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 119, in __call__ 
   response = self.app.full_dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request 
   rv = self.handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", line 270, in error_router 
   return original_handler(e)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", line 133, in handle_user_exception 
   return super(Flask, self).handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception 
   reraise(exc_type, exc_value, tb)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request 
   rv = self.dispatch_request()

Konsole output
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request 
   return self.view_functions[rule.endpoint](**req.view_args)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 164, in web_admin 
   handler=legacy_publisher)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 472, in application 
   handler(req)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 162, in legacy_publisher 
   possible_handler)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 630, in mp_legacy_publisher return _check_result(req, module_globals[possible_handler](req, **form)) 
 File "<string>", line 551, in preview_original_xml
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1146, in perform_request_preview_original_xml 
   from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id
ImportError: cannot import name get_oai_src_by_id
2015-08-07 15:19:18,344 ERROR: [in /home/system/.virtualenvs/invenio/src/invenio/invenio/ext/logging/wrappers.py:310] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 119, in __call__ 
   response = self.app.full_dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request 
   rv = self.handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", line 270, in error_router 
   return original_handler(e)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", line 133, in handle_user_exception 
   return super(Flask, self).handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception 
   reraise(exc_type, exc_value, tb)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request 
   rv = self.dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request 
   return self.view_functions[rule.endpoint](**req.view_args)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 164, in web_admin 
   handler=legacy_publisher)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 472, in application 
   handler(req)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 162, in legacy_publisher 
   possible_handler)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 630, in mp_legacy_publisher return _check_result(req, module_globals[possible_handler](req, **form)) 
 File "<string>", line 574, in preview_harvested_xml
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1154, in perform_request_preview_harvested_xml return harvest_postprocess_record(oai_src_id, record_id, treat_new=True) File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1075, in harvest_postprocess_record 
   from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id
ImportError: cannot import name get_oai_src_by_id
2015-08-07 15:19:18,385 ERROR: Exception on /admin/oaiharvest/oaiharvestadmin.py/preview_harvested_xml [GET] [in /home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py:1423] 
Traceback (most recent call last):
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 119, in __call__ 
   response = self.app.full_dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request 
   rv = self.handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask_restful/__init__.py", line 270, in error_router 
   return original_handler(e)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/base/wrappers.py", line 133, in handle_user_exception 
   return super(Flask, self).handle_user_exception(e)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception 
   reraise(exc_type, exc_value, tb)
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request 
   rv = self.dispatch_request()
File "/home/system/.virtualenvs/invenio/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request 
   return self.view_functions[rule.endpoint](**req.view_args)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 164, in web_admin 
   handler=legacy_publisher)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 472, in application 
   handler(req)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/ext/legacy/__init__.py", line 162, in legacy_publisher 
   possible_handler)
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/wsgi/__init__.py", line 630, in mp_legacy_publisher return _check_result(req, module_globals[possible_handler](req, **form)) 
 File "<string>", line 574, in preview_harvested_xml
File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1154, in perform_request_preview_harvested_xml 

Konsole output
return harvest_postprocess_record(oai_src_id, record_id, treat_new=True) File "/home/system/.virtualenvs/invenio/src/invenio/invenio/legacy/oaiharvest/admin.py", line 1075, in harvest_postprocess_record 
   from invenio.legacy.oaiharvest.dblayer import get_oai_src_by_id
ImportError: cannot import name get_oai_src_by_id


Best regards,
Werner





Am 07.08.2015 um 15:12 schrieb Jan Åge Lavik:

Hello Werner,
So the Internal Server Error happens when you save the repository in the OAI Harvest Admin UI? Can you dig our the logs from the server which has more details? Say, check in cdvirtualenv var/log/invenio.base.log file.
Regarding the workflow to choose, it is a bit confusing right now due to the state of the migration from legacy to new modules, but you can try selecting the workflows: 'oaiharvest_harvest_repositories' (which reads post-process arguments) or 'oaiharvest_repository_with_approval' (which uses an approval step in the "Holding Pen" available under /admin/holdingpen). 

Cheers,
Jan

---
Jan Age Lavik
System Developer
INSPIRE-HEP <http://inspirehep.net>

Github: @jalavik <https://github.com/jalavik>
Work phone: +41 22 76 78682
On Fri, Aug 7, 2015 at 2:40 PM, Werner Greßhoff <werner.gressh...@uni-muenster.de <mailto:werner.gressh...@uni-muenster.de>> wrote: 

    Hi!

    I tried the first URL leading to the same Internal Server Error as
    with our own repository. In the OAI Harvest Admin UI I entered a
    name (Desy), the URL (https://bib-pubdb1.desy.de/oai2d) and chose
    marcxml as Metadata-Prefix and the insert Workflow (which I replaced
    with marcxml_workflow after first Internal Server Error). The rest
    of the page (deprecated postprocessing options) I left blank. Then
    I searched for an identifier and tested the harvesting process.

    Where am I going wrong?

    Best regards,
    Werner

    Am 07.08.2015 um 12:28 schrieb Wagner, Alexander:

        Hi!

            The URL I tried was
            http://repositorium-dev.uni-muenster.de/oai/miami.
            This is our staging system, which is so far the only one
            to produce
            marcxml output.


        If it's only about to get Marc records, you may try one of
        the join2 repos, e.g.

        https://bib-pubdb1.desy.de/oai2d
        https://impulse.mlz-garching.de/oai2d
        https://juser.fz-juelich.de/oai2d
        https://publications.rwth-aachen.de/oai2d
        https://repository.gsi.de/oai2d

        However, I'd suggest to try the openaire or vdb sets. If you
        harvest blindly you'll get our authority records. They are
        nice as well, but probably not what you're striving for.
-- Mit freundlichen Grüßen 

    Werner Greßhoff
    Dezernat 2 - Digitale Dienste
    Universitäts- und Landesbibliothek Münster





--
Mit freundlichen Grüßen

Werner Greßhoff
Dezernat 2 - Digitale Dienste
Universitäts- und Landesbibliothek Münster

<<attachment: Werner_Gresshoff.vcf>>

Reply via email to