This is an automated notification sent by LCG Savannah.
It relates to:
task #11282, project CDS Invenio
==============================================================================
LATEST MODIFICATIONS of task #11282:
==============================================================================
Follow-up Comment #1, task #11282 (project cdsware):
(Here are quick notes on some possible natural follow-ups.)
1) Metadata batch upload - web interface
Described already.
***
2) Document batch upload - web interface
Joce's use case:
/afs/cern.ch/user/j/joce/public/foo/repno1.pdf
/afs/cern.ch/user/j/joce/public/foo/repno2.pdf
/afs/cern.ch/user/j/joce/public/foo/repno3.pdf
So you would take all files from a dir, do the matching via repno
to find recIDs, try to see if this file is already
there (e.g. md5sums), and attach-or-revise as specified if not.
The UI would look like:
Input dir: [/afs/cern.ch/....]
Filename matching: [report number / record ID / barcode / etc]
Upload mode: ( ) append (*) revise ( ) replace
[UPLOAD]
Another use case is scanning machine pushing CM-P files into a
dir the daemon will progressively look into.
***
3) Metadata batch upload - CLI daemon
Create new bibsched task daemon named ``batchupload''(?)
(see bibtaskex, fibonacci) that would look into directories like:
/opt/cds-invenio/var/batchupload/metadata/insert/foo.xml
/opt/cds-invenio/var/batchupload/metadata/append/
/opt/cds-invenio/var/batchupload/metadata/correct/
/opt/cds-invenio/var/batchupload/metadata/replace/
and would call bibupload and the moved the files to DONE folders
like:
/opt/cds-invenio/var/batchupload/metadata/insert/DONE/foo.xml_20091016112356
(YYYYMMDDhhmmss)
(or use task_low_level_submission() and get job ID)
***
4) Document batch upload - CLI daemon
Similarly to 3, but for documents. E.g.
/opt/cds-invenio/var/batchupload/document/append
/opt/cds-invenio/var/batchupload/document/revise
/opt/cds-invenio/var/batchupload/document/replace
Doc is called e.g. CERN-THESIS-2009-001.pdf, you do the matching
by report number.
Doc is called e.g. CM-P234234234.pdf,q you do the matching by
barcode.
Could be configurable by the user, or simply we match by order of
precedence.
==============================================================================
OVERVIEW of task #11282:
==============================================================================
URL:
<http://savannah.cern.ch/task/?11282>
Summary: webupload: simple human-friendly web interface
Project: CDS Invenio
Submitted by: simko
Submitted on: 2009-09-21 14:11
Should Start On: 2009-09-21 00:00
Should be Finished on: 2009-09-21 00:00
Category: BibUpload
Priority: 5 - Normal
Status: None
Privacy: Public
Percent Complete: 0%
Assigned to: jmartinm
Open/Closed: Open
Discussion Lock: Any
Effort: 0.00
_______________________________________________________
Currently, webupload at CERN is used by robots only, via curl like
call. It would be desirable to allow cataloguers to submit files via
a simple web UI as well. (The robots interface should be preserved.)
This task comprises roughly the following:
* Create simple UI for uploading MARC files, under /webupload. We
can treat filename as part of jobname, so if Erika submits file
called foo today, internally we will work with the job name
webupload_erika_20090921_foo. (after first checking its
existence)
* The identity check should be done not via IP (as for robots), but
via logged-in user identity. We should introduce a new RBAC
action `webupload' with optional argument `collection', and use
WebAccess to manage which users have rights to upload to which
collections.
* Create a new table named like hstWEBUPLOAD or the like, where the
webupload history would be stored (instead of the log file in use
now), so that we can offer to cataloguers a possibility to
consult their past webupload job history at a URL like
/webupload/history.
* For every webuploaded file, we can set the bibupload process name
to be the webuploaded filename, so that in the history display we
can also look into the bibupload task history table and see what
happened with the file and whether it was really bibuploaded at
the end or not. (This task name business may be similar here and
in other parts of bibupload input channels, so maybe we should
treat this ``job name'' bit in a more generic way later.)
_______________________________________________________
Follow-up Comments:
-------------------------------------------------------
Date: 2009-10-16 12:02 By: Tibor Simko <simko>
(Here are quick notes on some possible natural follow-ups.)
1) Metadata batch upload - web interface
Described already.
***
2) Document batch upload - web interface
Joce's use case:
/afs/cern.ch/user/j/joce/public/foo/repno1.pdf
/afs/cern.ch/user/j/joce/public/foo/repno2.pdf
/afs/cern.ch/user/j/joce/public/foo/repno3.pdf
So you would take all files from a dir, do the matching via repno
to find recIDs, try to see if this file is already
there (e.g. md5sums), and attach-or-revise as specified if not.
The UI would look like:
Input dir: [/afs/cern.ch/....]
Filename matching: [report number / record ID / barcode / etc]
Upload mode: ( ) append (*) revise ( ) replace
[UPLOAD]
Another use case is scanning machine pushing CM-P files into a
dir the daemon will progressively look into.
***
3) Metadata batch upload - CLI daemon
Create new bibsched task daemon named ``batchupload''(?)
(see bibtaskex, fibonacci) that would look into directories like:
/opt/cds-invenio/var/batchupload/metadata/insert/foo.xml
/opt/cds-invenio/var/batchupload/metadata/append/
/opt/cds-invenio/var/batchupload/metadata/correct/
/opt/cds-invenio/var/batchupload/metadata/replace/
and would call bibupload and the moved the files to DONE folders
like:
/opt/cds-invenio/var/batchupload/metadata/insert/DONE/foo.xml_20091016112356
(YYYYMMDDhhmmss)
(or use task_low_level_submission() and get job ID)
***
4) Document batch upload - CLI daemon
Similarly to 3, but for documents. E.g.
/opt/cds-invenio/var/batchupload/document/append
/opt/cds-invenio/var/batchupload/document/revise
/opt/cds-invenio/var/batchupload/document/replace
Doc is called e.g. CERN-THESIS-2009-001.pdf, you do the matching
by report number.
Doc is called e.g. CM-P234234234.pdf,q you do the matching by
barcode.
Could be configurable by the user, or simply we match by order of
precedence.
_______________________________________________________
Carbon-Copy List:
CC Address | Comment
------------------------------------+-----------------------------
1576 | -SUB-
==============================================================================
This item URL is:
<http://savannah.cern.ch/task/?11282>
_______________________________________________
Message sent via/by LCG Savannah
http://savannah.cern.ch/
