Kristian Rickert created OPENNLP-1872:
-----------------------------------------
Summary: Discussion: provenance-tagged enrichment metadata across
Tika, OpenNLP, and Gravitino
Key: OPENNLP-1872
URL: https://issues.apache.org/jira/browse/OPENNLP-1872
Project: OpenNLP
Issue Type: Brainstorming
Reporter: Kristian Rickert
There is a natural three-role split across the Apache family that nothing
currently connects: Tika is the metadata parser (it surfaces what a document
already carries: properties, EXIF, MIME types), OpenNLP is the metadata
producer (it derives metadata that never existed in the file: entities,
sentences, offsets, and soon coordinates and annotations), and Apache Gravitino
(TLP since 2025) is the metadata catalog (filesets, tables, models, lineage).
Today each half speaks its own shape, nothing feeds Gravitino document-level
enrichment, and most of this metadata dies inside whatever index a pipeline
writes to.
I would like to explore a small, consistent metadata record for that chain: for
each field, what produced it (component, model, version), from what source, and
with what confidence. OPENNLP-1852 is already introducing per-value provenance
at the data-file level (source; standard; or an explicit "project choice"
marker); this would lift the same discipline to component output, in a shape a
Gravitino catalog entry can consume. Parse, derive, catalog is itself a lineage
chain, so the mapping is direct.
Nothing here changes existing APIs; it would be an optional, additive record.
Looking for opinions before any code: is there appetite, does prior art exist
that I have missed, and if the idea has legs, does it belong in one project or
somewhere neutral?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
