[
https://issues.apache.org/jira/browse/TIKA-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18134100#comment-18134100
]
ASF GitHub Bot commented on TIKA-4766:
--------------------------------------
dpol1 commented on PR #3451:
URL: https://github.com/apache/tika/pull/3451#issuecomment-6492963983
> So, should I update this PR to follow the pattern with oneof() in #2961?
Yes, if it keeps what your Values had: the declared type and the original
strings. #2961 (I worked on it) drops the type when one value fails to convert,
so `extended-properties:TotalTime` arrives as int64 from a .doc and as plain
strings from an .odt (`PT1M7S`). It also turns zone-less dates into UTC.
Keeping `values` and `type` next to the oneof would cover both, at the cost of
sending typed keys twice. wdyt?
ParseBytes fits as the "Later" RPC in your design.
Two things here, from the code (not run):
https://github.com/apache/tika/blob/6480d6f333656e2f693b60d6b27e5600c10a9202/tika-grpc/src/main/java/org/apache/tika/pipes/grpc/FetchParseRuntime.java#L138-L139
- No emit strategy, so with the default DYNAMIC a result of 100 kB or more
goes to the empty `EmitKey` and fails. PASSBACK_ALL, as tika-server does, fixes
v1 and v2alpha1.
- `SKIP` returns no documents when the container throws. /rmeta uses EMIT
and returns what was parsed plus the stack trace. Should v2alpha1 match it?
#2961 and #3449 have the same SKIP.
> Replace tika-grpc fields map with a typed Document parse contract
> -----------------------------------------------------------------
>
> Key: TIKA-4766
> URL: https://issues.apache.org/jira/browse/TIKA-4766
> Project: Tika
> Issue Type: New Feature
> Components: tika-pipes
> Affects Versions: 4.0.0
> Reporter: Kristian Rickert
> Priority: Major
> Labels: grpc, pipes, protobuf
> Attachments: 20261028-grpc-v2-design.md
>
>
> Replace the flat {{FetchAndParseReply.fields}} map ({{map<string,string>}})
> with a typed parse result.
> Approach (reshaped from the original {{ParseResponse}} design after review in
> [PR #2916|https://github.com/apache/tika/pull/2916]): a single small
> {{Document}} proto (~200 lines) rather than per-format metadata messages.
> * *Content*: a structured markdown block tree -- headings, paragraphs, lists,
> tables, code blocks, inline runs (CommonMark + GFM) -- plus the rendered
> markdown string {{ToMarkdownContentHandler}} already produces (TIKA-4730).
> * *Typed common metadata*: title, authors, keywords, languages,
> created/modified as {{Timestamp}}s, page/word/character counts, dimensions,
> rights.
> * *Lossless tagged tail*: every remaining metadata key,
> multivalue-preserving, typed only where Tika's own {{Property}} declares a
> type, string otherwise -- never guessed.
> * *Embedded documents* recurse as fully typed child {{Document}}s.
> * Format specifics live in per-parser {{DocumentTransformer}} code
> ({{tika-grpc-mapper}}), never in the wire contract, so metadata churn never
> forces a client rebuild.
> Modules: {{tika-grpc-api}} (proto + generated messages + bundled
> {{FileDescriptorSet}}), {{tika-grpc-mapper}}, {{tika-grpc}} integration.
> Breaking change for clients reading {{fields}} (field number reserved).
> PR: [https://github.com/apache/tika/pull/2921] (supersedes
> [#2916|https://github.com/apache/tika/pull/2916])
> Follow-ups will be tracked in separate issues: pluggable external parsers
> (opaque {{Any}} extension results from registered gRPC services), and a
> Markdown input parser.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
