[
https://issues.apache.org/jira/browse/TIKA-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133762#comment-18133762
]
ASF GitHub Bot commented on TIKA-4766:
--------------------------------------
krickert commented on PR #3451:
URL: https://github.com/apache/tika/pull/3451#issuecomment-6477630494
So the plugin -
Protobufs are already strongly typed. Just write a java mapper class to
make the object go from one type of object to antoher.
Using SPI, it will bring the java protobuf class into the classpath. If it
is an Any object, you can just proxy it too if you want.
You really should try to have everything strongly typed and avoid converting
it from type->string->type. You can just do type->type with this solution.
```java
public class WarcRecordParser implements MessageParser<WarcRecord> {
static final Property STATUS =
Property.externalInteger("warc:http-status");
public Class<WarcRecord> messageType() { return WarcRecord.class; }
public List<Property> properties() { return List.of(STATUS); }
public List<Metadata> parse(WarcRecord record) {
Metadata m = new Metadata();
m.set(TikaCoreProperties.SOURCE_PATH, record.getTargetUri());
m.set(STATUS, record.getHttpStatus());
m.set(TikaCoreProperties.TIKA_CONTENT, record.getText());
return List.of(m);
}
}
```
and also to serialize we'd just call pack():
```java
ParseMessageResponse r = stub.parseMessage(
ParseMessageRequest.newBuilder().setMessage(Any.pack(record)).build());
```
The idea is that you don't convert it to a string. Converting some types
from data->string->data can cause serialization headaches.
> Replace tika-grpc fields map with a typed Document parse contract
> -----------------------------------------------------------------
>
> Key: TIKA-4766
> URL: https://issues.apache.org/jira/browse/TIKA-4766
> Project: Tika
> Issue Type: New Feature
> Components: tika-pipes
> Affects Versions: 4.0.0
> Reporter: Kristian Rickert
> Priority: Major
> Labels: grpc, pipes, protobuf
> Attachments: 20261028-grpc-v2-design.md
>
>
> Replace the flat {{FetchAndParseReply.fields}} map ({{map<string,string>}})
> with a typed parse result.
> Approach (reshaped from the original {{ParseResponse}} design after review in
> [PR #2916|https://github.com/apache/tika/pull/2916]): a single small
> {{Document}} proto (~200 lines) rather than per-format metadata messages.
> * *Content*: a structured markdown block tree -- headings, paragraphs, lists,
> tables, code blocks, inline runs (CommonMark + GFM) -- plus the rendered
> markdown string {{ToMarkdownContentHandler}} already produces (TIKA-4730).
> * *Typed common metadata*: title, authors, keywords, languages,
> created/modified as {{Timestamp}}s, page/word/character counts, dimensions,
> rights.
> * *Lossless tagged tail*: every remaining metadata key,
> multivalue-preserving, typed only where Tika's own {{Property}} declares a
> type, string otherwise -- never guessed.
> * *Embedded documents* recurse as fully typed child {{Document}}s.
> * Format specifics live in per-parser {{DocumentTransformer}} code
> ({{tika-grpc-mapper}}), never in the wire contract, so metadata churn never
> forces a client rebuild.
> Modules: {{tika-grpc-api}} (proto + generated messages + bundled
> {{FileDescriptorSet}}), {{tika-grpc-mapper}}, {{tika-grpc}} integration.
> Breaking change for clients reading {{fields}} (field number reserved).
> PR: [https://github.com/apache/tika/pull/2921] (supersedes
> [#2916|https://github.com/apache/tika/pull/2916])
> Follow-ups will be tracked in separate issues: pluggable external parsers
> (opaque {{Any}} extension results from registered gRPC services), and a
> Markdown input parser.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
