[
https://issues.apache.org/jira/browse/NIFI-5538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587971#comment-16587971
]
James Srinivasan commented on NIFI-5538:
----------------------------------------
No worries, glad I hadn't missed an easier way!
My processor uses [https://github.com/os72/protobuf-dynamic] to create a
DynamicMessage from the Field Descriptor Set and the serialised data, then the
canonical json encoding in the newer protobuf library. This means only proto3,
and the subset of proto2 supported by proto3 is supported. The code is
basically:
{{DynamicSchema ds = DynamicSchema.parseFrom(new
FileInputStream(pathToSchema));}}
{{DynamicMessage.Builder builder = ds.newMessageBuilder("Foos");}}
{{DynamicMessage dm = builder.mergeFrom(bytes).build();}}
{{JsonFormat.Printer printer =
JsonFormat.printer().includingDefaultValueFields().omittingInsignificantWhitespace();}}
{{System.out.println(printer.print(dm));}}
> Create a ConvertProtobufToJson procesor
> ---------------------------------------
>
> Key: NIFI-5538
> URL: https://issues.apache.org/jira/browse/NIFI-5538
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: James Srinivasan
> Priority: Major
>
> Protocol Buffers (protobuf) is a serialization format:
> [https://developers.google.com/protocol-buffers/]
> It is somewhat similar to AVRO, but the schema is not normally carried
> alongside the data, so messages are not self-describing. Instead, a separate
> .proto schema definition is written and the protoc binary tool used to
> generate bindings for the appropriate language (Java, Python, C++ etc).
> It would be great to be able to handle protobuf in NiFi. Full support as a
> record reader/writer would be ideal, but is a fair amount of work and when I
> (briefly) looked at it, the use of a separate schema definition language
> looked tricky.
> Pre-empting the question why not use
> [https://github.com/whiver/nifi-protobuf-processor,] that includes the
> capability to compile a .proto schema definition, which includes C++ binary
> executables inside the nar. For my use case that is not permitted (but I
> fully accept others may find this k). Instead, my processor requires the
> .proto schema to be compiled using the protoc tool to a Field Descriptor Set,
> which is a compiled protobuf encoded blob which describes the schema (see
> [https://developers.google.com/protocol-buffers/docs/techniques#self-description).]
> For my use case, conversion to JSON is adequate and something I have already
> implemented. If there is enough community interest, I will go through my
> organisation's release process (about as much effort as implementing and
> testing the processor, and much less fun!).
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
