James Srinivasan created NIFI-5538:
--------------------------------------
Summary: Create a ConvertProtobufToJson procesor
Key: NIFI-5538
URL: https://issues.apache.org/jira/browse/NIFI-5538
Project: Apache NiFi
Issue Type: New Feature
Components: Extensions
Reporter: James Srinivasan
Protocol Buffers (protobuf) is a serialization format:
[https://developers.google.com/protocol-buffers/]
It is somewhat similar to AVRO, but the schema is not normally carried
alongside the data, so messages are not self-describing. Instead, a separate
.proto schema definition is written and the protoc binary tool used to generate
bindings for the appropriate language (Java, Python, C++ etc).
It would be great to be able to handle protobuf in NiFi. Full support as a
record reader/writer would be ideal, but is a fair amount of work and when I
(briefly) looked at it, the use of a separate schema definition language looked
tricky.
Pre-empting the question why not use
[https://github.com/whiver/nifi-protobuf-processor,] that includes the
capability to compile a .proto schema definition, which includes C++ binary
executables inside the nar. For my use case that is not permitted (but I fully
accept others may find this k). Instead, my processor requires the .proto
schema to be compiled using the protoc tool to a Field Descriptor Set, which is
a compiled protobuf encoded blob which describes the schema (see
[https://developers.google.com/protocol-buffers/docs/techniques#self-description).]
For my use case, conversion to JSON is adequate and something I have already
implemented. If there is enough community interest, I will go through my
organisation's release process (about as much effort as implementing and
testing the processor, and much less fun!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
