Fair enough. I forgot that we had the advanced UI for the Jolt Spec. That would definitely be beneficial.
On Nov 20, 2019, at 10:47 AM, Joe Witt <[email protected]<mailto:[email protected]>> wrote: ...specific or generic - tradeoffs. A specific service would allow Jolt Spec editing to have context and meaning. A generic service for just blobs would not. I think our Jolt stuff has an advanced UI that makes writing the specs a bit easier. If true a more specific service would make sense. ....for this case anyway. On Wed, Nov 20, 2019 at 10:36 AM Mark Payne <[email protected]<mailto:[email protected]>> wrote: I would recommend that we also be careful about the naming here and tying this to Jolt. Really, this is just a mechanism for externalizing a big blob of text (or bytes). There are several other processors and controller services that do this, such as scripted components, Hadoop related processors that need things like core-site.xml, etc. It may be advantageous to consider this as a more generic way to access any such resource. A simple implementation would be purely configured through the UI but there could be other future implementations that are based on fetching from remote services, etc. Thanks -Mark Sent from my iPhone On Nov 20, 2019, at 10:28 AM, Joe Witt <[email protected]<mailto:[email protected]>> wrote: Yeah filing a JIRA would be good. Contributing a PR for it would be even better. It should have no impact on the schema registry controller service. This is different. Thanks On Wed, Nov 20, 2019 at 10:26 AM Etienne Jouvin <[email protected]<mailto:[email protected]>> wrote: Yes it would be a ControllerService as you described. There is currently three implementation : * AvroSchemaRegistry * ConfluentSchemaRegistry * HortonworksSchemaRegistry It could be based on something like them. May be I should send something on Jira or somewhere else to submit the idea to NiFi developers ? But it also means that the processor JoltTransformJSON and JoltTransformRecord should be changed. Le mer. 20 nov. 2019 à 16:08, Joe Witt <[email protected]<mailto:[email protected]>> a écrit : Hello Is the idea to have a place to store Jolt specifications that you could then access in various components? If so a simple ControllerService such as 'JoltSpecControllerService' which has a list of keys (names of specs) and values (the spec) would probably do the trick. Thanks On Wed, Nov 20, 2019 at 10:04 AM Otto Fowler <[email protected]<mailto:[email protected]>> wrote: I think that is a great idea, I’d suggest the same thing for protobuf specs as well. Even if the first step is the registry supporting raw bytes access and support…. On November 20, 2019 at 09:28:23, Etienne Jouvin ([email protected]<mailto:[email protected]>) wrote: Hello all. For reader and writers, there is the possibility to store the schema inside a schema registry. What do you think about having this type of mechanism for JolftTransformation ? Currently, I can put Jolt specification in variables and get them from it, but I think it could be nice tohave same as schema registry. Regards. Etienne Jouvin
