Re: how to copy root fields in Jolt transform?

2018-09-04 Thread Andy LoPresto
There are some slight changes with NiFi’s Jolt compared to the test site — NiFi is automatically applying a chain operation but you can change that in the processor configuration. I’d also recommend using the Custom View on the JTJ processor to have a better interface for applying your

Re: how to copy root fields in Jolt transform?

2018-09-04 Thread l vic
Thank you, works on the Jolt demo site. With real Nifi it doesn't. I am trying to print those values in ExecuteScript with Groovy script iterating flowfile attributets: import org.apache.commons.io.IOUtils import java.nio.charset.StandardCharsets import org.apache.nifi.processor.io.StreamCallback

Jolt doesn't work?

2018-09-04 Thread l vic
I have a json record that contains array "mylst": { "id": 0, "name": "Root", "mylist": [{ "id": 10, "info": "2am-3am" }, { "id": 11, "info": "3AM-4AM" } ] } I tested Jolt spec: [ { "operation": "shift", "spec": {

Re: Wrapping a JSON string

2018-09-04 Thread Steve Champagne
I was hoping to use the record processors otherwise I would definitely use jolt. I was eventually able to get what I was after using a ScriptedReader. I do like 'payload' better than 'json_string' though. :) Thanks, Steve On Mon, Sep 3, 2018 at 3:09 AM DEHAY Aurelien wrote: > Hello. > > > >

Re: how to copy root fields in Jolt transform?

2018-09-04 Thread Matt Burgess
Add the following to the end of your shift spec: "*": "&" This (at the root) matches any key not already matched and puts it in the output at the same location. Regards, Matt On Tue, Sep 4, 2018 at 11:39 AM l vic wrote: > > Hi, > I want to "flatten" the following object: > { > "id": 0, >

Re: Managing Secrets in NiFi Registry

2018-09-04 Thread Andy LoPresto
Jonathan, Bryan gave a really good response to this. As he mentioned, there has definitely been discussion about providing this feature and further securing the variable registry and the integration between NiFi and NiFi Registry. I’ve listed some open Jiras at the bottom which further

Re: Managing Secrets in NiFi Registry

2018-09-04 Thread Bryan Bende
Hello, You are correct that currently variables cannot be marked as sensitive, and for that reason they currently shouldn't be used to store sensitive values since they will be stored in plain-text in the flow.xml.gz and also in the versioned flows saved to registry. I do think the concept of

how to copy root fields in Jolt transform?

2018-09-04 Thread l vic
Hi, I want to "flatten" the following object: { "id": 0, "name": "Root", "mylist": [{ "id": 10, "info": "2am-3am" }, { "id": 11, "info": "3AM-4AM" }, { "id": 12, "info": "4am-5am" }] } I figured how to "flatten" array, but "root"

Managing Secrets in NiFi Registry

2018-09-04 Thread Jonathan Meran
Hello, I am looking for some guidance on managing sensitive property values for things such as credentials in a DBCPConnectionPool within the NiFi Registry Development Life Cycle. Currently we have rolled our own deployment tool in which we manage configuration files per environment (Dev, QA,

Re: Best practices for running Apache NiFi in production in a Docker container

2018-09-04 Thread Joe Percivall
Hi Peter, Thanks for the follow-up. Yup, I agree with the relationship between Xmx/Xms and UseCGroupMemoryLimitForHeap/MaxRAMFraction. For the MaxRAMFraction=1 though, it seems that it does leave at least some room for off-heap memory that said, that may not be enough for NiFi and it's normal