Re: Multiple NiFi clusters with 1 NiFi Rigistry

2018-11-19 Thread Koji Kawamura
Hi Chad, NiFi Registry uses NiFi user's identity to authorize request. Registry also checks NiFi instance's identity to authorize proxying user requests, but this can only authorize proxy capability. In order to control access such as bucket read/write, Registry uses NiFi user's identity. I

Multiple NiFi clusters with 1 NiFi Rigistry

2018-11-19 Thread Woodhead, Chad
I am standing up 3 new HDF 3.2 clusters (Dev, Cert, and Prod) and we will be focusing on NiFi (1.7.0) + NiFi Registry (0.2.0). We are using git as our FlowPersistenceProvider. My plan is to use 1 NiFi Registry (the Prod NiFi registry) for all 3 clusters, rather than having 3 NiFi Registries and

Re: UpdateAttributes use problems

2018-11-19 Thread l vic
"AttributesToJson" seems to work On Mon, Nov 19, 2018 at 10:14 AM Nathan Gough wrote: > UpdateAttribute only updates flowfile attributes and will not change the > content. Flowfile attributes and the actual content are two separate > objects and are not linked. Updating one does not affect the

Re: UpdateAttributes use problems

2018-11-19 Thread Nathan Gough
UpdateAttribute only updates flowfile attributes and will not change the content. Flowfile attributes and the actual content are two separate objects and are not linked. Updating one does not affect the other. If you want to manipulate your JSON content, look at the JoltTransformJSON processor:

Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue

2018-11-19 Thread Josef.Zahner1
@Mark: I’m now able to query the REST-API. However, the call nifi-api/processors//diagnostics does reply on all nodes “An unexpected error has occurred. Please check the logs for additional details.” If I remove the /diagnostics I’m getting a useful reply in JSON format. I tried on several

Re: Adding a new column to a CSV file

2018-11-19 Thread Elemir Stevko
Thanks a lot, Koji! On 19/11/18, 5:16 pm, "Koji Kawamura" wrote: Hi Elemir, Yes, you can use UpdateRecord to add a new timestamp column and set current timestamp. Important configurations: - The output schema has to have the whole columns, including the new

Re: CaptureChangeMySQL - throwing Binlog connector communications failure

2018-11-19 Thread Koji Kawamura
Another possibility is that the command 'SHOW MASTER STATUS' and the ROTATE event just return the correct position, but the binlog file itself contain some issue. I'd suggest checking MySQL master server side logs for any disk related issue. In that case, some data loss is inevitable. I tried