Re: ERRORs when trying to make new RPGs

Hello It appears to be throwing an NPE when establishing the position of the component you're trying to add. While clearly an unfriendly error message as-is did you specify the position in your request? Thanks JOe On Wed, Apr 19, 2017 at 10:36 AM, Pushkara R wrote: >

UpdateAttribute Advanced tab

Hello all, Using Advanced feature of UpdateAttribute I do not believe that there is a way to delete an attribute ( action ) ? Example if attribute1 = A ( condition ) Delete attributeY But we need attributeY for many other flowfiles that dont have attribute1 and we "enrich" through this

ERRORs when trying to make new RPGs

Hi, I'm writing a custom REST client for NiFi. I am having troubles with the POST method on the /process-groups//remote-process-groups endpoint. The body of the POST method is as follows : { "component": { "targetUri": target_uri, "communicationsTimeout": "30 sec",

Re: LookupAttribute Processor how to config

Another question: I tried to use the InMemoryLookupTableService, but was not able to find out how to fill it with key/values. Where/How can I add entries? Thanks Mika> On 04/17/2017 12:22 AM, Mika Borner wrote: Thanks for your answer. After playing around with your example, I think I'm

Re: UpdateAttribute Advanced tab

Joe, Thanks that's exactly what we need too. On Wed, Apr 19, 2017 at 1:24 PM Joe Skora wrote: > Juan, > > I believe there is already a ticket for this, check NIFI-3168 > and see if that covers > what you want. > > Regards, >

Re: ERRORs when trying to make new RPGs

Thanks a lot matt. I was not concerned with position which is why I did not provide it in the payload. Post requests to create processors and ports worked without the position so I assumed it would do so for RPGs as well. Can I use 0,0 as a default position? Thanks Pushkar On 19-Apr-2017 21:42,

Re: ERRORs when trying to make new RPGs

One way to see what the payload should look like would be to open up the network tab in your browser and perform the action you're trying to automate. You should be able to see what requests the NiFi UI performs and replicate those. Thanks, Bryan On Wed, Apr 19, 2017 at 11:49 AM, Joe Witt

Re: UpdateAttribute Advanced tab

Juan, I believe there is already a ticket for this, check NIFI-3168 and see if that covers what you want. Regards, Joe On Wed, Apr 19, 2017 at 11:37 AM, Juan Sequeiros wrote: > Hello all, > > Using Advanced feature of

Re: Provenance Event Use Cases?

Simon, We use NIFI's data provenance capabilities, to track the like cycle of a "flowFile" / data object as it goes through its system lifecycle. ( LINEAGE ) We also use it for troubleshooting as we can see the nifi attributes ( metadata ) and its content ( if configured ) You can also use

Re: Decompressing content consumed from Kafka topics?

Hello UnpackContent is for archive formats. gzip/snappy/lz4 aren't archive formats but are compression types. You want CompressContent in decompress mode. I think the first question is why they didn't decompress. Did you verify the were indeed still compressed? You can use IdentifyMimeType

Re: Provenance Event performance

Simon, I feel that " provenance event is emitted for each flowfile for each processor." is accurate understanding "each processor" means the unique processors the flowFile goes through. The provenance database is a lucene database and 1 million provenance events is not unreasonable. It would

Re: Provenance Event performance

You're right that the generation and indexing of provenance data creates overhead. We've put considerable effort in minimizing that overhead to a point where you should not have to think about it and still get all the powerful user experience/auditing gains it provides. However, when you're

Re: ERRORs when trying to make new RPGs

This is a known issue [1] that has been addressed in the upcoming 1.2.0 release. If you include a position then you should be able to creation the component successfully. Sorry for the inconvenience. Matt [1] https://issues.apache.org/jira/browse/NIFI-2994 On Wed, Apr 19, 2017 at 12:00 PM,

HIVE Controller service and convertJSONtoSQL processor

Hey guys, Has anyone tried to add a CONVERTJSONtoSQL processor on the graph and then configure a HIVEconnectionPool controller service on it? When I try to add a JDBC connection, it allows me to add the DBCPPool controller service, but if I try to add the HIVE option, it only gives me a

Re: Provenance Event Use Cases?

Additionally it is important to note that flow level changes are now exposed and available to reporting tasks as well. It is envisioned this will be used to report to systems like Apache Atlas for that flow level metadata you describe but made far more powerful by combining it with event level

Decompressing content consumed from Kafka topics?

Hi, I have NiFi 1.1.1 and would like to compress messages before sending to a Kafka topic. I can do this in the PublishKafka_0_10 processor. There is a "Compression Type" property where I can choose: - none - gzip - snappy - lz4 But in the ConsumeKafka_0_10 processor, there is no option to

Re: Provenance Event Use Cases?

Simon, The provenance capability is definitely used by many users for governance and regulatory purposes. For example, when dealing with geolocation data, many countries regulate the export of this data outside their borders. With provenance, you can provably demonstrate that every flowfile

provenance events and performance

Hi All, In some parts of the NiFi documentation, it is stated that a provenance event is emitted for each flowfile for each processor. However elsewhere it is stated that no provenance-event is generated for a flowfile sent to the “success” output of a processor - which is true? And are

Provenance Event performance

Hi All, In some parts of the NiFi documentation, it is stated that a provenance event is emitted for each flowfile for each processor. However elsewhere it is stated that no provenance-event is generated for a flowfile sent to the “success” output of a processor - which is true? And are

Provenance Event Use Cases?

Hi All, Can someone explain to me the business-level use cases that "provenance events" are intended to solve? I can see that they are useful for "flow developers" to debug problems. But is that their only use? Can they be used to address some kinds of regulatory compliance requirements?