Re: Apache NIFI report:Caused by: java.lang.OutOfMemoryError: Compressed class space

Hello, Did you check nifi-bootstrap.log? Since the output is logged to stdout, such information is logged to nifi-bootstrap.log instead of nifi-app.log. Thanks, Koji On Thu, Oct 10, 2019 at 8:14 PM abellnotring wrote: > Hi,all > I’m running two nodes NIFI cluster. some day, it reported an

Re: Can CaptureChangeMySQL be scheduled to all nodes instead of primary node?

Hi Lei, I don't know any NiFi built-in feature to achieve that. To distribute CaptureChangeMySQL load among nodes, I'd deploy separate standalone NiFi (or even MiNiFi Java) in addition to the main NiFi cluster for the main data flow. For example, if there are 5 databases and 3 NiFi nodes, deploy

Re: NIFI - Fetchfile - Execute SQL - Put Database Record

Hi Asmath, How about using PutSQL? FetchFile -> PutSQL -> PutDatabaseRecord You can specify a SQL statement at PutSQL 'SQL Statement' property, using FlowFile attribute. For example, delete from tbl where file_name = '${filename}' This way, the FlowFile content can be passed to PutDatabaseRecord

Re: Start InvokeHTTP using Cron and run the InvokeHTTP multiple times for paginated results

Devaraj, I've created a mockup of the flow you might use, with some notes to help you. https://tinyurl.com/y6gp5r4z Notice how you start the dataflow using GenerateFlowFile, call InvokeHTTP with the initial URL, then process the results. When you extract the results, create a new attribute for

Start InvokeHTTP using Cron and run the InvokeHTTP multiple times for paginated results

Hi All, I am looking for a solution to below problem. I want to have cron job (starts at 2 AM daily) which pulls data from certain API using GET request. The results will be sent to downstream component and if there are more results to pull based on nextpage key in the response, I need to call

Re: Apache Nifi with IBM Event Streams

Thanks Bryan. I am using the docker container for running Apache Nifi. I got the JABA_HOME PATH and navigated to the cacerts file and configured SSLSecurityContext and it is working now. Regards Devaraj -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: InvokeHTTP with dynamic query parameters based on previous request response

Hi Vijay, Thanks for your response. I want to have only one InvokeHTTP request which continuously query for the API with dynamic parameters. I am new to Apache Nifi, if you could provided little more details, it would be great. Regards Devaraj -- Sent from:

Re: Apache Nifi with IBM Event Streams

You can't create a TLS connection to something if you don't have access to the public key of the CA. This is how all TLS/https works, your browser has the public keys of all the major CAs in order to trust the websites you visit over https. If IBM Cloud is generating certificates and signing them

Re: InvokeHTTP with dynamic query parameters based on previous request response

Devraj, You will need to parse the response JSON produced by the first InvokeHTTP. You can do extract the timestamp you want using the EL syntax. If you only care when you made the previous request and want to store that timestamp somewhere, you can do so by adding a new attribute in the

Re: Apache Nifi with IBM Event Streams

Hi Bryan, The problem is I am using Kafka hosted in IBM Cloud and I don't have access to any certificates/keys. Regards Devaraj -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

InvokeHTTP with dynamic query parameters based on previous request response

Hi, I am planning to pull data from github API URL https://api.github.com/repos/kubernetes/kubernetes/issues?since=2018-01-01T00:00:00Z using InvokeHTTP processor. Once the processor fetch data, I would

Re: access token (secured NiFi) in InvokeHTTP, PostHTTP

Tomas It just need someone to implement various standards. Right now I believe it is purely TLS one way or mutual auth and also supports basic and digest. Thanks On Thu, Oct 10, 2019 at 11:00 AM Tomas Hudik wrote: > Hi Erik > thank you very much for the mail. > > My try is not about coding

Re: access token (secured NiFi) in InvokeHTTP, PostHTTP

Hi Erik thank you very much for the mail. My try is not about coding API. I just want to know whether processors like *InvokeHTTP, PostHTTP *can work with the access token(if you have some kerberized site). (the example I have provided is just for fast try) Do you know whether those processors

Re: Apache Nifi with IBM Event Streams

You need to set the truststore filename and password. The truststore needs to have public key of the CA that signed the certs used by the Kafka broker (i.e. it needs to trust the certs used by the Kafka broker). On Thu, Oct 10, 2019 at 10:00 AM dkommineni wrote: > > I think I have messed up

RE: Apache Nifi with IBM Event Streams

I think I have messed up with attachments.

Re: Apache Nifi with IBM Event Streams

https://issues.apache.org/jira/browse/NIFI-4820 On Thu, Oct 10, 2019 at 9:56 AM Joe Witt wrote: > > You can use SASL/Plain today with the kafka 2 procs in NiFi but as noted it > is unpleasant to configure. A much easier/clear configuration is being > worked on right now. Not positive what

Re: Apache Nifi with IBM Event Streams

You can use SASL/Plain today with the kafka 2 procs in NiFi but as noted it is unpleasant to configure. A much easier/clear configuration is being worked on right now. Not positive what the JIRA for it is though. Should be available quite soon. Thanks On Thu, Oct 10, 2019 at 9:31 AM

RE: Apache Nifi with IBM Event Streams

Thanks Chris. I am trying to setup Kafka for "PublishKafka_2_0". I did followed the same steps, when I try to setup SSLContextService, I am not sure what details to provide for keystore/truststore file name/path(s). When I run Java application to connect to IBM Event Streams, the kafka log

Apache NIFI report:Caused by: java.lang.OutOfMemoryError: Compressed class space

Hi,all    I’m running two nodes NIFI cluster. some day, it reported an OOM problem with this message:Caused by: java.lang.OutOfMemoryError: Compressed class space.    I added "java.arg.18=-XX:+TraceClassUnloading            java.arg.19=-XX:+TraceClassLoading” to

RE: Apache Nifi with IBM Event Streams

Hi It seems you want to configure ConsumeKafka / ConsumeKafkaRecord for SASL_SSL PLAIN. It is possible but not straightforward. The "Kerberos Service Name" cannot be empty but its content does not matter since it will not be used in this case. The trick is to specify the 2 additional properties