1. The HandleHttpRequest Processor get the message. 2. The message route to other processors based on the attribute. 3. We have our customised processor to process the message. 4. Then message would be redirected to the HandleHttpResponse.
On Mon, Sep 25, 2017 at 3:20 PM, Joe Witt <[email protected]> wrote: > What is the flow doing in between the request/response portion? > Please share more details about the configuration overall. > > Thanks > > On Mon, Sep 25, 2017 at 9:16 AM, Lou Tian <[email protected]> wrote: > > Hi Joe, > > > > java version: 1.8.0_121 > > heap size: > > # JVM memory settings > > java.arg.2=-Xms512m > > java.arg.3=-Xmx512m > > nifi version: 1.3.0 > > > > Also, we put Nifi in the Docker. > > > > Kind Regrads, > > Tian > > > > On Mon, Sep 25, 2017 at 2:39 PM, Joe Witt <[email protected]> wrote: > >> > >> Tian, > >> > >> Please provide information on the JRE being used (java -version) and > >> the environment configuration. How large is your heap? This can be > >> found in conf/bootstrap.conf. What version of nifi are you using? > >> > >> Thanks > >> > >> On Mon, Sep 25, 2017 at 8:29 AM, Lou Tian <[email protected]> > wrote: > >> > Hi, > >> > > >> > We are doing performance test for our NIFI flow with Gatling. But > after > >> > several run, the NIFI always has the OutOfMemory error. I did not find > >> > similar questions in the mailing list, if you already answered similar > >> > questions please let me know. > >> > > >> > Problem description: > >> > We have the Nifi flow. The normal flow works fine. To evaluate whether > >> > our > >> > flow can handle the load, we decided to do the performance test with > >> > Gatling. > >> > > >> > 1) We add the two processors HandleHttpRequest at the start of the > flow > >> > and > >> > HandleHttpResponse at the end of the flow. So our nifi is like a > >> > webservice > >> > and Gatling will evaluate the response time. 2) Then we continuously > >> > push > >> > messages to HandleHttpRequest processor. > >> > > >> > Problem: > >> > Nifi can only handle two runs. Then the third time, it failed and we > >> > have to > >> > restart the NIFI. I copied some error log here. > >> > > >> >> o.a.n.p.standard.HandleHttpRequest HandleHttpRequest[id=**] > >> >> HandleHttpRequest[id=**] failed to process session due to > >> >> java.lang.OutOfMemoryError: Java heap space: {} > >> >> o.a.n.p.standard.HandleHttpRequest HandleHttpRequest[id=**] > >> >> HandleHttpRequest[id=**] failed to process session due to > >> >> java.lang.OutOfMemoryError: Java heap space: {} > >> >> java.lang.OutOfMemoryError: Java heap space > >> >> at java.util.HashMap.values(HashMap.java:958) > >> >> at > >> >> > >> >> org.apache.nifi.controller.repository.StandardProcessSession. > resetWriteClaims(StandardProcessSession.java:2720) > >> >> at > >> >> > >> >> org.apache.nifi.controller.repository.StandardProcessSession. > checkpoint(StandardProcessSession.java:213) > >> >> at > >> >> > >> >> org.apache.nifi.controller.repository.StandardProcessSession.commit( > StandardProcessSession.java:318) > >> >> at > >> >> > >> >> org.apache.nifi.processor.AbstractProcessor.onTrigger( > AbstractProcessor.java:28) > >> >> at > >> >> > >> >> org.apache.nifi.controller.StandardProcessorNode.onTrigger( > StandardProcessorNode.java:1120) > >> >> at > >> >> > >> >> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call( > ContinuallyRunProcessorTask.java:147) > >> >> at > >> >> > >> >> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call( > ContinuallyRunProcessorTask.java:47) > >> >> at > >> >> > >> >> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1. > run(TimerDrivenSchedulingAgent.java:132) > >> >> at > >> >> java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > >> >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > >> >> at > >> >> > >> >> java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > >> >> at > >> >> > >> >> java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > >> >> at > >> >> > >> >> java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > >> >> at > >> >> > >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > >> >> at java.lang.Thread.run(Thread.java:748) > >> > > >> > > >> > So our final questions: > >> > 1. Do you think it is the HandleHttpRequest processors problem? Or > there > >> > is > >> > something wrong in our configuration. Anything we can do to avoid such > >> > problem? > >> > 2. If it's the processor, will you plan to fix it in the coming > version? > >> > > >> > Thank you so much for your reply. > >> > > >> > Kind Regards, > >> > Tian > >> > > > > > > > > > > > -- > > Kind Regards, > > > > Tian Lou > > > -- Kind Regards, Tian Lou
