Thanks Oleg. Just want to clarify: PutKafka is unusable in 0.6.1? On Thu, May 26, 2016 at 4:49 PM, Oleg Zhurakousky < [email protected]> wrote:
> Igor > > Thanks a lot > > That’s clearly a bug, simply because NPE originates form our code (must > never happen). Thankfully it was fixed already and will be available in 0.7 > release. But if you brave enough you can build from master and use it in > your current environment in the meantime (let me know if you need help with > it). > Otherwise, the 0.7 is not to far away. > > Cheers > Oleg > > On May 26, 2016, at 4:39 PM, Igor Kravzov <[email protected]> wrote: > > Actually I wanted to write new email, but will ask here. > I am trying to use PutKafka with Hortonworks HDP 2.4. Latest. The template > is simple and shown bellow. The warning above I received when tried to > change processor's configuration and start again. > > <image.png> > > Actually I am unable to send file to Kafka. Getting exception bellow. > > > 2016-05-26 16:25:56,227 INFO [StandardProcessScheduler Thread-4] > o.a.k.clients.producer.ProducerConfig ProducerConfig values: > compression.type = none > metric.reporters = [] > metadata.max.age.ms = 300000 > metadata.fetch.timeout.ms = 30000 > acks = 0 > batch.size = 1 > reconnect.backoff.ms = 10 > bootstrap.servers = [10.1.132.113:6667] > receive.buffer.bytes = 32768 > retry.backoff.ms = 100 > buffer.memory = 5242880 > timeout.ms = 30000 > key.serializer = class > org.apache.kafka.common.serialization.ByteArraySerializer > retries = 0 > max.request.size = 1048576 > block.on.buffer.full = true > value.serializer = class > org.apache.kafka.common.serialization.ByteArraySerializer > metrics.sample.window.ms = 30000 > send.buffer.bytes = 131072 > max.in.flight.requests.per.connection = 5 > metrics.num.samples = 2 > linger.ms = 5000 > client.id = MyTEst > > 2016-05-26 16:25:56,443 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@6e7a6be5 // Another save > pending = false > 2016-05-26 16:26:00,771 WARN [StandardProcessScheduler Thread-4] > o.a.k.clients.producer.ProducerConfig The configuration partitioner.class = > null was supplied but isn't a known config. > 2016-05-26 16:26:00,771 INFO [StandardProcessScheduler Thread-8] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > PutKafka[id=cfe1c4de-ac82-4d7e-86db-44fa0f151c55] to run with 1 threads > 2016-05-26 16:26:04,979 INFO [pool-16-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile > Repository > 2016-05-26 16:26:05,216 INFO [pool-16-thread-1] > org.wali.MinimalLockingWriteAheadLog > org.wali.MinimalLockingWriteAheadLog@1227013b checkpointed with 1 Records > and 0 Swap Files in 237 milliseconds (Stop-the-world time = 110 > milliseconds, Clear Edit Logs time = 124 millis), max Transaction ID > 252557730 > 2016-05-26 16:26:05,216 INFO [pool-16-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 1 records in 237 milliseconds > 2016-05-26 16:27:00,963 WARN [Timer-Driven Process Thread-2] > o.a.nifi.processors.kafka.KafkaPublisher Timed out while waiting for acks > from Kafka > 2016-05-26 16:27:00,963 WARN [Timer-Driven Process Thread-2] > o.apache.nifi.processors.kafka.PutKafka > PutKafka[id=cfe1c4de-ac82-4d7e-86db-44fa0f151c55] Timed out while waiting > for acks from Kafka > 2016-05-26 16:27:00,963 ERROR [Timer-Driven Process Thread-2] > o.apache.nifi.processors.kafka.PutKafka > PutKafka[id=cfe1c4de-ac82-4d7e-86db-44fa0f151c55] > PutKafka[id=cfe1c4de-ac82-4d7e-86db-44fa0f151c55] failed to process due to > java.lang.NullPointerException; rolling back session: > java.lang.NullPointerException > 2016-05-26 16:27:00,965 ERROR [Timer-Driven Process Thread-2] > o.apache.nifi.processors.kafka.PutKafka > java.lang.NullPointerException: null > at java.lang.String.<init>(Unknown Source) ~[na:1.8.0_91] > at > org.apache.nifi.processors.kafka.PutKafka.buildFailedFlowFileAttributes(PutKafka.java:396) > ~[na:na] > at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:308) > ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.6.1.jar:0.6.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) > [nifi-framework-core-0.6.1.jar:0.6.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-0.6.1.jar:0.6.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-0.6.1.jar:0.6.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) > [nifi-framework-core-0.6.1.jar:0.6.1] > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) > [na:1.8.0_91] > at java.util.concurrent.FutureTask.runAndReset(Unknown Source) > [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown > Source) [na:1.8.0_91] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown > Source) [na:1.8.0_91] > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > [na:1.8.0_91] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > [na:1.8.0_91] > at java.lang.Thread.run(Unknown Source) [na:1.8.0_91] > > On Thu, May 26, 2016 at 4:28 PM, Oleg Zhurakousky < > [email protected]> wrote: > >> That is because the processor is already in the process of being stopped >> and you probably tried to start it again. >> That said, the fact that you see it is probably a bug since you should >> not be able to see ‘start’ action until processor is fully stopped. >> >> Could you please share more details as to what are you doing (in terms of >> sequence of events) >> >> Thanks >> Oleg >> >> > On May 26, 2016, at 4:12 PM, Igor Kravzov <[email protected]> >> wrote: >> > >> > Why I am getting warning bellow? >> > <image.png> >> > >> > >> > >> >> > >
