Re: Removing the Kafka DEBUG logs in catalina.out using log4j.properties

2018-05-14 Thread Karthick Kumar
Hi Andras/Kaufman, I have tried both logger properties, But it is not working for me. Here I share the some DEBUG logger message which is print in my tomcat node DEBUG AbstractCoordinator:704 - Sending Heartbeat request for group apptivodb5-newsfeed-messages-4-consumer to coordinator xx.xx.xx.x

Re: Kafka consumer slowly consume the data

2018-05-14 Thread Karthick Kumar
Hi Ted, I'm using kafka_2.11-0.10.2.0 version, I figure out that the issue from the wildfly node because of each consumed messages are passed by the Tomcat node and it sends the details to the wildfly node. When the wildfly node process the details again it will consume the next offset messages.

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

2018-05-14 Thread Ted Yu
I found the source code for Windows related jdk: https://github.com/frohoff/jdk8u-jdk/blob/master/src/windows/classes/sun/nio/fs/WindowsFileSystemProvider.java It has been about 11 years since I last worked on Windows code :-) I think the only way to gain more information is to modify Kafka sour

How to reset the GroupCoordinator's saved session timeout for a group?

2018-05-14 Thread Jeff Widman
How do you reset the GroupCoordinator's stored session timeout for a group before the session timeout actually expires? We have an emergency situation where we need to reduce the session timeout for a consumer group. The old session timeout was 12 hours (a temp hack to get around another problem)

WG: Exactly Once, Producer configuration

2018-05-14 Thread Thomas.Kass
Hi, the following article describes how to configure Apache Kafka system to enforce exactly once approach: https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/ One recommendation states that parameter transactional.id should be set in producer.prope

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

2018-05-14 Thread Ted Yu
Looks like you were trying to show some screen which didn't go thru. Examining LogSegment.scala, I wonder if the time index was being appended to (which caused the deletion to fail): // append an entry to the index (if needed) if(bytesSinceLastIndexEntry > indexIntervalBytes) {

Re: Kafka 2.11-1.1.0 crashes brokers and brings cluster down on Windows

2018-05-14 Thread M. Manna
I am having some difficulties debugging the cluster. IN CWIKI for kafka, I can see a setup guide for eclipse, but nothing as such for debugger setup. The issue is that every time the log cleaner thread is run, it's having a FATAL shutdown. I tried to close all producers and consumers, but it still