[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709138#comment-15709138 ] Soumyajit Sahu commented on KAFKA-1194: --- [~haraldk] could you please provide all of your server.properties. I will see if I can reproduce and dig more. Thanks! > The kafka broker cannot delete the old log files after the configured time > -- > > Key: KAFKA-1194 > URL: https://issues.apache.org/jira/browse/KAFKA-1194 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.8.1 > Environment: window >Reporter: Tao Qin >Assignee: Jay Kreps > Labels: features, patch > Fix For: 0.10.2.0 > > Attachments: KAFKA-1194.patch, Untitled.jpg, kafka-1194-v1.patch, > kafka-1194-v2.patch, screenshot-1.png > > Original Estimate: 72h > Remaining Estimate: 72h > > We tested it in windows environment, and set the log.retention.hours to 24 > hours. > # The minimum age of a log file to be eligible for deletion > log.retention.hours=24 > After several days, the kafka broker still cannot delete the old log file. > And we get the following exceptions: > [2013-12-19 01:57:38,528] ERROR Uncaught exception in scheduled task > 'kafka-log-retention' (kafka.utils.KafkaScheduler) > kafka.common.KafkaStorageException: Failed to change the log file suffix from > to .deleted for log segment 1516723 > at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:249) > at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:638) > at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:629) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at > scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) > at scala.collection.immutable.List.foreach(List.scala:76) > at kafka.log.Log.deleteOldSegments(Log.scala:418) > at > kafka.log.LogManager.kafka$log$LogManager$$cleanupExpiredSegments(LogManager.scala:284) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:316) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:314) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:743) > at scala.collection.Iterator$class.foreach(Iterator.scala:772) > at > scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) > at > scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:742) > at kafka.log.LogManager.cleanupLogs(LogManager.scala:314) > at > kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:143) > at kafka.utils.KafkaScheduler$$anon$1.run(KafkaScheduler.scala:100) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > I think this error happens because kafka tries to rename the log file when it > is still opened. So we should close the file first before rename. > The index file uses a special data structure, the MappedByteBuffer. Javadoc > describes it as: > A mapped byte buffer and the file mapping that it represents remain valid > until the buffer itself is garbage-collected. > Fortunately, I find a forceUnmap function in kafka code, and perhaps it can > be used to free the MappedByteBuffer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15706415#comment-15706415 ] Soumyajit Sahu commented on KAFKA-3123: --- I am closing this previously opened (and now obsolete) PR. > Follower Broker cannot start if offsets are already out of range > > > Key: KAFKA-3123 > URL: https://issues.apache.org/jira/browse/KAFKA-3123 > Project: Kafka > Issue Type: Bug > Components: core, replication >Affects Versions: 0.9.0.0 >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu >Priority: Critical > Labels: patch > Fix For: 0.10.2.0 > > Attachments: > 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch > > > I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one > machine at a time. Our logs have just 2 hours of retention. I had re-imaged > the test machine under consideration, and got the following error in loop > after starting afresh with 0.9.0 broker: > [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica > 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to > current leader 169595708's start offset 334086 > (kafka.server.ReplicaFetcherThread) > [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error > getting offset for partition [EventLogs4,1] to broker 169595708 > (kafka.server.ReplicaFetcherThread) > java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] > cannot be aborted and paused since it is in LogCleaningPaused state. > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) > at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:141) > at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:304) > at > kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:185) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:152) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:122) > at scala.Option.foreach(Option.scala:236) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:122) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:120) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > I could unblock myself with a code change. I deleted the action for "case s > =>" in the LogCleanerManager.scala's abortAndPauseCleaning(). I think we > should not throw exception if the state is already LogCleaningAborted or > LogCleaningPaused in this function, but instead just let it roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15706222#comment-15706222 ] Soumyajit Sahu commented on KAFKA-1194: --- Thanks for confirming that [~haraldk]. Glad to be of help. I hope we can merge this into the trunk. [~abhit011] Your issue seems to be your environment problem, and not related to the actual problem here. I will try to reach you over email later. > The kafka broker cannot delete the old log files after the configured time > -- > > Key: KAFKA-1194 > URL: https://issues.apache.org/jira/browse/KAFKA-1194 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.8.1 > Environment: window >Reporter: Tao Qin >Assignee: Jay Kreps > Labels: features, patch > Fix For: 0.10.2.0 > > Attachments: KAFKA-1194.patch, Untitled.jpg, kafka-1194-v1.patch, > kafka-1194-v2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > We tested it in windows environment, and set the log.retention.hours to 24 > hours. > # The minimum age of a log file to be eligible for deletion > log.retention.hours=24 > After several days, the kafka broker still cannot delete the old log file. > And we get the following exceptions: > [2013-12-19 01:57:38,528] ERROR Uncaught exception in scheduled task > 'kafka-log-retention' (kafka.utils.KafkaScheduler) > kafka.common.KafkaStorageException: Failed to change the log file suffix from > to .deleted for log segment 1516723 > at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:249) > at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:638) > at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:629) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at > scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) > at scala.collection.immutable.List.foreach(List.scala:76) > at kafka.log.Log.deleteOldSegments(Log.scala:418) > at > kafka.log.LogManager.kafka$log$LogManager$$cleanupExpiredSegments(LogManager.scala:284) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:316) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:314) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:743) > at scala.collection.Iterator$class.foreach(Iterator.scala:772) > at > scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) > at > scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:742) > at kafka.log.LogManager.cleanupLogs(LogManager.scala:314) > at > kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:143) > at kafka.utils.KafkaScheduler$$anon$1.run(KafkaScheduler.scala:100) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > I think this error happens because kafka tries to rename the log file when it > is still opened. So we should close the file first before rename. > The index file uses a special data structure, the MappedByteBuffer. Javadoc > describes it as: > A mapped byte buffer and the file mapping that it represents remain valid > until the buffer itself is garbage-collected. > Fortunately, I find a forceUnmap function in kafka code, and perhaps it can > be used to free the MappedByteBuffer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time
[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702991#comment-15702991 ] Soumyajit Sahu commented on KAFKA-1194: --- [~abhit011], [~bradvido] Could you please give the following a try (note, branch = SiphonRelease): https://github.com/Microsoft/Kafka/tree/SiphonRelease I have been running Kafka on Windows for a while now from this repo/branch. I have temporarily shared a (scala version 2.11) build here: https://1drv.ms/u/s!AvtbGhqVbZGYkTFIN23sIPE6rG5a. You will of course need to edit the server.properties and set JAVA_HOME, and then run the start.bat. [~haraldk] has been helping with the validation, but another eye would help. > The kafka broker cannot delete the old log files after the configured time > -- > > Key: KAFKA-1194 > URL: https://issues.apache.org/jira/browse/KAFKA-1194 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.8.1 > Environment: window >Reporter: Tao Qin >Assignee: Jay Kreps > Labels: features, patch > Fix For: 0.10.2.0 > > Attachments: KAFKA-1194.patch, kafka-1194-v1.patch, > kafka-1194-v2.patch > > Original Estimate: 72h > Remaining Estimate: 72h > > We tested it in windows environment, and set the log.retention.hours to 24 > hours. > # The minimum age of a log file to be eligible for deletion > log.retention.hours=24 > After several days, the kafka broker still cannot delete the old log file. > And we get the following exceptions: > [2013-12-19 01:57:38,528] ERROR Uncaught exception in scheduled task > 'kafka-log-retention' (kafka.utils.KafkaScheduler) > kafka.common.KafkaStorageException: Failed to change the log file suffix from > to .deleted for log segment 1516723 > at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:249) > at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:638) > at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:629) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:418) > at > scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) > at scala.collection.immutable.List.foreach(List.scala:76) > at kafka.log.Log.deleteOldSegments(Log.scala:418) > at > kafka.log.LogManager.kafka$log$LogManager$$cleanupExpiredSegments(LogManager.scala:284) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:316) > at > kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:314) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:743) > at scala.collection.Iterator$class.foreach(Iterator.scala:772) > at > scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) > at > scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615) > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:742) > at kafka.log.LogManager.cleanupLogs(LogManager.scala:314) > at > kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:143) > at kafka.utils.KafkaScheduler$$anon$1.run(KafkaScheduler.scala:100) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > I think this error happens because kafka tries to rename the log file when it > is still opened. So we should close the file first before rename. > The index file uses a special data structure, the MappedByteBuffer. Javadoc > describes it as: > A mapped byte buffer and the file mapping that it represents remain valid > until the buffer itself is garbage-collected. > Fortunately, I find a forceUnmap function in kafka code, and perhaps it can > be used to free the MappedByteBuffer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows
[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515023#comment-15515023 ] Soumyajit Sahu commented on KAFKA-2170: --- [~haraldk], I have updated/fixed the https://github.com/apache/kafka/pull/1757 to delete the .deleted files. I had missed closing file channels while using the java FileChannel API, which had led to runaway open file handles. Testing on a Windows server 2012 (and default log.segment.delete.delay.ms), I can see that the files get deleted successfully this time. > 10 LogTest cases failed for file.renameTo failed under windows > --- > > Key: KAFKA-2170 > URL: https://issues.apache.org/jira/browse/KAFKA-2170 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.10.1.0 > Environment: Windows >Reporter: Honghai Chen >Assignee: Jay Kreps > > get latest code from trunk, then run test > gradlew -i core:test --tests kafka.log.LogTest > Got 10 cases failed for same reason: > kafka.common.KafkaStorageException: Failed to change the log file suffix from > to .deleted for log segment 0 > at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:259) > at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:756) > at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:747) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:514) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:514) > at scala.collection.immutable.List.foreach(List.scala:318) > at kafka.log.Log.deleteOldSegments(Log.scala:514) > at kafka.log.LogTest.testAsyncDelete(LogTest.scala:633) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:48) > at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at $Proxy2.processTestClass(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:105) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.
[jira] [Commented] (KAFKA-2170) 10 LogTest cases failed for file.renameTo failed under windows
[ https://issues.apache.org/jira/browse/KAFKA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427096#comment-15427096 ] Soumyajit Sahu commented on KAFKA-2170: --- [~haraldk] Thanks for checking it out. I didn't try with compaction enabled. I will be able to dig more again after September 20th and get back to you. At a glance, it looks like there must have been a previous error which left the original file as it is, and hence leading to this: java.nio.file.FileAlreadyExistsException: C:\Users\hk\tmp\kafka-data\hktest-0\.index -> C:\Users\hk\tmp\kafka-data\hktest-0\.index.deleted In the meanwhile, could you kindly take in all the 3 patches below, and give it another test run? https://github.com/apache/kafka/pull/1757 https://github.com/apache/kafka/pull/1716 https://github.com/apache/kafka/pull/1718 Thanks! > 10 LogTest cases failed for file.renameTo failed under windows > --- > > Key: KAFKA-2170 > URL: https://issues.apache.org/jira/browse/KAFKA-2170 > Project: Kafka > Issue Type: Bug > Components: log >Affects Versions: 0.10.1.0 > Environment: Windows >Reporter: Honghai Chen >Assignee: Jay Kreps > > get latest code from trunk, then run test > gradlew -i core:test --tests kafka.log.LogTest > Got 10 cases failed for same reason: > kafka.common.KafkaStorageException: Failed to change the log file suffix from > to .deleted for log segment 0 > at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:259) > at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:756) > at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:747) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:514) > at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:514) > at scala.collection.immutable.List.foreach(List.scala:318) > at kafka.log.Log.deleteOldSegments(Log.scala:514) > at kafka.log.LogTest.testAsyncDelete(LogTest.scala:633) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) > at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) > at org.junit.runners.ParentRunner.run(ParentRunner.java:220) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:48) > at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at $Proxy2.processTestC
[jira] [Assigned] (KAFKA-4031) Check DirectBuffer's cleaner to be not null before using
[ https://issues.apache.org/jira/browse/KAFKA-4031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-4031: - Assignee: Soumyajit Sahu > Check DirectBuffer's cleaner to be not null before using > > > Key: KAFKA-4031 > URL: https://issues.apache.org/jira/browse/KAFKA-4031 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.10.0.1 >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu > Fix For: 0.10.0.2 > > > Found the following exception stack in our broker logs. > The fix should be straight forward with a null check. > [2016-08-09 17:10:24,451] WARN Error when freeing index buffer > (kafka.log.OffsetIndex) > java.lang.NullPointerException > at > kafka.log.OffsetIndex.kafka$log$OffsetIndex$$forceUnmap(OffsetIndex.scala:312) > at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:294) > at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:287) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231) > at kafka.log.OffsetIndex.resize(OffsetIndex.scala:287) > at kafka.log.Log.loadSegments(Log.scala:245) > at kafka.log.Log.(Log.scala:101) > at > kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:151) > at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-4031) Check DirectBuffer's cleaner to be not null before using
Soumyajit Sahu created KAFKA-4031: - Summary: Check DirectBuffer's cleaner to be not null before using Key: KAFKA-4031 URL: https://issues.apache.org/jira/browse/KAFKA-4031 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.10.0.1 Reporter: Soumyajit Sahu Fix For: 0.10.0.2 Found the following exception stack in our broker logs. The fix should be straight forward with a null check. [2016-08-09 17:10:24,451] WARN Error when freeing index buffer (kafka.log.OffsetIndex) java.lang.NullPointerException at kafka.log.OffsetIndex.kafka$log$OffsetIndex$$forceUnmap(OffsetIndex.scala:312) at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:294) at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:287) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231) at kafka.log.OffsetIndex.resize(OffsetIndex.scala:287) at kafka.log.Log.loadSegments(Log.scala:245) at kafka.log.Log.(Log.scala:101) at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:151) at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-3123: - Assignee: Soumyajit Sahu (was: Neha Narkhede) > Follower Broker cannot start if offsets are already out of range > > > Key: KAFKA-3123 > URL: https://issues.apache.org/jira/browse/KAFKA-3123 > Project: Kafka > Issue Type: Bug > Components: core, replication >Affects Versions: 0.9.0.0 >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu >Priority: Critical > Labels: patch > Fix For: 0.10.0.2 > > Attachments: > 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch > > > I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one > machine at a time. Our logs have just 2 hours of retention. I had re-imaged > the test machine under consideration, and got the following error in loop > after starting afresh with 0.9.0 broker: > [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica > 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to > current leader 169595708's start offset 334086 > (kafka.server.ReplicaFetcherThread) > [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error > getting offset for partition [EventLogs4,1] to broker 169595708 > (kafka.server.ReplicaFetcherThread) > java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] > cannot be aborted and paused since it is in LogCleaningPaused state. > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) > at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:141) > at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:304) > at > kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:185) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:152) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:122) > at scala.Option.foreach(Option.scala:236) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:122) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:120) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > I could unblock myself with a code change. I deleted the action for "case s > =>" in the LogCleanerManager.scala's abortAndPauseCleaning(). I think we > should not throw exception if the state is already LogCleaningAborted or > LogCleaningPaused in this function, but instead just let it roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414500#comment-15414500 ] Soumyajit Sahu commented on KAFKA-3123: --- I hit another (similar) exception while trying out Kafka from 0.10.0.1. I think abortAndPauseCleaning() should be a no-op if the state is already LogCleaningPaused. I will create a PR. java.lang.IllegalStateException: Compaction for partition [simplestress_5,7] cannot be aborted and paused since it is in LogCleaningPaused state. at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231) at kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:148) at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:307) at kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:218) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:157) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:127) at scala.Option.foreach(Option.scala:257) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:127) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:125) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) at scala.collection.mutable.HashMap.foreach(HashMap.scala:99) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:125) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:125) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:125) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:123) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > Follower Broker cannot start if offsets are already out of range > > > Key: KAFKA-3123 > URL: https://issues.apache.org/jira/browse/KAFKA-3123 > Project: Kafka > Issue Type: Bug > Components: core, replication >Affects Versions: 0.9.0.0 >Reporter: Soumyajit Sahu >Assignee: Neha Narkhede >Priority: Critical > Labels: patch > Fix For: 0.10.0.2 > > Attachments: > 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch > > > I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one > machine at a time. Our logs have just 2 hours of retention. I had re-imaged > the test machine under consideration, and got the following error in loop > after starting afresh with 0.9.0 broker: > [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica > 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to > current leader 169595708's start offset 334086 > (kafka.server.ReplicaFetcherThread) > [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error > getting offset for partition [EventLogs4,1] to broker 169595708 > (kafka.server.ReplicaFetcherThread) > java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] > cannot be aborted and paused since it is in LogCleaningPaused state. > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.log.LogCleanerManager.abort
[jira] [Closed] (KAFKA-3448) Support zone index in IPv6 regex
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu closed KAFKA-3448. - > Support zone index in IPv6 regex > > > Key: KAFKA-3448 > URL: https://issues.apache.org/jira/browse/KAFKA-3448 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 > Environment: Windows,Linux >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu > Fix For: 0.10.0.0 > > > When an address is written textually, the zone index is appended to the > address, separated by a percent sign (%). The actual syntax of zone indices > depends on the operating system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KAFKA-2305) Cluster Monitoring and Management UI
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu closed KAFKA-2305. - > Cluster Monitoring and Management UI > > > Key: KAFKA-2305 > URL: https://issues.apache.org/jira/browse/KAFKA-2305 > Project: Kafka > Issue Type: Wish > Components: admin, replication >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu >Priority: Minor > > At present, we don't have a Admin and Monitoring UI for Kafka cluster. > We need a view from the perspective of the machines in the cluster. > Following issues need to be addressed using a UI: > 1) Resource usage and Kafka statistics by machine. > 2) View of the partition and replication layout by machine. > 3) View of spindle usage (or different log directories usage) pattern by > machine. > 4) Ability to move replicas among brokers using the UI and by leveraging the > Reassign Partitions Tool. > More details in the doc in the External Issue Url field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KAFKA-2305) Cluster Monitoring and Management UI
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu resolved KAFKA-2305. --- Resolution: Invalid > Cluster Monitoring and Management UI > > > Key: KAFKA-2305 > URL: https://issues.apache.org/jira/browse/KAFKA-2305 > Project: Kafka > Issue Type: Wish > Components: admin, replication >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu >Priority: Minor > > At present, we don't have a Admin and Monitoring UI for Kafka cluster. > We need a view from the perspective of the machines in the cluster. > Following issues need to be addressed using a UI: > 1) Resource usage and Kafka statistics by machine. > 2) View of the partition and replication layout by machine. > 3) View of spindle usage (or different log directories usage) pattern by > machine. > 4) Ability to move replicas among brokers using the UI and by leveraging the > Reassign Partitions Tool. > More details in the doc in the External Issue Url field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2305) Cluster Monitoring and Management UI
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-2305: - Assignee: Soumyajit Sahu (was: Neha Narkhede) > Cluster Monitoring and Management UI > > > Key: KAFKA-2305 > URL: https://issues.apache.org/jira/browse/KAFKA-2305 > Project: Kafka > Issue Type: Wish > Components: admin, replication >Reporter: Soumyajit Sahu >Assignee: Soumyajit Sahu >Priority: Minor > > At present, we don't have a Admin and Monitoring UI for Kafka cluster. > We need a view from the perspective of the machines in the cluster. > Following issues need to be addressed using a UI: > 1) Resource usage and Kafka statistics by machine. > 2) View of the partition and replication layout by machine. > 3) View of spindle usage (or different log directories usage) pattern by > machine. > 4) Ability to move replicas among brokers using the UI and by leveraging the > Reassign Partitions Tool. > More details in the doc in the External Issue Url field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3448) Support zone index in IPv6 regex
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Summary: Support zone index in IPv6 regex (was: IPV6 Regex is missing % character) > Support zone index in IPv6 regex > > > Key: KAFKA-3448 > URL: https://issues.apache.org/jira/browse/KAFKA-3448 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 > Environment: Windows,Linux >Reporter: Soumyajit Sahu > Fix For: 0.10.1.0 > > > When an address is written textually, the zone index is appended to the > address, separated by a percent sign (%). The actual syntax of zone indices > depends on the operating system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3448) IPV6 Regex is missing % character
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Description: When an address is written textually, the zone index is appended to the address, separated by a percent sign (%). The actual syntax of zone indices depends on the operating system. (was: IPV6 addresses could have the % character. When an address is written textually, the zone index is appended to the address, separated by a percent sign (%). Reference: https://en.wikipedia.org/wiki/IPv6_address Example: Link-local IPv6 Address . . . . . : fe80::b1da:69ca:57f7:63d8%3(Preferred) Then, the broker would throw the IllegalStateException(s"connectionId has unexpected format: $connectionId")) > IPV6 Regex is missing % character > - > > Key: KAFKA-3448 > URL: https://issues.apache.org/jira/browse/KAFKA-3448 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 > Environment: Windows,Linux >Reporter: Soumyajit Sahu > Fix For: 0.10.1.0 > > > When an address is written textually, the zone index is appended to the > address, separated by a percent sign (%). The actual syntax of zone indices > depends on the operating system. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3448) IPV6 Regex is missing % character
[ https://issues.apache.org/jira/browse/KAFKA-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3448: -- Fix Version/s: 0.10.1.0 Status: Patch Available (was: Open) > IPV6 Regex is missing % character > - > > Key: KAFKA-3448 > URL: https://issues.apache.org/jira/browse/KAFKA-3448 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 > Environment: Windows,Linux >Reporter: Soumyajit Sahu > Fix For: 0.10.1.0 > > > IPV6 addresses could have the % character. > When an address is written textually, the zone index is appended to the > address, separated by a percent sign (%). > Reference: https://en.wikipedia.org/wiki/IPv6_address > Example: Link-local IPv6 Address . . . . . : > fe80::b1da:69ca:57f7:63d8%3(Preferred) > Then, the broker would throw the IllegalStateException(s"connectionId has > unexpected format: $connectionId") -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3448) IPV6 Regex is missing % character
Soumyajit Sahu created KAFKA-3448: - Summary: IPV6 Regex is missing % character Key: KAFKA-3448 URL: https://issues.apache.org/jira/browse/KAFKA-3448 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.9.0.0 Environment: Windows,Linux Reporter: Soumyajit Sahu IPV6 addresses could have the % character. When an address is written textually, the zone index is appended to the address, separated by a percent sign (%). Reference: https://en.wikipedia.org/wiki/IPv6_address Example: Link-local IPv6 Address . . . . . : fe80::b1da:69ca:57f7:63d8%3(Preferred) Then, the broker would throw the IllegalStateException(s"connectionId has unexpected format: $connectionId") -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3123: -- Attachment: 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch Submitting a patch for the code change I have mentioned in the Description > Follower Broker cannot start if offsets are already out of range > > > Key: KAFKA-3123 > URL: https://issues.apache.org/jira/browse/KAFKA-3123 > Project: Kafka > Issue Type: Bug > Components: core, replication >Affects Versions: 0.9.0.0 >Reporter: Soumyajit Sahu >Assignee: Neha Narkhede > Labels: patch > Attachments: > 0001-Fix-Follower-crashes-when-offset-out-of-range-during.patch > > > I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one > machine at a time. Our logs have just 2 hours of retention. I had re-imaged > the test machine under consideration, and got the following error in loop > after starting afresh with 0.9.0 broker: > [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica > 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to > current leader 169595708's start offset 334086 > (kafka.server.ReplicaFetcherThread) > [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error > getting offset for partition [EventLogs4,1] to broker 169595708 > (kafka.server.ReplicaFetcherThread) > java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] > cannot be aborted and paused since it is in LogCleaningPaused state. > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) > at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:141) > at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:304) > at > kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:185) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:152) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:122) > at scala.Option.foreach(Option.scala:236) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:122) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:120) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > I could unblock myself with a code change. I deleted the action for "case s > =>" in the LogCleanerManager.scala's abortAndPauseCleaning(). I think we > should not throw exception if the state is already LogCleaningAborted or > LogCleaningPaused in this function, but instead just let it roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
[ https://issues.apache.org/jira/browse/KAFKA-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu updated KAFKA-3123: -- Labels: patch (was: ) Status: Patch Available (was: Open) Submitting a patch for the code change I have mentioned in the Description > Follower Broker cannot start if offsets are already out of range > > > Key: KAFKA-3123 > URL: https://issues.apache.org/jira/browse/KAFKA-3123 > Project: Kafka > Issue Type: Bug > Components: core, replication >Affects Versions: 0.9.0.0 >Reporter: Soumyajit Sahu >Assignee: Neha Narkhede > Labels: patch > > I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one > machine at a time. Our logs have just 2 hours of retention. I had re-imaged > the test machine under consideration, and got the following error in loop > after starting afresh with 0.9.0 broker: > [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica > 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to > current leader 169595708's start offset 334086 > (kafka.server.ReplicaFetcherThread) > [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error > getting offset for partition [EventLogs4,1] to broker 169595708 > (kafka.server.ReplicaFetcherThread) > java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] > cannot be aborted and paused since it is in LogCleaningPaused state. > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at > kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) > at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:141) > at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:304) > at > kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:185) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:152) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:122) > at scala.Option.foreach(Option.scala:236) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:122) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:120) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > I could unblock myself with a code change. I deleted the action for "case s > =>" in the LogCleanerManager.scala's abortAndPauseCleaning(). I think we > should not throw exception if the state is already LogCleaningAborted or > LogCleaningPaused in this function, but instead just let it roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3123) Follower Broker cannot start if offsets are already out of range
Soumyajit Sahu created KAFKA-3123: - Summary: Follower Broker cannot start if offsets are already out of range Key: KAFKA-3123 URL: https://issues.apache.org/jira/browse/KAFKA-3123 Project: Kafka Issue Type: Bug Components: core, replication Affects Versions: 0.9.0.0 Reporter: Soumyajit Sahu Assignee: Neha Narkhede I was trying to upgrade our test Windows cluster from 0.8.1.1 to 0.9.0 one machine at a time. Our logs have just 2 hours of retention. I had re-imaged the test machine under consideration, and got the following error in loop after starting afresh with 0.9.0 broker: [2016-01-19 13:57:28,809] WARN [ReplicaFetcherThread-1-169595708], Replica 15588 for partition [EventLogs4,1] reset its fetch offset from 0 to current leader 169595708's start offset 334086 (kafka.server.ReplicaFetcherThread) [2016-01-19 13:57:28,809] ERROR [ReplicaFetcherThread-1-169595708], Error getting offset for partition [EventLogs4,1] to broker 169595708 (kafka.server.ReplicaFetcherThread) java.lang.IllegalStateException: Compaction for partition [EXO_EventLogs4,1] cannot be aborted and paused since it is in LogCleaningPaused state. at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply$mcV$sp(LogCleanerManager.scala:149) at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) at kafka.log.LogCleanerManager$$anonfun$abortAndPauseCleaning$1.apply(LogCleanerManager.scala:140) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) at kafka.log.LogCleanerManager.abortAndPauseCleaning(LogCleanerManager.scala:140) at kafka.log.LogCleaner.abortAndPauseCleaning(LogCleaner.scala:141) at kafka.log.LogManager.truncateFullyAndStartAt(LogManager.scala:304) at kafka.server.ReplicaFetcherThread.handleOffsetOutOfRange(ReplicaFetcherThread.scala:185) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:152) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$2.apply(AbstractFetcherThread.scala:122) at scala.Option.foreach(Option.scala:236) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:122) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2$$anonfun$apply$mcV$sp$1.apply(AbstractFetcherThread.scala:120) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply$mcV$sp(AbstractFetcherThread.scala:120) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$2.apply(AbstractFetcherThread.scala:120) at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) I could unblock myself with a code change. I deleted the action for "case s =>" in the LogCleanerManager.scala's abortAndPauseCleaning(). I think we should not throw exception if the state is already LogCleaningAborted or LogCleaningPaused in this function, but instead just let it roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2305) Cluster Monitoring and Management UI
Soumyajit Sahu created KAFKA-2305: - Summary: Cluster Monitoring and Management UI Key: KAFKA-2305 URL: https://issues.apache.org/jira/browse/KAFKA-2305 Project: Kafka Issue Type: Wish Components: admin, replication Reporter: Soumyajit Sahu Assignee: Neha Narkhede Priority: Minor At present, we don't have a Admin and Monitoring UI for Kafka cluster. We need a view from the perspective of the machines in the cluster. Following issues need to be addressed using a UI: 1) Resource usage and Kafka statistics by machine. 2) View of the partition and replication layout by machine. 3) View of spindle usage (or different log directories usage) pattern by machine. 4) Ability to move replicas among brokers using the UI and by leveraging the Reassign Partitions Tool. More details in the doc in the External Issue Url field. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
