[
https://issues.apache.org/jira/browse/KAFKA-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mickael Maison updated KAFKA-8564:
----------------------------------
Description:
If brokers restart when topics are being deleted, it's possible to end up with
a partition folder with the deleted suffix but without any log segments:
{quote}ls -la ./kafka-logs/3part3rep5-1.f2ce83b86df9416abe50d2e2299009c2-delete/
total 8
drwxr-xr-x@ 4 mickael staff 128 6 Jun 14:35 .
drwxr-xr-x@ 61 mickael staff 1952 6 Jun 14:35 ..
-rw-r--r--@ 1 mickael staff 10 6 Jun 14:32 00000000000023261863.snapshot
-rw-r--r--@ 1 mickael staff 0 6 Jun 14:35 leader-epoch-checkpoint
{quote}
>From 2.2.1, brokers fail to start when loading such folders:
{quote}[2019-06-19 09:40:48,123] ERROR There was an error in one of the threads
during logs loading: java.lang.NullPointerException (kafka.log.LogManager)
[2019-06-19 09:40:48,126] ERROR [KafkaServer id=1] Fatal error during
KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.NullPointerException
at kafka.log.Log.activeSegment(Log.scala:1896)
at kafka.log.Log.<init>(Log.scala:295)
at kafka.log.Log$.apply(Log.scala:2186)
at kafka.log.LogManager.loadLog(LogManager.scala:275)
at kafka.log.LogManager.$anonfun$loadLogs$12(LogManager.scala:345)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{quote}
With 2.2.0, upon loading such folders, brokers create a new empty log segment
and load that successfully.
The change of behaviour was introduced in
[https://github.com/apache/kafka/commit/f000dab5442ce49c4852823c257b4fb0cdfe15aa]
was:
If brokers restart when topics are being deleted, it's possible to end up with
a partition folder with the deleted suffix but without any log segments:
{quote}{{ls -la
./kafka-logs/3part3rep5-1.f2ce83b86df9416abe50d2e2299009c2-delete/}}
{{ total 8}}
{{ drwxr-xr-x@ 4 mickael staff 128 6 Jun 14:35 .}}
{{ drwxr-xr-x@ 61 mickael staff 1952 6 Jun 14:35 ..}}
{{ -rw-r--r--@ 1 mickael staff 10 6 Jun 14:32 00000000000023261863.snapshot}}
{{ -rw-r--r--@ 1 mickael staff 0 6 Jun 14:35 leader-epoch-checkpoint}}
{quote}
>From 2.2.1, brokers fail to start when loading such folders:
{quote}[2019-06-19 09:40:48,123] ERROR There was an error in one of the threads
during logs loading: java.lang.NullPointerException (kafka.log.LogManager)
[2019-06-19 09:40:48,126] ERROR [KafkaServer id=1] Fatal error during
KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.NullPointerException
at kafka.log.Log.activeSegment(Log.scala:1896)
at kafka.log.Log.<init>(Log.scala:295)
at kafka.log.Log$.apply(Log.scala:2186)
at kafka.log.LogManager.loadLog(LogManager.scala:275)
at kafka.log.LogManager.$anonfun$loadLogs$12(LogManager.scala:345)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{quote}
With 2.2.0, upon loading such folders, brokers create a new empty log segment
and load that successfully.
The change of behaviour was introduced in
[https://github.com/apache/kafka/commit/f000dab5442ce49c4852823c257b4fb0cdfe15aa]
> NullPointerException when loading logs at startup
> -------------------------------------------------
>
> Key: KAFKA-8564
> URL: https://issues.apache.org/jira/browse/KAFKA-8564
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 2.3.0, 2.2.1
> Reporter: Mickael Maison
> Priority: Blocker
>
> If brokers restart when topics are being deleted, it's possible to end up
> with a partition folder with the deleted suffix but without any log segments:
> {quote}ls -la
> ./kafka-logs/3part3rep5-1.f2ce83b86df9416abe50d2e2299009c2-delete/
> total 8
> drwxr-xr-x@ 4 mickael staff 128 6 Jun 14:35 .
> drwxr-xr-x@ 61 mickael staff 1952 6 Jun 14:35 ..
> -rw-r--r--@ 1 mickael staff 10 6 Jun 14:32 00000000000023261863.snapshot
> -rw-r--r--@ 1 mickael staff 0 6 Jun 14:35 leader-epoch-checkpoint
> {quote}
> From 2.2.1, brokers fail to start when loading such folders:
> {quote}[2019-06-19 09:40:48,123] ERROR There was an error in one of the
> threads during logs loading: java.lang.NullPointerException
> (kafka.log.LogManager)
> [2019-06-19 09:40:48,126] ERROR [KafkaServer id=1] Fatal error during
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.NullPointerException
> at kafka.log.Log.activeSegment(Log.scala:1896)
> at kafka.log.Log.<init>(Log.scala:295)
> at kafka.log.Log$.apply(Log.scala:2186)
> at kafka.log.LogManager.loadLog(LogManager.scala:275)
> at kafka.log.LogManager.$anonfun$loadLogs$12(LogManager.scala:345)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {quote}
> With 2.2.0, upon loading such folders, brokers create a new empty log segment
> and load that successfully.
> The change of behaviour was introduced in
> [https://github.com/apache/kafka/commit/f000dab5442ce49c4852823c257b4fb0cdfe15aa]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
