I dug further into this and log.cleaner.enable is false by default in
0.8.2.x which pretty much explains all this. Fixing it now.
Thanks!
On 09.05.2016 08:17, Jörg Wagner wrote:
Thank you James.
The log cleaner log is empty and we didn't notice any issue there. The
Topic however is ~174GB per Node (3) which would support your
information about the log cleaner dying.
Since this is a production system I do not think debug logging is
viable even for a short time. Could you maybe point me in the
direction of debugging compaction where it went wrong?
Thanks
Jörg
On 06.05.2016 22:21, James Cheng wrote:
Is the log compaction thread correctly working? The offsets are
stored in a log compacted topic, and we have seen issues where the
log cleaner thread dies and therefore the offsets topic just grows
forever, which means it will take a long time to read in the topic.
You can look in the log-cleaner.log debuglog file to see if there are
any error messages there.
-James
On May 6, 2016, at 6:28 AM, Jörg Wagner <joerg.wagn...@1und1.de> wrote:
After a bit more looking into this we found out that the
Offsetmanager is single threaded and due to our setup (few, powerful
servers: rather bad for kafka I know..) it seems we are limiting
ourselves severely by using kafka offsets.
Any more insight is still welcome of course.
-------- Forwarded Message --------
Subject: unknown (kafka) offsets after restart
Date: Fri, 6 May 2016 14:12:24 +0200
From: Jörg Wagner <joerg.wagn...@1und1.de>
Reply-To: users@kafka.apache.org
To: users@kafka.apache.org
We're using Kafka 0.8.2 and are puzzled by the offset behaviour when
they are stored in kafka topics.
Upon restart of the Kafka cluster (e.g. due to reconfiguration) it can
happen that the offsets are unknown and therefore stop consumers from
consuming without knowing their offset.
kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group group
--zookeeper localhost:2181
Could not fetch offset for [topic,19] due to
kafka.common.OffsetsLoadInProgressException.
Group Topic Pid Offset
logSize Lag Owner
group topic 0 unknown 326606 unknown none
I currently have no other solution to this than to wait.. but it
takes a
very long time (hours.. the servers are hopelessly underutilized!), I
would be grateful for any advice.
Thanks
Jörg
--
Mit freundlichem Gruß
Jörg Wagner
Systemadministrator
Search & Account Security
1&1 Mail & Media Development & Technology GmbH | Sapporobogen 6-8 | 80637
München | Germany
Phone: +49 89 14339 324
E-Mail: joerg.wagn...@1und1.de | Web: www.1und1.de
Amtsgericht Montabaur, HRB 5452
Geschäftsführer: Frank Einhellinger, Thomas Ludwig, Jan Oetjen
Member of United Internet
