[jira] [Created] (KAFKA-12981) Ensure LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are read/updated in sync
Tom Scott created KAFKA-12981: - Summary: Ensure LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are read/updated in sync Key: KAFKA-12981 URL: https://issues.apache.org/jira/browse/KAFKA-12981 Project: Kafka Issue Type: Sub-task Reporter: Tom Scott KIP-734 extends listOffsetRequest to fetch offsets by max timestamp as well as by start and end offset. This relies on LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar but there is currently no synchronisation between the 2 meaning that one could be updated whilst the other is being read. This ticket ensure that LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are locked on read to ensure synchronisation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-12541) AdminClient.listOffsets should return the offset for the record with highest timestamp
Tom Scott created KAFKA-12541:
-
Summary: AdminClient.listOffsets should return the offset for the
record with highest timestamp
Key: KAFKA-12541
URL: https://issues.apache.org/jira/browse/KAFKA-12541
Project: Kafka
Issue Type: Improvement
Components: admin
Reporter: Tom Scott
In Kafka 2.7 the following method was added to AdminClient that provides this
information:
{panel}
{panel}
|{{public}} {{ListOffsetsResult listOffsets(Map
topicPartitionOffsets,}}
{{ }}{{ListOffsetsOptions options)}}|
[https://kafka.apache.org/27/javadoc/org/apache/kafka/clients/admin/KafkaAdminClient.html#listOffsets-java.util.Map-org.apache.kafka.clients.admin.ListOffsetsOptions-]
where OffsetSpec can be:
* OffsetSpec.EarliestSpec
* OffsetSpec.LatestSpec
* OffsetSpec.TimestampSpec
This ticket introduces a new spec:
{panel}
{panel}
|{{OffsetSpec.MaxTimestampSpec }}{{// this returns the offset and timestamp for
the record with the highest timestamp.}}|
This indicates to the AdminClient that we want to fetch the timestamp and
offset for the record with the largest timestamp produced to a partition.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KAFKA-12234) Extend OffsetFetch requests to accept multiple group ids.
Tom Scott created KAFKA-12234: - Summary: Extend OffsetFetch requests to accept multiple group ids. Key: KAFKA-12234 URL: https://issues.apache.org/jira/browse/KAFKA-12234 Project: Kafka Issue Type: Improvement Affects Versions: 2.8.0 Reporter: Tom Scott More details are in the KIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173084258 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-7746) sasl.jaas.config dynamic broker configuration does not accept "=" in value
Tom Scott created KAFKA-7746:
Summary: sasl.jaas.config dynamic broker configuration does not
accept "=" in value
Key: KAFKA-7746
URL: https://issues.apache.org/jira/browse/KAFKA-7746
Project: Kafka
Issue Type: Bug
Components: config, security
Reporter: Tom Scott
In KIP-226 it give an example of setting sasl.jaas.config using dynmaic broker
configuration:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration#KIP-226-DynamicBrokerConfiguration-NewBrokerConfigurationOption]
However, as most SASL module configurations contain the "=" symbol this ends up
with the error:
{code:java}
requirement failed: Invalid entity config: all configs to be added must be in
the format “key=val”.{code}
I have tried various escape sequences but have not so far been successful.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KAFKA-6314) Add a tool to delete consumer offsets for a given group
Tom Scott created KAFKA-6314: Summary: Add a tool to delete consumer offsets for a given group Key: KAFKA-6314 URL: https://issues.apache.org/jira/browse/KAFKA-6314 Project: Kafka Issue Type: New Feature Components: consumer, core, tools Reporter: Tom Scott Priority: Minor Add a tool to delete consumer offsets for a given group similar to the reset tool. It could look something like this: kafka-consumer-groups --bootstrap-server localhost:9092 --delete-offsets --group somegroup The case for this is as follows: 1. Consumer group with id: group1 subscribes to topic1 2. The group is stopped 3. The subscription changed to topic2 but the id is kept as group1 Now the out output of kafka-consumer-groups --describe for the group will show topic1 even though the group is not subscribed to that topic. This is bad for monitoring as it will show lag on topic1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
