There are two cases that zero-copy fetch thanks to sendfile don't work.
- SSL encryption is enabled
* Need to encrypt on Kafka process before sending to client
-
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/network/SslTransportLayer.java#L946
Hi.
One possible solution I can imagine is to use replication throttle.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas
You can set lower replication quota for other brokers -> failed broker
traffic during catching-up to prevent exhausting storage throughput.
ault auto.offset.reset setting to
> the earliest.
> The auto.offset.reset setting starts before kafka V1.0, which means, there
> are already a lot of users using it and get used to it now.
>
> Thanks.
> Luke
>
> On Fri, Feb 19, 2021 at 2:24 PM Haruki Okada wrote:
>
> &g
Hi, Kafka.
Recently I noticed that splitting partition may cause message delivery loss
for consumers with auto.offset.reset=latest.
I described details in https://issues.apache.org/jira/browse/KAFKA-12261 .
Since delivery loss is undesired in most cases, I think this should be
described in
Xiangyuan LI :
> Hi Haruki Okada:
> i read your comment, thx for your detail explain!
> add backlog parameter is a useful suggestion, hope it could added to
> kafka.
>
> Haruki Okada 于2021年8月2日周一 上午7:43写道：
>
> > Hi, Kafka.
> >
> > I would like to star
property. I think we
> should name the property socket.listen.backlog.size for better clarity on
> what the property configures. Besides that, the proposal looks good to me.
>
> David
>
> On Wed, Jul 28, 2021 at 8:09 AM Haruki Okada wrote:
>
> > Hi, Kafka.
> >
>
Hi, Kafka.
I would like to start a vote on KIP that makes SocketServer acceptor's
backlog size configurable.
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-764%3A+Configurable+backlog+size+for+creating+Acceptor
Discussion thread:
Currently, KIP-764 got
+1 binding
+2 non-binding
votes. Could anyone help checking the KIP and join the vote?
Thanks,
2021年10月18日(月) 18:21 Haruki Okada :
> Hi Lucas.
>
> Thanks for the vote.
> As far as I understand, it should be enough to adjust `net.core.somaxconn`
>
+1 (binding). Thanks for the KIP!
>
> Best,
> David
>
> On Wed, Oct 20, 2021 at 12:09 PM Haruki Okada wrote:
>
> > Currently, KIP-764 got
> > +1 binding
> > +2 non-binding
> >
> > votes. Could anyone help checking the KIP and join the vote?
> &
(月) 11:25 Haruki Okada :
> Thanks for your comment LI-san.
>
> Could anyone else review and vote for the KIP?
>
> I think the situation described in the KIP's motivation can happen in any
> large-scale Kafka deployment, so may be helpful for many users while the
> proposed cha
don't think everyone is familiar with the definition of the method
> parameters.
>
> Thank you.
> Luke
>
> On Mon, Oct 18, 2021 at 3:43 PM Haruki Okada wrote:
>
> > Hi Kafka.
> >
> > Let me bump this VOTE thread for the KIP.
> > We applied proposed changes i
ented in the config documentation
> > (e.g. net.core.somaxconn)?
> >
> >
> > On Mon, Oct 18, 2021 at 4:23 PM Haruki Okada
> wrote:
> >
> >> Hi Luke.
> >>
> >> Thank you for the vote.
> >> Updated KIP to link to ServerSocket#bind java
Hi, Dongjin,
Sorry for interrupting the discussion.
And thank you for your hard work about KIP-653, KIP-719.
I understand that KIP-653 is already accepted so log4j2 is the choice of
the Kafka community though, I'm now feeling that logback is a better choice
here.
Reasons:
- even after
this, so it requires analyzing the implementation itself.
> (what I actually did with log4j2) About c, logback does not provide a Kafka
> Appender so we have to provide an equivalent.
>
>
> It is why I prefer to use log4j2. How do you think?
>
>
> Thanks,
>
> Dongj
Hi Bill.
Thank you for your quick response!
2021年7月22日(木) 7:40 Bill Bejeck :
> Hi Haruki,
>
> You're all set with the wiki permissions now.
>
> Regards,
> Bill
>
> On Wed, Jul 21, 2021 at 1:43 PM Haruki Okada wrote:
>
> > Hi, Kafka.
> >
> > I woul
Hi, Kafka.
I would like to request permission for KIP creation under
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
to propose a config-addition related to
https://issues.apache.org/jira/browse/KAFKA-9648 .
JIRA id: ocadaruma
email: ocadar...@gmail.com
Thanks,
Hi, Kafka.
I proposed KIP-764, which tries to add new KafkaConfig to adjust Acceptor's
backlog size.
As described in the KIP and the ticket KAFKA-9648, currently backlog size
is fixed value (50) and it may not be enough to handle incoming connections
from massive clients.
So we would like to
Hi, Kafka.
Does anyone have any thoughts or suggestions about the KIP?
If there seems to be no, I would like to start a VOTE later.
Thanks,
2021年7月22日(木) 16:17 Haruki Okada :
> Hi, Kafka.
>
> I proposed KIP-764, which tries to add new KafkaConfig to adjust
> Acceptor's
Hi Bruno,
Thank you for driving the release !
Can we add KIP-764 to the plan? (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-764%3A+Configurable+backlog+size+for+creating+Acceptor
)
Thanks,
2022年2月16日(水) 18:22 Bruno Cadonna :
> Hi Julien,
>
> Thank you for the feedback on the release
Hi Jack,
Thank you for the great work, not only the spec but also for the
comprehensive documentation about the replication.
Actually I wrote some TLA+ spec to verify unclean leader election behavior
before so I will double-check my understanding with your complete spec :)
Thanks,
Hi!
I found that ReplicaFetcherThreadBenchmark (and other few benchmarks) is
not working now and reported as KAFKA-15567.
I submitted a patch for that in https://github.com/apache/kafka/pull/14513.
Could anyone review the PR?
Thanks,
--
Okada Haruki
Hi, Kafka.
We found that the race in topic-deletion procedure could cause the
preferred replica imbalance metric to be negative.
The phenomenon can easily happen when many topics are deleted at once, and
since we use the metric for monitoring, we have to restart the controller
to fix the metric
Hi.
That also sounds good to me.
TLA+ spec helps us to understand the protocol from a high-level.
Also I agree with Tim's point that the spread of familiarity with
formal methods would be beneficial in the long term.
2022年7月26日(火) 23:58 Tom Bentley :
> Hi,
>
> I noticed that TLA+ has featured
You can just use Producer#partitionsFor
2023年3月9日(木) 11:13 jian fu :
> Hi All:
>
> If anyone can help to give some comments or suggestions for the proposal.
> Thanks in advance!
>
> Regards
> Jian
>
> jian fu 于2023年3月6日周一 17:00写道：
>
> > Hi Everyone:
> > Nice to meet you.
> >
> > I created one
successful metadata even when it timed
out.
So I'm not sure if it makes sense to add a new API or timeout config (for
KIP-912), considering KafkaProducer already has many timeout parameters so
it could introduce another complexity.
2023年3月9日(木) 12:18 jian fu :
> Hi Haruki Okada:
>
> Ther
Hi, Omnia.
Thanks for the KIP.
The feature sounds indeed helpful and the strategy to use bloom-filter
looks good.
I have three questions:
1. Do you have any memory-footprint estimation
for TimeControlledBloomFilter?
* If I read the KIP correctly, TimeControlledBloomFilter will be
allocated
Hi, Luke.
Though this proposal definitely looks interesting, as others pointed out,
the leader election implementation would be the hard part.
And I think even LEO-based-election is not safe, which could cause silent
committed-data loss easily.
Let's say we have replicas A,B,C and A is the
Hi.
KafkaConsumer can subscribe to topics by pattern:
https://kafka.apache.org/36/javadoc/org/apache/kafka/clients/consumer/Consumer.html#subscribe(java.util.regex.Pattern)
2023年12月1日(金) 22:05 Chunlin Yang :
> Hi team,
>
> My use case is I have a central controller to manage tens of thousands
Hi, Divij.
This isn't about config default value/constraint change though, I found
there's a behavior discrepancy in max.block.ms config, which may cause
breaking change if we change the behavior.
The detail is described in the ticket:
https://issues.apache.org/jira/browse/KAFKA-16372
What do
Haruki Okada created KAFKA-10690:
Summary: Produce-response delay caused by lagging replica fetch
which blocks in-sync one
Key: KAFKA-10690
URL: https://issues.apache.org/jira/browse/KAFKA-10690
Haruki Okada created KAFKA-12261:
Summary: auto.offset.reset should be earliest by default
Key: KAFKA-12261
URL: https://issues.apache.org/jira/browse/KAFKA-12261
Project: Kafka
Issue Type
Haruki Okada created KAFKA-13403:
Summary: KafkaServer crashes when deleting topics due to the race
in log deletion
Key: KAFKA-13403
URL: https://issues.apache.org/jira/browse/KAFKA-13403
Project
Haruki Okada created KAFKA-15567:
Summary: ReplicaFetcherThreadBenchmark is not working
Key: KAFKA-15567
URL: https://issues.apache.org/jira/browse/KAFKA-15567
Project: Kafka
Issue Type
Haruki Okada created KAFKA-14445:
Summary: Producer doesn't request metadata update on
REQUEST_TIMED_OUT
Key: KAFKA-14445
URL: https://issues.apache.org/jira/browse/KAFKA-14445
Project: Kafka
Haruki Okada created KAFKA-15046:
Summary: Produce performance issue under high disk load
Key: KAFKA-15046
URL: https://issues.apache.org/jira/browse/KAFKA-15046
Project: Kafka
Issue Type
Haruki Okada created KAFKA-15917:
Summary: Flaky test - OffsetsApiIntegrationTest.
testAlterSinkConnectorOffsetsZombieSinkTasks
Key: KAFKA-15917
URL: https://issues.apache.org/jira/browse/KAFKA-15917
Haruki Okada created KAFKA-15918:
Summary: Flaky test - OffsetsApiIntegrationTest.
testResetSinkConnectorOffsets
Key: KAFKA-15918
URL: https://issues.apache.org/jira/browse/KAFKA-15918
Project: Kafka
Haruki Okada created KAFKA-15919:
Summary: Flaky test -
BrokerLifecycleManagerTest.testAlwaysSendsAccumulatedOfflineDirs
Key: KAFKA-15919
URL: https://issues.apache.org/jira/browse/KAFKA-15919
Haruki Okada created KAFKA-15920:
Summary: Flaky test - PlaintextConsumerTest.testCoordinatorFailover
Key: KAFKA-15920
URL: https://issues.apache.org/jira/browse/KAFKA-15920
Project: Kafka
Haruki Okada created KAFKA-15921:
Summary: Flaky test -
SaslScramSslEndToEndAuthorizationTest.testAuthentications
Key: KAFKA-15921
URL: https://issues.apache.org/jira/browse/KAFKA-15921
Project
Haruki Okada created KAFKA-15924:
Summary: Flaky test -
QuorumControllerTest.testFatalMetadataReplayErrorOnActive
Key: KAFKA-15924
URL: https://issues.apache.org/jira/browse/KAFKA-15924
Project
Haruki Okada created KAFKA-16372:
Summary: max.block.ms behavior inconsistency with javadoc and the
config description
Key: KAFKA-16372
URL: https://issues.apache.org/jira/browse/KAFKA-16372
Project
Haruki Okada created KAFKA-16393:
Summary: SslTransportLayer doesn't implement write(ByteBuffer[],
int, int) correctly
Key: KAFKA-16393
URL: https://issues.apache.org/jira/browse/KAFKA-16393
Project
Haruki Okada created KAFKA-16541:
Summary: Potential leader epoch checkpoint file corruption on OS
crash
Key: KAFKA-16541
URL: https://issues.apache.org/jira/browse/KAFKA-16541
Project: Kafka
44 matches
Mail list logo
