ang.Double, it would be more
> helpful to say something equivalent, like "the float is serialized as a
> 64-bit IEEE 754 value, ordered as big-endian." That way people who are
> developing clients in other languages can more easily understand the
> protocol.
> >
> >
[
https://issues.apache.org/jira/browse/KAFKA-10158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-10158.
-
Resolution: Fixed
> Fix flaky
> kafka.admin.TopicCommandWithAdminClie
[
https://issues.apache.org/jira/browse/KAFKA-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-10278.
-
Resolution: Won't Fix
Hi Kaushik,
The command is set to only return modified (non-default
My apologies, the command mentioned should include --all:
kafka-configs ... --entity-type brokers --entity-name 1 --describe --all
| grep DYNAMIC
Brian
On Fri, Jun 12, 2020 at 8:45 AM Brian Byrne wrote:
> Hi Nag,
>
> Correct, --all will include both. Removing --all should give
those were overridden . Is there any command if i wanted to query
> only the overridden values and the rest will be default values
>
> On Fri, Jun 12, 2020 at 7:58 PM Brian Byrne wrote:
>
> > Hi Nag,
> >
> > To address (2) first, the --entity-default flag requests the
Hi Nag,
To address (2) first, the --entity-default flag requests the default
configuration that all brokers inherit. An individual broker may override
any of the default config's entries, which is done by specifying the broker
ID to the --entity-name flag.
The reason you're getting blank output
[
https://issues.apache.org/jira/browse/KAFKA-9139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-9139.
Resolution: Duplicate
> Dynamic broker config types aren't being discove
[
https://issues.apache.org/jira/browse/KAFKA-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-9713.
Resolution: Duplicate
> Remove BufferExhausedExcept
+1 (non-binding) - thanks!
My only suggestion would be to make the enum-to-int conversion explicit for
the new ConfigType, with a surrounding comment, to ensure that no
accidental reordering and for easier readability should the response
message message be read.
Brian
On Fri, Mar 20, 2020 at
Brian Byrne created KAFKA-9713:
--
Summary: Remove BufferExhausedException
Key: KAFKA-9713
URL: https://issues.apache.org/jira/browse/KAFKA-9713
Project: Kafka
Issue Type: Task
[
https://issues.apache.org/jira/browse/KAFKA-8904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-8904.
Fix Version/s: 2.5.0
Reviewer: Rajini Sivaram
Resolution: Fixed
> Reduce metad
[
https://issues.apache.org/jira/browse/KAFKA-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-8623.
Fix Version/s: 2.3.0
Resolution: Fixed
This appears to be due to an issue concerning
Brian Byrne created KAFKA-9510:
--
Summary: Quotas may resolve to incorrect value if user is empty
Key: KAFKA-9510
URL: https://issues.apache.org/jira/browse/KAFKA-9510
Project: Kafka
Issue Type
wrote:
>
> > +1 (binding)
> >
> > Thanks for the KIP, Brian!
> >
> > Regards,
> >
> > Rajini
> >
> > On Mon, Jan 27, 2020 at 10:35 PM Colin McCabe
> wrote:
> >
> > > Thanks, Brian.
> > >
> > > +1 (bi
r the KIP, Brian!
> >
> > Regards,
> >
> > Rajini
> >
> > On Thu, Jan 23, 2020 at 7:34 PM Jason Gustafson
> > wrote:
> >
> > > Sounds good. +1 from me.
> > >
> > > On Thu, Jan 23, 2020 at 9:00 AM Brian Byrne
> wrote:
> > &g
Hello all,
I'd like to start a vote on KIP-456: Add quota-specific APIs to the Admin
Client, redux
The KIP is here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+quota-specific+APIs+to+the+Admin+Client%2C+redux
The discussion thread is here:
ting specifically for /config/users/”user-1”, which is 200 in this case.
>
> Is my understanding correct?
>
> Thanks,
>
> Anna
>
>
> On Fri, Jan 24, 2020 at 11:32 AM Brian Byrne wrote:
>
> > My apologies, Rajini. My hasty edits omitted a couple spots. I'v
Brian Byrne created KAFKA-9474:
--
Summary: Kafka RPC protocol should support type 'double'
Key: KAFKA-9474
URL: https://issues.apache.org/jira/browse/KAFKA-9474
Project: Kafka
Issue Type
API. if so, we should mention why under
> Rejected Alternatives. We actually use request quotas < 1 in integration
> tests to ensure we can throttle easily.
>
>
>
> On Fri, Jan 24, 2020 at 5:28 PM Brian Byrne wrote:
>
> > Thanks again, Rajini,
> >
> > Units will
gt; value as cluster-wide-bytes-per-second. We could update callbacks to work
> with units, but as you said, it may be better to leave it out initially and
> address later.
>
>
>
> On Thu, Jan 23, 2020 at 6:29 PM Brian Byrne wrote:
>
> > Thanks Rajini,
> >
> > 1
;
>
> Regards,
>
> Rajini
>
>
> On Wed, Jan 22, 2020 at 7:28 PM Brian Byrne wrote:
>
> > Hi Jason,
> >
> > I agree on (1). It was Colin's original suggestion, too, but he had
> changed
> > his mind in preference for enums. Strings are the more ge
ropose we do the synchronous mini batching still but obviously
> > it is already there :) I'm +1 on the current proposal scope.
> >
> > Guozhang
> >
> > On Tue, Jan 21, 2020 at 10:16 AM Brian Byrne
> wrote:
> >
> > > Hi Guozhang,
> > >
> >
[
https://issues.apache.org/jira/browse/KAFKA-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-9082.
Resolution: Duplicate
The outstanding work to be completed is now identical to KAFKA-7740. Marking
need both options. Basically I'm finding the
> "config-centric" mode not very intuitive.
>
> Thanks,
> Jason
>
>
> On Fri, Jan 17, 2020 at 2:14 PM Brian Byrne wrote:
>
> > Thanks Colin, I've updated the KIP with the relevant changes.
> >
> > On
cases
> here, assuming that's was what we've seen in 8904). Since these two steps
> are very light-weighted, doing that in a synchronized block would not hurt
> the concurrency too much.
>
>
> Guozhang
>
>
> On Tue, Jan 21, 2020 at 9:39 AM Brian Byrne wrote:
>
> >
r approach is
> fine too.
>
> c. fixing 3) with a new config, which is relatively orthogonal to a) and
> b).
>
>
>
> Guozhang
>
>
>
>
> On Tue, Jan 14, 2020 at 10:39 AM Brian Byrne wrote:
>
> > Hello all,
> >
> > After further o
rts: the entry, and a list of overridden
entries (where an entry is the value, along with the source). Perhaps the
Value is poorly named, or maybe there's a simpler structure to be had?
Thanks,
Brian
> On Tue, Jan 14, 2020, at 13:32, Brian Byrne wrote:
> > Hi Colin,
> >
> >
Brian Byrne created KAFKA-9449:
--
Summary: Producer's BufferPool may block the producer from closing.
Key: KAFKA-9449
URL: https://issues.apache.org/jira/browse/KAFKA-9449
Project: Kafka
Issue
ser might
provide {type=user, name=x} and it would return all entities that match,
with their resulting quota values? Should I scrap pattern matching for now,
since it's a simple extension that can be done at a future time?
Thanks,
Brian
> On Wed, Dec 11, 2019, at 15:30, Brian Byrne wrote:
> >
have been cleared. My apologies for continually interrupting and
making changes to the KIP, but hopefully this is an agreeable minimum
solution to move forward.
Thanks,
Brian
On Mon, Jan 6, 2020 at 5:23 PM Colin McCabe wrote:
> On Mon, Jan 6, 2020, at 14:40, Brian Byrne wrote:
> >
[
https://issues.apache.org/jira/browse/KAFKA-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-9395.
Assignee: Rajini Sivaram (was: Brian Byrne)
Resolution: Done
> Improve Kafka schedule
Brian Byrne created KAFKA-9395:
--
Summary: Improve Kafka scheduler's periodic maybeShrinkIsr()
Key: KAFKA-9395
URL: https://issues.apache.org/jira/browse/KAFKA-9395
Project: Kafka
Issue Type
Hello all,
Does anyone else have opinions on the issue of RPC frequency? Would it be
better to remove the fetching of non-urgent topics altogether, so that the
refreshes are contained in a larger batch?
Thanks,
Brian
On Mon, Jan 6, 2020 at 2:40 PM Brian Byrne wrote:
>
> So the perfo
, Jan 6, 2020 at 1:31 PM Colin McCabe wrote:
> On Mon, Jan 6, 2020, at 13:07, Brian Byrne wrote:
> > Hi Colin,
> >
> > Thanks again for the feedback!
> >
> > On Mon, Jan 6, 2020 at 12:07 PM Colin McCabe wrote:
> >
> > > Metadata requests don't (a
isn't
necessarily about reducing the total number of RPCs, but rather to
distribute the load more evenly over time. For example, if a large number
of topics need to be refreshed at the approximate same time (common for
startups cases that hit a large number of topics), the updates are more
evenly
ary.
I've updated the KIP to change the default eviction time to 2 minutes.
Thanks,
Brian
On Thu, Jan 2, 2020 at 1:28 PM Guozhang Wang wrote:
> On Thu, Jan 2, 2020 at 12:42 PM Brian Byrne wrote:
>
> > Hi Guozhang,
> >
> > You're correct in that, with the current d
metadata.max.age is P3. Then the optimal
> scenario in terms of metadata update frequency seems to be P1 = P2 < P3. So
> I feel making the default value for metadata.evict.ms smaller is fine,
> e.g.
> say 1minute or 30seconds. WDYT?
>
>
> Guozhang
>
> On Thu, Dec 26
ent
> > metadataUpdater.handleCompletedMetadataResponse
> >
> > so
> > 1. we should also add a metadataUpdater to KafkaProducer?
> > 2. if the topic really does not exists? the intermediate record queue
> will
> > become too large?
> > 3. and
ata.max.age.ms` after this KIP? Are we adding a
> new, more proactive metadata fetch for topics in |U|?
>
> Thanks,
> Stanislav
>
> On Thu, Dec 19, 2019 at 11:37 PM Brian Byrne wrote:
>
> > Hello everyone,
> >
> > For all interested, please take a look at t
nce in practice.
>
> On Mon, Dec 9, 2019 at 2:07 PM Brian Byrne wrote:
>
> > Hi Guozhang,
> >
> > I see, we agree on the topic threshold not applying to urgent topics, but
> > differ slightly on what should be considered urgent. I would argue that
>
Hello all,
I'm reviving the discussion for adding a quotas API to the admin client by
submitting a new proposal. There are some notable changes from previous
attempts, namely a way to deduce the effective quota for a client (entity),
a way to query for configured quotas, and the concept of
but should only be added if there are sending data pending.
>
>
> Guozhang
>
> On Mon, Dec 9, 2019 at 10:57 AM Brian Byrne wrote:
>
> > Hi Guozhang,
> >
> > Thanks for the feedback!
> >
> > On Sun, Dec 8, 2019 at 6:25 PM Guozhang Wang wrote:
> >
> &
ever,
if a topic transitions to urgent and there's several non-urgent ones, we'll
piggyback the non-urgent updates up to the target size.
Thanks,
Brian
On Wed, Nov 20, 2019 at 7:00 PM deng ziming
> wrote:
>
> > I think it's ok, and you can add another issue about `asynchronous
> >
[
https://issues.apache.org/jira/browse/KAFKA-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Byrne resolved KAFKA-9164.
Resolution: Invalid
Marking issue invalid since the producer logic was actually handling
possible.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-526
%3A+Reduce+Producer+Metadata+Lookups+for+Large+Number+of+Topics
Thanks,
Brian
On Fri, Oct 4, 2019 at 10:04 AM Brian Byrne wrote:
> Lucas, Guozhang,
>
> Thank you for the comments. Good point on METADATA_MAX_A
ved, but this would be a large improvement on the current
situation.
Thanks,
Brian
> I think the above 2 ways are enough to solve the current problem.
>
> On Tue, Nov 19, 2019 at 3:20 AM Colin McCabe wrote:
>
> > On Mon, Nov 18, 2019, at 10:05, Brian Byrne wrote:
> > >
after this KIP,
> but
> > > they will be somewhat different. But it's not clear to me what the
> > > differences are.
> > >
> > > In general, if load is a problem, we should probably consider adding
> some
> > > kind of jitter on the clien
Excuse me, that should read "4 binding +1 votes".
Brian
On Wed, Nov 13, 2019 at 9:26 AM Brian Byrne wrote:
> Hello all,
>
> With 5 binding +1 votes from Colin McCabe, Gwen Shapira, Manikumar Reddy,
> and Jason Gustafson, and 2 non-binding +1 votes from Viktor Somogyi-V
Hello all,
With 5 binding +1 votes from Colin McCabe, Gwen Shapira, Manikumar Reddy,
and Jason Gustafson, and 2 non-binding +1 votes from Viktor Somogyi-Vass
and Jose Armando Garcia Sancio, the vote passes.
Thanks,
Brian
On Fri, Nov 8, 2019 at 2:23 PM Brian Byrne wrote:
>
> Thanks ev
ring herd problem with metadata updates. Adding jitter would spread
> the load across time rather than creating a spike every 5 minutes in this
> case.
>
> best,
> Colin
>
>
> On Fri, Nov 8, 2019, at 08:59, Ismael Juma wrote:
> > I think this KIP affects when we block
t 7:00 PM Manikumar
> wrote:
>
> > +1 (binding), Thanks for the KIP.
> >
> >
> >
> > On Fri, Nov 8, 2019 at 8:14 AM Gwen Shapira wrote:
> >
> > > +1 (binding)
> > >
> > > Thank you.
> > >
> > > On Thu, Nov 7, 20
Brian Byrne created KAFKA-9164:
--
Summary: Don't evict active topics' metadata from the producer's
cache
Key: KAFKA-9164
URL: https://issues.apache.org/jira/browse/KAFKA-9164
Project: Kafka
.e. how we
> maintain the topic metadata in producer cache is never committed to users),
> I wonder if we still need a KIP for the proposed change any more?
>
>
> Guozhang
>
> On Thu, Nov 7, 2019 at 12:43 PM Brian Byrne wrote:
>
> > Hello all,
> >
> > I'd
Hello all,
I'd like to propose a vote for a producer change to improve producer
behavior when dealing with a large number of topics, in part by reducing
the amount of metadata fetching performed.
The full KIP is provided here:
Hello all,
Ping. Any further votes or opinions?
Thanks,
Brian
On Tue, Oct 29, 2019 at 9:39 AM Brian Byrne wrote:
> Hello all,
>
> I'd like to call a vote on KIP-543: Expand ConfigCommand's non-ZK
> functionality, linked here: https://cwiki.apache.org/confluence/x/ww-3Bw
>
> Thanks,
> Brian
>
Brian Byrne created KAFKA-9139:
--
Summary: Dynamic broker config types aren't being discovered
Key: KAFKA-9139
URL: https://issues.apache.org/jira/browse/KAFKA-9139
Project: Kafka
Issue Type
, you're the
first to voice it.
Thanks,
Brian
On Wed, Oct 30, 2019 at 5:01 AM Viktor Somogyi-Vass
wrote:
> Hi Brian,
>
> Have you thought about having one letter abbreviations like '-b' for
> --brokers etc.?
>
> Thanks,
> Viktor
>
> On Tue, Oct 29, 2019 at 5:39 PM Brian By
Hello all,
I'd like to call a vote on KIP-543: Expand ConfigCommand's non-ZK
functionality, linked here: https://cwiki.apache.org/confluence/x/ww-3Bw
Thanks,
Brian
all the broker-loggers
> resource at once. The output is very verbose - I count more than 150 lines.
> Although since I imagine more options to describe internal state wouldn't
> hurt, I don't feel strongly about this.
>
> Thanks,
> Stanislav
>
> On Wed, Oct 23, 2019 at
for the operations proposed for later, it
> might be nice to have a different color, like yellow, to indicate that
> we'll need a follow-on KIP to do them.
>
> best,
> Colin
>
>
> On Tue, Oct 22, 2019, at 12:42, Brian Byrne wrote:
> > Hello all,
> >
> > I wrote
Brian Byrne created KAFKA-9082:
--
Summary: Move ConfigCommand to use KafkaAdminClient APIs
Key: KAFKA-9082
URL: https://issues.apache.org/jira/browse/KAFKA-9082
Project: Kafka
Issue Type
Hello all,
I wrote a KIP about expanding the ConfigCommand's functionality when not
accessing ZooKeeper directly, i.e. when --bootstrap-servers is specified
instead of --zookeeper. This should bring the two into parity.
There's also a bit of bikeshedding about additional flags for shortening
see no real
> > advantages to this approach compared to the async method you’ve proposed,
> > but maybe we could add it to the rejected alternatives section?
> >
> > Thanks,
> >
> > Lucas
> >
> > On Fri, 20 Sep 2019 at 11:46, Brian Byrne wrote:
> >
I've updated the 'Proposed Changes' to include two new producer
configuration variables: topic.expiry.ms and topic.refresh.ms. Please take
a look.
Thanks,
Brian
On Tue, Sep 17, 2019 at 12:59 PM Brian Byrne wrote:
> Dev team,
>
> Requesting discussion for improvement to the prod
Dev team,
Requesting discussion for improvement to the producer when dealing with a
large number of topics.
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-526%3A+Reduce+Producer+Metadata+Lookups+for+Large+Number+of+Topics
JIRA: https://issues.apache.org/jira/browse/KAFKA-8904
Hello,
I'm requesting permission to the Kafka Wiki, specifically to create a KIP.
Wiki ID is 'bbyrne'.
Thanks,
Brian
Brian Byrne created KAFKA-8904:
--
Summary: Reduce metadata lookups when producting to a large number
of topics
Key: KAFKA-8904
URL: https://issues.apache.org/jira/browse/KAFKA-8904
Project: Kafka
67 matches
Mail list logo
