Re: JMX prot

2015-11-11 Thread Shaun Senecal
Take a look here: https://kafka.apache.org/08/quickstart.html Basically, you need to set the JMX_PORT variable before starting Kafka From: , Roy Sent: November 11, 2015 3:19 PM To: users@kafka.apache.org Subject: JMX prot Hi We are

Re: It's 5.41am, we're after 20+ hours of debugging our prod cluster. See NotAssignedReplicaException and UnknownException errors. Help?

2015-10-20 Thread Shaun Senecal
I can't say this is the same issue, but it sounds similar to a situation we experienced with Kafka 0.8.2.[1-2]. After restarting a broker, the cluster would never really recover (ISRs constantly changing, replication failing, etc). We found the only way to fully recover the cluster was to

Shrinking ISR with no load on brokers or incoming messages

2015-10-01 Thread Shaun Senecal
Hi I have noticed that when our brokers have no incoming connections (just connections to other brokers and to the ZK cluster) we get messages about shrinking the ISR for some partitions [2015-10-02 00:58:31,239] INFO Partition [lia.stage.raw_events,9] on broker 1: Shrinking ISR for

Re: number of topics given many consumers and groups within the data

2015-09-30 Thread Shaun Senecal
all consumers consume everything” and “one >> topic >>> per consumer” as you vary the number of partitions in the topic, albeit >> at >>> the cost of a little extra complexity. >>> >>> Also, not sure if you’ve seen it but there is quite a good section

Re: number of topics given many consumers and groups within the data

2015-09-30 Thread Shaun Senecal
nd partition sizing. B > On 29 Sep 2015, at 18:48, Shaun Senecal <shaun.sene...@lithium.com> wrote: > > Hi > > > I heave read Jay Kreps post regarding the number of topics that can be > handled by a broker > (https://www.quora.com/How-many-topics-can-be-created-in-A

number of topics given many consumers and groups within the data

2015-09-29 Thread Shaun Senecal
Hi I heave read Jay Kreps post regarding the number of topics that can be handled by a broker (https://www.quora.com/How-many-topics-can-be-created-in-Apache-Kafka), and it has left me with more questions that I dont see answered anywhere else. We have a data stream which will be consumed