Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-02 Thread Becket Qin
Thanks for the explanation, Colin. A few more questions. >The session epoch is not complex. It's just a number which increments >on each incremental fetch. The session epoch is also useful for >debugging-- it allows you to match up requests and responses when >looking at log files. Currently

[GitHub] kafka pull request #4288: KAFKA-6300 SelectorTest may fail with ConcurrentMo...

2017-12-02 Thread tedyu
GitHub user tedyu opened a pull request: https://github.com/apache/kafka/pull/4288 KAFKA-6300 SelectorTest may fail with ConcurrentModificationException Synchronization is added w.r.t. sockets ArrayList to avoid ConcurrentModificationException ### Committer Checklist

[jira] [Created] (KAFKA-6300) SelectorTest may fail with ConcurrentModificationException

2017-12-02 Thread Ted Yu (JIRA)
Ted Yu created KAFKA-6300: - Summary: SelectorTest may fail with ConcurrentModificationException Key: KAFKA-6300 URL: https://issues.apache.org/jira/browse/KAFKA-6300 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

2017-12-02 Thread Ted Yu
bq. It the controller_epoch of the incoming MetadataResponse, or if the controller_epoch is the same but the controller_metadata_epoch Can you update the above sentence so that the intention is clearer ? Thanks On Fri, Dec 1, 2017 at 6:33 PM, Dong Lin wrote: > Hi all, > >

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-02 Thread Colin McCabe
On Fri, Dec 1, 2017, at 11:46, Dong Lin wrote: > On Thu, Nov 30, 2017 at 9:37 AM, Colin McCabe wrote: > > > On Wed, Nov 29, 2017, at 18:59, Dong Lin wrote: > > > Hey Colin, > > > > > > Thanks much for the update. I have a few questions below: > > > > > > 1. I am not very sure

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-02 Thread Colin McCabe
On Thu, Nov 30, 2017, at 23:29, Jan Filipiak wrote: > Hi, > > this discussion is going a little bit far from what I intended this > thread for. > I can see all of this beeing related. > > To let you guys know what I am currently thinking is the following: > > I do think the handling of Id's

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

2017-12-02 Thread Colin McCabe
On Thu, Nov 30, 2017, at 16:47, Becket Qin wrote: > Hi Colin, > > Thanks for updating the KIP. I have two comments: > > 1. The session epoch seems introducing some complexity. It would be good > if we don't have to maintain the epoch. The session epoch is not complex. It's just a number which

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

2017-12-02 Thread Colin McCabe
On Tue, Nov 28, 2017, at 14:48, Rajini Sivaram wrote: > Hi Colin, > > Thank you for reviewing the KIP. > > *kaka-configs.sh* will be converted to use *AdminClient* under > KAFKA-5722. > This is targeted for the next release (1.1.0). Under this KIP, we will > implement *AdminClient#alterConfigs*

[jira] [Created] (KAFKA-6299) Fix AdminClient error handling when metadata changes

2017-12-02 Thread Colin P. McCabe (JIRA)
Colin P. McCabe created KAFKA-6299: -- Summary: Fix AdminClient error handling when metadata changes Key: KAFKA-6299 URL: https://issues.apache.org/jira/browse/KAFKA-6299 Project: Kafka Issue

[jira] [Resolved] (KAFKA-3502) Build is killed during kafka streams tests due to `pure virtual method called` error

2017-12-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3502. -- Resolution: Cannot Reproduce > Build is killed during kafka streams tests due to `pure virtual

[jira] [Resolved] (KAFKA-2081) testUncleanLeaderElectionEnabledByTopicOverride transient failure

2017-12-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-2081. -- Resolution: Cannot Reproduce > testUncleanLeaderElectionEnabledByTopicOverride transient

[jira] [Resolved] (KAFKA-1534) transient unit test failure in testBasicPreferredReplicaElection

2017-12-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-1534. -- Resolution: Cannot Reproduce We have not seen this issue any more; closing as not re-produce.

Build failed in Jenkins: kafka-trunk-jdk8 #2255

2017-12-02 Thread Apache Jenkins Server
See -- [...truncated 2.96 MB...] org.apache.kafka.common.config.ConfigDefTest > testValidateCannotParse STARTED org.apache.kafka.common.config.ConfigDefTest > testValidateCannotParse

Build failed in Jenkins: kafka-trunk-jdk8 #2254

2017-12-02 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-6118: Fix transient failure -- [...truncated 3.37 MB...] kafka.zookeeper.ZooKeeperClientTest > testZNodeChildChangeHandlerForChildChange

Build failed in Jenkins: kafka-trunk-jdk9 #235

2017-12-02 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-6118: Fix transient failure -- [...truncated 1.44 MB...] kafka.zk.ZKPathTest > testMakeSurePersistsPathExists PASSED

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

2017-12-02 Thread Steven Aerts
Hi Tom, I just made changes to the proposal of KIP-218, to make everything more backwards compatible as suggested by Collin. For me it is now in a state where starts to become final. I propose to wait a few days so everybody can take a look and open the votes when I do not receive any major

Build failed in Jenkins: kafka-0.11.0-jdk7 #344

2017-12-02 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: remove mis-merges from the previous commit -- [...truncated 2.45 MB...] org.apache.kafka.streams.integration.EosIntegrationTest >

Build failed in Jenkins: kafka-trunk-jdk7 #3015

2017-12-02 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-6118: Fix transient failure -- [...truncated 407.25 KB...] kafka.security.auth.SimpleAclAuthorizerTest > testAclManagementAPIs STARTED

[jira] [Reopened] (KAFKA-6078) Investigate failure of ReassignPartitionsClusterTest.shouldExpandCluster

2017-12-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-6078: -- Assignee: (was: Dong Lin) > Investigate failure of