[jira] [Created] (KAFKA-6470) no continuous offset for function seek

[chao.wu (JIRA)]

chao.wu created KAFKA-6470:
--

 Summary: no continuous offset for function seek
 Key: KAFKA-6470
 URL: https://issues.apache.org/jira/browse/KAFKA-6470
 Project: Kafka
  Issue Type: Bug
  Components: clients, core
Affects Versions: 0.10.0.1
Reporter: chao.wu


A topic-partition "adn-tracking,15"  in kafka  who's   earliest offset is  
1255644602 and  latest offset is 1271253441.  

while starting a spark streaming to process the data from the topic ,  we got a 
exception with "Got wrong record   even after seeking to offset 1266921577".

I  implemented a simple project to use consumer to  seek offset 1266921577. But 
it return the offset 1266921578. Then while  seek to 1266921576, it return the 
1266921576 exactly。 

Why ?  How to fix that ?

 

 

There is the code:

public class consumerDemo {




 public static void main(String[] argv){
 Properties props = new Properties();
 props.put("bootstrap.servers", "172.31.29.31:9091");
 props.put("group.id", "consumer-tutorial-demo");
 props.put("key.deserializer", StringDeserializer.class.getName());
 props.put("value.deserializer", StringDeserializer.class.getName());
 KafkaConsumer consumer = new KafkaConsumer(props);
 TopicPartition tp = new TopicPartition("adn-tracking-click", 15);
 Collection collection = new ArrayList();
 collection.add(tp);
 consumer.assign(collection);
 consumer.seek(tp, 1266921576);
 ConsumerRecords consumerRecords = consumer.poll(1);

 List> listR = consumerRecords.records(tp);
 Iterator > iter = listR.iterator();
 ConsumerRecord record = iter.next();
 System.out.println(" the next record " + record.offset() + " recode topic " + 
record.topic());


 }
}

 

 

 

    



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-6469) ISR change notification queue has a maximum size

[Kyle Ambroff-Kao (JIRA)]

Kyle Ambroff-Kao created KAFKA-6469:
---

 Summary: ISR change notification queue has a maximum size
 Key: KAFKA-6469
 URL: https://issues.apache.org/jira/browse/KAFKA-6469
 Project: Kafka
  Issue Type: Bug
Reporter: Kyle Ambroff-Kao


When the writes /isr_change_notification in ZooKeeper (which is effectively a 
queue of ISR change events for the controller) happen at a rate high enough 
that the node with a watch can't keep up dequeuing them, the trouble starts.

The watcher kafka.controller.IsrChangeNotificationListener is fired in the 
controller when a new entry is written to /isr_change_notification, and the 
zkclient library sends a GetChildrenRequest to zookeeper to fetch all child 
znodes. The size of the GetChildrenResponse returned by ZooKeeper is the 
problem. Reading through the code and running some tests to confirm shows that 
an empty GetChildrenResponse is 4 bytes on the wire, and every child node name 
minimum 4 bytes as well. Since these znodes are length 21, that means every 
child znode will account for 25 bytes in the response.

A GetChildrenResponse with 42k child nodes of the same length will be just 
about 1.001MB, which is larger than the 1MB data frame that ZooKeeper uses. 
This causes the ZooKeeper server to drop the broker's session.

So if 42k ISR changes happen at once, and the controller pauses at just the 
right time, you'll end up with a queue that can no longer be drained.

We've seen this happen in one of our test clusters as the partition count 
started to climb north of 60k per broker. We had a hardware failure that lead 
to the cluster writing so many child nodes to /isr_change_notification that the 
controller could no longer list its children, effectively bricking the cluster.

This can be partially mitigated by chunking ISR notifications to increase the 
maximum number of partitions a broker can host.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-6468) Replication high watermark checkpoint file read for every LeaderAndIsrRequest

[Kyle Ambroff-Kao (JIRA)]

Kyle Ambroff-Kao created KAFKA-6468:
---

 Summary: Replication high watermark checkpoint file read for every 
LeaderAndIsrRequest
 Key: KAFKA-6468
 URL: https://issues.apache.org/jira/browse/KAFKA-6468
 Project: Kafka
  Issue Type: Bug
Reporter: Kyle Ambroff-Kao


The high watermark for each partition in a given log directory is written to 
disk every _replica.high.watermark.checkpoint.interval.ms_ milliseconds. This 
checkpoint file is used to create replicas when joining the cluster.

[https://github.com/apache/kafka/blob/b73c765d7e172de4742a3aa023d5a0a4b7387247/core/src/main/scala/kafka/cluster/Partition.scala#L180]

Unfortunately this file is read every time 
kafka.cluster.Partition#getOrCreateReplica is invoked. For most clusters this 
isn't a big deal, but for a small cluster with lots of partitions all of the 
reads of this file really add up.

On my local test cluster of three brokers with around 40k partitions, the 
initial LeaderAndIsrRequest refers to every partition in the cluster, and it 
can take 20 to 30 minutes to create all of the replicas because the 
_replication-offset-checkpoint_ is nearly 2MB.

Changing this code so that we only read this file once on startup reduces the 
time to create all replicas to around one minute.

Credit to [~onurkaraman] for finding this one.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: kafka-trunk-jdk7 #3110

[Apache Jenkins Server]

See 


Changes:

[wangguoz] MINOR: increase timeout for unstable

[me] MINOR: Add async and different sync startup modes in connect service

--
[...truncated 1.89 MB...]

org.apache.kafka.streams.kstream.SessionWindowsTest > 
retentionTimeMustNotBeNegative STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
retentionTimeMustNotBeNegative PASSED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldNotBeEqualWhenMaintainMsDifferent STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldNotBeEqualWhenMaintainMsDifferent PASSED

org.apache.kafka.streams.kstream.SessionWindowsTest > windowSizeMustNotBeZero 
STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > windowSizeMustNotBeZero 
PASSED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
windowSizeMustNotBeNegative STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
windowSizeMustNotBeNegative PASSED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldSetWindowRetentionTime STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldSetWindowRetentionTime PASSED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldNotBeEqualWhenGapIsDifferent STARTED

org.apache.kafka.streams.kstream.SessionWindowsTest > 
shouldNotBeEqualWhenGapIsDifferent PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullTopicsWhenAddingSourceWithPattern STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullTopicsWhenAddingSourceWithPattern PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowZeroTopicsWhenAddingSource STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowZeroTopicsWhenAddingSource PASSED

org.apache.kafka.streams.TopologyTest > shouldFailOnUnknownSource STARTED

org.apache.kafka.streams.TopologyTest > shouldFailOnUnknownSource PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullNameWhenAddingSink 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullNameWhenAddingSink 
PASSED

org.apache.kafka.streams.TopologyTest > 
multipleSourcesShouldHaveDistinctSubtopologies STARTED

org.apache.kafka.streams.TopologyTest > 
multipleSourcesShouldHaveDistinctSubtopologies PASSED

org.apache.kafka.streams.TopologyTest > 
testNamedTopicMatchesAlreadyProvidedPattern STARTED

org.apache.kafka.streams.TopologyTest > 
testNamedTopicMatchesAlreadyProvidedPattern PASSED

org.apache.kafka.streams.TopologyTest > 
processorsWithSharedStateShouldHaveSameSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
processorsWithSharedStateShouldHaveSameSubtopology PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddTopicTwice STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddTopicTwice PASSED

org.apache.kafka.streams.TopologyTest > 
processorWithMultipleSourcesShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
processorWithMultipleSourcesShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddStateStoreToSink 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddStateStoreToSink 
PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAddNullStateStoreSupplier 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAddNullStateStoreSupplier 
PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddStateStoreToNonExistingProcessor STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddStateStoreToNonExistingProcessor PASSED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullStoreNameWhenConnectingProcessorAndStateStores STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullStoreNameWhenConnectingProcessorAndStateStores PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullNameWhenAddingSourceWithTopic STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullNameWhenAddingSourceWithTopic PASSED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorWithStateShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorWithStateShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullTopicWhenAddingSink 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullTopicWhenAddingSink 
PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddGlobalStoreWithSourceNameEqualsProcessorName STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddGlobalStoreWithSourceNameEqualsProcessorName PASSED

org.apache.kafka.streams.TopologyTest > 
singleSourceWithListOfTopicsShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest 

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

[Jason Gustafson]

+1 (binding)

Just one minor comment. It seems a little surprising that HeaderConverter
does not use the Header interface. I expected something like this:

Header toConnectHeader(String topic, String headerKey, byte[] value);
byte[] fromConnectHeader(String topic, Header header);

Was there a reason not to do it this way?

Thanks,
Jason

On Mon, Jan 22, 2018 at 4:45 PM, Ted Yu  wrote:

> +1
>
> On Mon, Jan 22, 2018 at 2:48 PM, Gwen Shapira  wrote:
>
> > +1 (binding)
> >
> > This is going to be HUGE! Thank you Randall.
> >
> > On Mon, Jan 22, 2018 at 1:18 PM Konstantine Karantasis <
> > konstant...@confluent.io> wrote:
> >
> > > Great addition!
> > >
> > > +1 (non-binding)
> > >
> > > Konstantine
> > >
> > > On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava <
> > e...@confluent.io>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks for the work on this -- not a small upgrade to the Connect
> APIs!
> > > >
> > > > -Ewen
> > > >
> > > > On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch 
> > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I'd like to start the voting on this KIP to add support for headers
> > in
> > > > > Connect.:
> > > > >
> > > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 145+-+Expose+Record+Headers+in+Kafka+Connect
> > > > >  > > > > 145+-+Expose+Record+Headers+in+Kafka+Connect>*
> > > > >
> > > > > This does add a fair number of interfaces to our public API, and
> > > defines
> > > > > some behavioral changes as well.
> > > > >
> > > > > Thanks! Your feedback is highly appreciated.
> > > > >
> > > > > Randall
> > > > >
> > > >
> > >
> >
>

[jira] [Resolved] (KAFKA-6466) Kafka connect task sometimes fails on start-up

[Randall Hauch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Randall Hauch resolved KAFKA-6466.
--
Resolution: Duplicate

> Kafka connect task sometimes fails on start-up
> --
>
> Key: KAFKA-6466
> URL: https://issues.apache.org/jira/browse/KAFKA-6466
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 1.0.0
>Reporter: Alexander Koval
>Priority: Major
>
> We use kafka connect for indexing into Elasticsearch. Sometimes when we 
> updating our kafka-connect application one or several tasks get {{FAILED}} 
> status. Restarting connector with failed task usually helps.
>  
> Below is an example of the failed task's stacktrace:
> {code:java}
> java.lang.IllegalArgumentException: A metric named 'MetricName 
> [name=offset-commit-max-time-ms, group=connector-task-metrics, 
> description=The maximum time in milliseconds taken by this task to commit 
> offsets., tags={connector=prom-ua-catalog-product, task=2}]' already exists, 
> can't register another one.
> at 
> org.apache.kafka.common.metrics.Metrics.registerMetric(Metrics.java:532)
> at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:256)
> at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:241)
> at 
> org.apache.kafka.connect.runtime.WorkerTask$TaskMetricsGroup.(WorkerTask.java:328)
> at org.apache.kafka.connect.runtime.WorkerTask.(WorkerTask.java:69)
> at 
> org.apache.kafka.connect.runtime.WorkerSinkTask.(WorkerSinkTask.java:98)
> at 
> org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:449)
> at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:404)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:852)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:108)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:866)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:862)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: kafka-trunk-jdk9 #324

[Apache Jenkins Server]

See 


Changes:

[me] MINOR: Add async and different sync startup modes in connect service

--
[...truncated 1.86 MB...]
org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowInvalidStoreNameOnReduce STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowInvalidStoreNameOnReduce PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenSubtractorIsNull STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenSubtractorIsNull PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableNotSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableNotSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableJoin PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedReduceIfReducerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedReduceIfReducerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfInitializerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfInitializerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMergerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMergerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeCount STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeCount PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeWithoutSpecifyingSerdes STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeWithoutSpecifyingSerdes PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeAggregated STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeAggregated PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnCountIfMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnCountIfMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnAggregateIfAggregatorIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnAggregateIfAggregatorIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnReduceIfReducerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnReduceIfReducerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldAggregateSessionWindowed STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldAggregateSessionWindowed PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldCountSessionWindowed STARTED

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

[Ted Yu]

+1

On Mon, Jan 22, 2018 at 2:48 PM, Gwen Shapira  wrote:

> +1 (binding)
>
> This is going to be HUGE! Thank you Randall.
>
> On Mon, Jan 22, 2018 at 1:18 PM Konstantine Karantasis <
> konstant...@confluent.io> wrote:
>
> > Great addition!
> >
> > +1 (non-binding)
> >
> > Konstantine
> >
> > On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava <
> e...@confluent.io>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks for the work on this -- not a small upgrade to the Connect APIs!
> > >
> > > -Ewen
> > >
> > > On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch 
> wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I'd like to start the voting on this KIP to add support for headers
> in
> > > > Connect.:
> > > >
> > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 145+-+Expose+Record+Headers+in+Kafka+Connect
> > > >  > > > 145+-+Expose+Record+Headers+in+Kafka+Connect>*
> > > >
> > > > This does add a fair number of interfaces to our public API, and
> > defines
> > > > some behavioral changes as well.
> > > >
> > > > Thanks! Your feedback is highly appreciated.
> > > >
> > > > Randall
> > > >
> > >
> >
>

Build failed in Jenkins: kafka-trunk-jdk7 #3109

[Apache Jenkins Server]

See 

--
Started by an SCM change
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-eu2 (ubuntu trusty) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://github.com/apache/kafka.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:825)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1092)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1123)
at hudson.scm.SCM.checkout(SCM.java:495)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Caused by: hudson.plugins.git.GitException: Command "git config 
remote.origin.url https://github.com/apache/kafka.git; returned status code 4:
stdout: 
stderr: error: failed to write new configuration file .git/config.lock

at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1938)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1934)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1572)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1584)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.setRemoteUrl(CliGitAPIImpl.java:1218)
at hudson.plugins.git.GitAPI.setRemoteUrl(GitAPI.java:160)
at sun.reflect.GeneratedMethodAccessor439.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
ubuntu-eu2
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:281)
at com.sun.proxy.$Proxy110.setRemoteUrl(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.setRemoteUrl(RemoteGitImpl.java:295)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:813)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1092)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1123)
at hudson.scm.SCM.checkout(SCM.java:495)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 

Re: [VOTE] KIP-227: Introduce Fetch Requests that are Incremental to Increase Partition Scalability

[Jason Gustafson]

>
> What if we want to have fetch sessions for non-incremental fetches in the
> future, though?  Also, we don't expect this configuration to be changed
> often, so it doesn't really need to be short.


Hmm.. But in that case, I'm not sure we'd need to distinguish the two
cases. If the non-incremental sessions are occupying space proportional to
the fetched partitions, using the same config for both would be reasonable.
If they are not (which is more likely), we probably wouldn't need a config
at all. Given that, I'd probably still opt for the more concise name. It's
not a blocker for me though.

+1 on the KIP.

-Jason

On Mon, Jan 22, 2018 at 3:56 PM, Colin McCabe  wrote:

> On Mon, Jan 22, 2018, at 15:42, Jason Gustafson wrote:
> > Hi Colin,
> >
> > This is looking good to me. A few comments:
> >
> > 1. The fetch type seems unnecessary in the request and response schemas
> > since it can be inferred by the sessionId/epoch.
>
> Hi Jason,
>
> Fair enough... if we need it later, we can always bump the RPC version.
>
> > 2. I agree with Jun that a separate array for partitions to remove would
> be
> > more intuitive.
>
> OK.  I'll switch it to using a separate array.
>
> > 3. I'm not super thrilled with the cache configuration since it seems to
> > tie us a bit too closely to the implementation. You've mostly convinced
> me
> > on the need for the slots config, but I wonder if we can at least do
> > without "min.incremental.fetch.session.eviction.ms"? For one, I think
> the
> > broker should reserve the right to evict sessions at will. We shouldn't
> be
> > stuck maintaining a small session at the expense of a much larger one
> just
> > to enforce this timeout. Internally, I think having some cache stickiness
> > to avoid thrashing makes sense, but I think static values are likely to
> be
> > good enough and that lets us retain some flexibility to change the
> behavior
> > in the future.
>
> OK.
>
> > 4. I think the word "incremental" is redundant in the config names. Maybe
> > it could just be "max.fetch.session.cache.slots" for example?
>
> What if we want to have fetch sessions for non-incremental fetches in the
> future, though?  Also, we don't expect this configuration to be changed
> often, so it doesn't really need to be short.
>
> best,
> Colin
>
> >
> > Thanks,
> > Jason
> >
> >
> >
> > On Sat, Jan 20, 2018 at 12:54 PM, Colin McCabe 
> wrote:
> >
> > > On Fri, Jan 19, 2018, at 15:02, Jun Rao wrote:
> > > > Hi, Colin,
> > > >
> > > > Thanks for the KIP. Looks good to me overall. Just a couple of more
> > > > comments.
> > > >
> > > > 1. As I mentioned earlier, it might be useful to add some metrics for
> > > > monitoring the usage of the session cache. For example, it would be
> > > useful
> > > > to know how many slots are being used (or unused), # of total
> partitions
> > > in
> > > > the cached slots (to understand space), the eviction rate (to see if
> > > there
> > > > is any churn), etc.
> > >
> > > Thanks, Jun.  Sorry-- I meant to address this earlier, but I forgot
> about
> > > it.  I just added some proposed metrics to the KIP wiki.
> > >
> > > >
> > > > 2. Using max_bytes to 0 represent the removal of a partition seems
> > > > unintuitive. Perhaps it's better to either add a flag per partition
> or
> > > add
> > > > a removed partition list.
> > >
> > > Perhaps if we use max_bytes -1 to represent removal, it will be more
> > > intuitive?  After all, -1 bytes is clearly not a valid amount of bytes
> to
> > > fetch.  Or should be have a separate array of removed TopicPartitions?
> > >
> > > On a related note, in the FetchResponse#PartitionData, we have an
> "error"
> > > field, plus highWatermark, lastStableOffset, logStartOffset, etc.  But
> when
> > > the "error" field is set, those other fields are not used.  Perhaps we
> > > could save some space by just having a separate array of "partitions
> with
> > > errors."  In the common case where there are no errors, this would
> save 2
> > > bytes per partition, which could be quite significant in large
> responses.
> > >
> > > best,
> > > Colin
> > >
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Thu, Jan 18, 2018 at 6:15 PM, Colin McCabe 
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I updated the KIP.  There is also an implementation of this KIP
> here:
> > > > > https://github.com/apache/kafka/pull/4418
> > > > >
> > > > > The updated implementation simplifies a few things, and adds the
> > > ability
> > > > > to incrementally add or remove individual partitions in an
> incremental
> > > > > fetch request.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > >
> > > > > On Tue, Dec 19, 2017, at 19:28, Colin McCabe wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > I'd like to start the vote on KIP-227: Incremental Fetch
> Requests.
> > > > > >
> > > > > > The KIP is here:
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 

[VOTE] KIP-212: Enforce set of legal characters for connector names

[Sönke Liebau]

All,

this KIP has been discussed for quite some time now and I believe we
addressed all major concerns in the current revision, so I'd like to
start a vote.

KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-212%3A+Enforce+set+of+legal+characters+for+connector+names

Let me know what you think.

Kind regards,
Sönke

Build failed in Jenkins: kafka-trunk-jdk7 #3108

[Apache Jenkins Server]

See 


Changes:

[wangguoz] MINOR: update docs with regard to improved resilience of Kafka 
Streams

--
[...truncated 1.90 MB...]

org.apache.kafka.streams.tools.StreamsResetterTest > shouldSeekToEndOffset 
STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > shouldSeekToEndOffset 
PASSED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenBetweenBeginningAndEndOffset STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenBetweenBeginningAndEndOffset PASSED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenBeforeBeginningOffset STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenBeforeBeginningOffset PASSED

org.apache.kafka.streams.tools.StreamsResetterTest > 
shouldThrowOnInvalidDateFormat STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > 
shouldThrowOnInvalidDateFormat PASSED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenAfterEndOffset STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testResetUsingPlanWhenAfterEndOffset PASSED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testShiftOffsetByWhenBeforeBeginningOffset STARTED

org.apache.kafka.streams.tools.StreamsResetterTest > 
testShiftOffsetByWhenBeforeBeginningOffset PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldResetToDefaultIfProducerEnableIdempotenceIsOverriddenIfEosEnabled STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldResetToDefaultIfProducerEnableIdempotenceIsOverriddenIfEosEnabled PASSED

org.apache.kafka.streams.StreamsConfigTest > shouldUseNewConfigsWhenPresent 
STARTED

org.apache.kafka.streams.StreamsConfigTest > shouldUseNewConfigsWhenPresent 
PASSED

org.apache.kafka.streams.StreamsConfigTest > testGetConsumerConfigs STARTED

org.apache.kafka.streams.StreamsConfigTest > testGetConsumerConfigs PASSED

org.apache.kafka.streams.StreamsConfigTest > shouldAcceptAtLeastOnce STARTED

org.apache.kafka.streams.StreamsConfigTest > shouldAcceptAtLeastOnce PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldUseCorrectDefaultsWhenNoneSpecified STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldUseCorrectDefaultsWhenNoneSpecified PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingProducerEnableIdempotenceIfEosDisabled STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingProducerEnableIdempotenceIfEosDisabled PASSED

org.apache.kafka.streams.StreamsConfigTest > defaultSerdeShouldBeConfigured 
STARTED

org.apache.kafka.streams.StreamsConfigTest > defaultSerdeShouldBeConfigured 
PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldSetDifferentDefaultsIfEosEnabled STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldSetDifferentDefaultsIfEosEnabled PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldNotOverrideUserConfigRetriesIfExactlyOnceEnabled STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldNotOverrideUserConfigRetriesIfExactlyOnceEnabled PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldSupportPrefixedConsumerConfigs STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldSupportPrefixedConsumerConfigs PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldOverrideStreamsDefaultConsumerConfigs STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldOverrideStreamsDefaultConsumerConfigs PASSED

org.apache.kafka.streams.StreamsConfigTest > testGetProducerConfigs STARTED

org.apache.kafka.streams.StreamsConfigTest > testGetProducerConfigs PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingProducerMaxInFlightRequestPerConnectionsWhenEosDisabled 
STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingProducerMaxInFlightRequestPerConnectionsWhenEosDisabled PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldThrowStreamsExceptionIfValueSerdeConfigFails STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldThrowStreamsExceptionIfValueSerdeConfigFails PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldResetToDefaultIfConsumerAutoCommitIsOverridden STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldResetToDefaultIfConsumerAutoCommitIsOverridden PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldThrowExceptionIfNotAtLestOnceOrExactlyOnce STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldThrowExceptionIfNotAtLestOnceOrExactlyOnce PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingConsumerIsolationLevelIfEosDisabled STARTED

org.apache.kafka.streams.StreamsConfigTest > 
shouldAllowSettingConsumerIsolationLevelIfEosDisabled PASSED

org.apache.kafka.streams.StreamsConfigTest > 
shouldSupportPrefixedProducerConfigs STARTED

org.apache.kafka.streams.StreamsConfigTest > 

[jira] [Created] (KAFKA-6467) Enforce layout of dependencies within a Connect plugin to be deterministic

[Konstantine Karantasis (JIRA)]

Konstantine Karantasis created KAFKA-6467:
-

 Summary: Enforce layout of dependencies within a Connect plugin to 
be deterministic
 Key: KAFKA-6467
 URL: https://issues.apache.org/jira/browse/KAFKA-6467
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Affects Versions: 0.11.0.2, 1.0.0
Reporter: Konstantine Karantasis
Assignee: Konstantine Karantasis
 Fix For: 1.0.1, 0.11.0.3


In principle, Connect plugins that intend to load their dependencies in 
isolation should not contain any conflicts among the classes they package as 
dependencies. In other words, the order in which a plugin's dependencies are 
laid out and passed to its plugin class loader should not matter.

However, in practice, there are rare and suboptimal situations where a plugin 
needs to bundle a few packages with conflicting dependencies because it doesn't 
control packaging of third-party modules. In those cases depending on a 
deterministic ordering within the class loader's path can help the Connect 
plugin enforce loading of the desired classes as needed. (For example, see 
[HDFS connector with MapR 
libs|https://github.com/confluentinc/kafka-connect-hdfs/issues/270] or [HDFS 
connector with Hive's extended 
jar|https://github.com/confluentinc/kafka-connect-hdfs/issues/261)

To achieve such ordering, this improvement suggests ordering a plugin's 
dependencies in a nested directory structure by sorting such paths 
alphanumerically. This way the deterministic order is implicit (no extra 
configuration is required) and a specific dependency can be put earlier or 
later in the class loader's path with appropriate naming of its package path 
(e.g. within the plugin's directory).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: kafka-trunk-jdk9 #323

[Apache Jenkins Server]

See 


Changes:

[wangguoz] MINOR: update docs with regard to improved resilience of Kafka 
Streams

[wangguoz] MINOR: increase timeout for unstable

--
[...truncated 1.86 MB...]
org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowInvalidStoreNameOnReduce STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldNotAllowInvalidStoreNameOnReduce PASSED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenSubtractorIsNull STARTED

org.apache.kafka.streams.kstream.internals.KGroupedTableImplTest > 
shouldThrowNullPointerOnReduceWhenSubtractorIsNull PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > testJoin 
PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testNotSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableNotSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableNotSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testSendingOldValues PASSED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableKTableJoinTest > 
testQueryableJoin PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedReduceIfReducerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedReduceIfReducerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfInitializerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfInitializerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMergerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnMaterializedAggregateIfMergerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeCount STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeCount PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeWithoutSpecifyingSerdes STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeWithoutSpecifyingSerdes PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeAggregated STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldMaterializeAggregated PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnCountIfMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnCountIfMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnAggregateIfAggregatorIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnAggregateIfAggregatorIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnReduceIfReducerIsNull STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldThrowNullPointerOnReduceIfReducerIsNull PASSED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldAggregateSessionWindowed STARTED

org.apache.kafka.streams.kstream.internals.SessionWindowedKStreamImplTest > 
shouldAggregateSessionWindowed PASSED

Re: [VOTE] KIP-227: Introduce Fetch Requests that are Incremental to Increase Partition Scalability

[Colin McCabe]

On Mon, Jan 22, 2018, at 15:42, Jason Gustafson wrote:
> Hi Colin,
> 
> This is looking good to me. A few comments:
> 
> 1. The fetch type seems unnecessary in the request and response schemas
> since it can be inferred by the sessionId/epoch.

Hi Jason,

Fair enough... if we need it later, we can always bump the RPC version.

> 2. I agree with Jun that a separate array for partitions to remove would be
> more intuitive.

OK.  I'll switch it to using a separate array.

> 3. I'm not super thrilled with the cache configuration since it seems to
> tie us a bit too closely to the implementation. You've mostly convinced me
> on the need for the slots config, but I wonder if we can at least do
> without "min.incremental.fetch.session.eviction.ms"? For one, I think the
> broker should reserve the right to evict sessions at will. We shouldn't be
> stuck maintaining a small session at the expense of a much larger one just
> to enforce this timeout. Internally, I think having some cache stickiness
> to avoid thrashing makes sense, but I think static values are likely to be
> good enough and that lets us retain some flexibility to change the behavior
> in the future.

OK.

> 4. I think the word "incremental" is redundant in the config names. Maybe
> it could just be "max.fetch.session.cache.slots" for example?

What if we want to have fetch sessions for non-incremental fetches in the 
future, though?  Also, we don't expect this configuration to be changed often, 
so it doesn't really need to be short.

best,
Colin

> 
> Thanks,
> Jason
> 
> 
> 
> On Sat, Jan 20, 2018 at 12:54 PM, Colin McCabe  wrote:
> 
> > On Fri, Jan 19, 2018, at 15:02, Jun Rao wrote:
> > > Hi, Colin,
> > >
> > > Thanks for the KIP. Looks good to me overall. Just a couple of more
> > > comments.
> > >
> > > 1. As I mentioned earlier, it might be useful to add some metrics for
> > > monitoring the usage of the session cache. For example, it would be
> > useful
> > > to know how many slots are being used (or unused), # of total partitions
> > in
> > > the cached slots (to understand space), the eviction rate (to see if
> > there
> > > is any churn), etc.
> >
> > Thanks, Jun.  Sorry-- I meant to address this earlier, but I forgot about
> > it.  I just added some proposed metrics to the KIP wiki.
> >
> > >
> > > 2. Using max_bytes to 0 represent the removal of a partition seems
> > > unintuitive. Perhaps it's better to either add a flag per partition or
> > add
> > > a removed partition list.
> >
> > Perhaps if we use max_bytes -1 to represent removal, it will be more
> > intuitive?  After all, -1 bytes is clearly not a valid amount of bytes to
> > fetch.  Or should be have a separate array of removed TopicPartitions?
> >
> > On a related note, in the FetchResponse#PartitionData, we have an "error"
> > field, plus highWatermark, lastStableOffset, logStartOffset, etc.  But when
> > the "error" field is set, those other fields are not used.  Perhaps we
> > could save some space by just having a separate array of "partitions with
> > errors."  In the common case where there are no errors, this would save 2
> > bytes per partition, which could be quite significant in large responses.
> >
> > best,
> > Colin
> >
> > >
> > > Jun
> > >
> > >
> > > On Thu, Jan 18, 2018 at 6:15 PM, Colin McCabe 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I updated the KIP.  There is also an implementation of this KIP here:
> > > > https://github.com/apache/kafka/pull/4418
> > > >
> > > > The updated implementation simplifies a few things, and adds the
> > ability
> > > > to incrementally add or remove individual partitions in an incremental
> > > > fetch request.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > On Tue, Dec 19, 2017, at 19:28, Colin McCabe wrote:
> > > > > Hi all,
> > > > >
> > > > > I'd like to start the vote on KIP-227: Incremental Fetch Requests.
> > > > >
> > > > > The KIP is here:
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 227%3A+Introduce+Incremental+FetchRequests+to+Increase+
> > > > Partition+Scalability
> > > > >
> > > > > and discussion thread earlier:
> > > > > https://www.mail-archive.com/dev@kafka.apache.org/msg83115.html
> > > > >
> > > > > thanks,
> > > > > Colin
> > > >
> >

Re: [VOTE] KIP-227: Introduce Fetch Requests that are Incremental to Increase Partition Scalability

[Jason Gustafson]

Hi Colin,

This is looking good to me. A few comments:

1. The fetch type seems unnecessary in the request and response schemas
since it can be inferred by the sessionId/epoch.
2. I agree with Jun that a separate array for partitions to remove would be
more intuitive.
3. I'm not super thrilled with the cache configuration since it seems to
tie us a bit too closely to the implementation. You've mostly convinced me
on the need for the slots config, but I wonder if we can at least do
without "min.incremental.fetch.session.eviction.ms"? For one, I think the
broker should reserve the right to evict sessions at will. We shouldn't be
stuck maintaining a small session at the expense of a much larger one just
to enforce this timeout. Internally, I think having some cache stickiness
to avoid thrashing makes sense, but I think static values are likely to be
good enough and that lets us retain some flexibility to change the behavior
in the future.
4. I think the word "incremental" is redundant in the config names. Maybe
it could just be "max.fetch.session.cache.slots" for example?

Thanks,
Jason



On Sat, Jan 20, 2018 at 12:54 PM, Colin McCabe  wrote:

> On Fri, Jan 19, 2018, at 15:02, Jun Rao wrote:
> > Hi, Colin,
> >
> > Thanks for the KIP. Looks good to me overall. Just a couple of more
> > comments.
> >
> > 1. As I mentioned earlier, it might be useful to add some metrics for
> > monitoring the usage of the session cache. For example, it would be
> useful
> > to know how many slots are being used (or unused), # of total partitions
> in
> > the cached slots (to understand space), the eviction rate (to see if
> there
> > is any churn), etc.
>
> Thanks, Jun.  Sorry-- I meant to address this earlier, but I forgot about
> it.  I just added some proposed metrics to the KIP wiki.
>
> >
> > 2. Using max_bytes to 0 represent the removal of a partition seems
> > unintuitive. Perhaps it's better to either add a flag per partition or
> add
> > a removed partition list.
>
> Perhaps if we use max_bytes -1 to represent removal, it will be more
> intuitive?  After all, -1 bytes is clearly not a valid amount of bytes to
> fetch.  Or should be have a separate array of removed TopicPartitions?
>
> On a related note, in the FetchResponse#PartitionData, we have an "error"
> field, plus highWatermark, lastStableOffset, logStartOffset, etc.  But when
> the "error" field is set, those other fields are not used.  Perhaps we
> could save some space by just having a separate array of "partitions with
> errors."  In the common case where there are no errors, this would save 2
> bytes per partition, which could be quite significant in large responses.
>
> best,
> Colin
>
> >
> > Jun
> >
> >
> > On Thu, Jan 18, 2018 at 6:15 PM, Colin McCabe 
> wrote:
> >
> > > Hi all,
> > >
> > > I updated the KIP.  There is also an implementation of this KIP here:
> > > https://github.com/apache/kafka/pull/4418
> > >
> > > The updated implementation simplifies a few things, and adds the
> ability
> > > to incrementally add or remove individual partitions in an incremental
> > > fetch request.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Tue, Dec 19, 2017, at 19:28, Colin McCabe wrote:
> > > > Hi all,
> > > >
> > > > I'd like to start the vote on KIP-227: Incremental Fetch Requests.
> > > >
> > > > The KIP is here:
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 227%3A+Introduce+Incremental+FetchRequests+to+Increase+
> > > Partition+Scalability
> > > >
> > > > and discussion thread earlier:
> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg83115.html
> > > >
> > > > thanks,
> > > > Colin
> > >
>

Build failed in Jenkins: kafka-trunk-jdk8 #2344

[Apache Jenkins Server]

See 


Changes:

[wangguoz] MINOR: update docs with regard to improved resilience of Kafka 
Streams

[wangguoz] MINOR: increase timeout for unstable

[me] MINOR: Add async and different sync startup modes in connect service

--
[...truncated 135.76 KB...]
:277:
 error writing class TestPurgatoryPerformance$CompletionQueue$Scheduled: 
:
 

 is not a directory
private class Scheduled(val operation: FakeOperation) extends Delayed {
  ^
:185:
 error writing class TestPurgatoryPerformance$ExponentialDistribution: 
:
 

 is not a directory
  private class ExponentialDistribution(lambda: Double) {
^
:240:
 error writing class TestPurgatoryPerformance$FakeOperation: 
:
 

 is not a directory
  private class FakeOperation(delayMs: Long, size: Int, val latencyMs: Long, 
latch: CountDownLatch) extends DelayedOperation(delayMs) {
^
:217:
 error writing class TestPurgatoryPerformance$IntervalSamples: 
:
 

 is not a directory
  private class IntervalSamples(sampleSize: Int, requestPerSecond: Double) {
^
:222:
 error writing <$anon: Function1>: 
:
 

 is not a directory
  (0 until sampleSize).map { _ =>
   ^
:235:
 error writing <$anon: Function1>: 
:
 

 is not a directory
  .format(1000d / (samples.map(_.toDouble).sum / sampleSize.toDouble), 
samples.min, samples.max)
 ^
:196:
 error writing class TestPurgatoryPerformance$LatencySamples: 
:
 

 is not a directory
  private class LatencySamples(sampleSize: Int, pct75: Double, pct50: Double) {
^
:202:
 error writing <$anon: Function1>: 
:
 

 is not a directory
  (0 until sampleSize).map { _ => dist.next().toLong }.toArray
   ^
:175:
 error writing class TestPurgatoryPerformance$LogNormalDistribution: 
:
 

Re: [VOTE] KIP-229: DeleteGroups API

[Jason Gustafson]

+1 (binding)

On Mon, Jan 22, 2018 at 11:06 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Bumping this thread as this KIP requires only one more binding +1 to make
> it to 1.1.0.
>
> Thanks.
> --Vahid
>
>
>
> From:   "Vahid S Hashemian" 
> To: dev@kafka.apache.org
> Date:   01/16/2018 10:41 AM
> Subject:Re: [VOTE] KIP-229: DeleteGroups API
>
>
>
> Thanks Colin.
>
> Your suggestion is fair. I added a GROUP_ID_NOT_FOUND error to cover the
> group deletion case where the group id does not exist.
> I hope this change is what you had in mind, and also hope it does not void
>
> the votes cast so far :)
>
> --Vahid
>
>
>
>
> From:   Colin McCabe 
> To: dev@kafka.apache.org
> Date:   01/16/2018 10:06 AM
> Subject:Re: [VOTE] KIP-229: DeleteGroups API
>
>
>
> Thanks for this KIP, Vahid.  +1 (non-binding).
>
> How about creating a GROUP_ID_NOT_FOUND error, rather than re-using
> INVALID_GROUP_ID here?  INVALID_GROUP_ID is used to indicate that the
> group id itself is bad (contains invalid characters, is an empty string,
> etc.).  A group ID not being found on the server is different than the ID
> itself not being valid.  We should probably not combine these two error
> cases.
>
> On Tue, Jan 16, 2018, at 09:31, Guozhang Wang wrote:
> > Thanks Vahid, +1 (binding) from me.
> >
> > A minor question as for tooling improvements you mentioned in the wiki,
> are
> > we going to add a `--delete` option on the `kafka-consumer-groups.sh`
> > script? I thought it is "yes" but the Proposed Changes section does not
> > explicitly mention it.
>
> +1 for a --delete option on kafka-consumer-groups.sh
>
> regards,
> Colin
>
> >
> >
> > Guozhang
> >
> >
> > On Tue, Jan 16, 2018 at 5:55 AM, Mickael Maison
> 
> > wrote:
> >
> > > +1 (non binding)
> > > Thanks for the KIP
> > >
> > > On Tue, Jan 16, 2018 at 11:54 AM, Rajini Sivaram
> > >  wrote:
> > > > Hi Vahid,
> > > >
> > > > +1 (binding)
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > > > On Tue, Jan 16, 2018 at 10:24 AM, Edoardo Comar 
> > > wrote:
> > > >
> > > >> +1 (non binding) - thanks Vahid
> > > >>
> > > >> --
> > > >>
> > > >> Edoardo Comar
> > > >>
> > > >> IBM Message Hub
> > > >>
> > > >> IBM UK Ltd, Hursley Park, SO21 2JN
> > > >>
> > > >>
> > > >>
> > > >> From:   Ted Yu 
> > > >> To: dev@kafka.apache.org
> > > >> Date:   15/01/2018 20:33
> > > >> Subject:Re: [VOTE] KIP-229: DeleteGroups API
> > > >>
> > > >>
> > > >>
> > > >> +1
> > > >>
> > > >> On Mon, Jan 15, 2018 at 12:22 PM, Jeff Widman 
> > > wrote:
> > > >>
> > > >> > +1 (non-binding)
> > > >> >
> > > >> > On Jan 15, 2018 10:23 AM, "Vahid S Hashemian"
> > > >> 
> > > >> > wrote:
> > > >> >
> > > >> > > Happy Monday,
> > > >> > >
> > > >> > > I believe the concerns on this KIP have been addressed in the
> > > current
> > > >> > > version of the KIP:
> > > >> > >
> > > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> > > >> apache.org_confluence_display_KAFKA_KIP-2D=DwIBaQ=jf_
> > > >>
> iaSHvJObTbx-siA1ZOg=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ=
> > > >> gy9vDnDO2DIaWpZBxdO89w7Pxx5mZNKksKLbjB4_Yp4=XYsVEz3hq1Qyb8Oho_
> > > >> z7iSlv88IVioUD4iIQ9gp_Trc=
> > > >>
> > > >> > > 229%3A+DeleteGroups+API
> > > >> > > So I'd like to start a vote.
> > > >> > >
> > > >> > > Thanks.
> > > >> > > --Vahid
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> Unless stated otherwise above:
> > > >> IBM United Kingdom Limited - Registered in England and Wales with
> number
> > > >> 741598.
> > > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> PO6
> > > 3AU
> > > >>
> > >
> >
> >
> >
> > --
> > -- Guozhang
>
>
>
>
>
>
>
>
>
>

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

[Gwen Shapira]

+1 (binding)

This is going to be HUGE! Thank you Randall.

On Mon, Jan 22, 2018 at 1:18 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:

> Great addition!
>
> +1 (non-binding)
>
> Konstantine
>
> On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava 
> wrote:
>
> > +1 (binding)
> >
> > Thanks for the work on this -- not a small upgrade to the Connect APIs!
> >
> > -Ewen
> >
> > On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch  wrote:
> >
> > > Hi everyone,
> > >
> > > I'd like to start the voting on this KIP to add support for headers in
> > > Connect.:
> > >
> > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 145+-+Expose+Record+Headers+in+Kafka+Connect
> > >  > > 145+-+Expose+Record+Headers+in+Kafka+Connect>*
> > >
> > > This does add a fair number of interfaces to our public API, and
> defines
> > > some behavioral changes as well.
> > >
> > > Thanks! Your feedback is highly appreciated.
> > >
> > > Randall
> > >
> >
>

[jira] [Resolved] (KAFKA-3988) KafkaConfigBackingStore assumes configs will be stored as schemaless maps

[Ewen Cheslack-Postava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ewen Cheslack-Postava resolved KAFKA-3988.
--
Resolution: Won't Fix

Not going to fix since KIP-174 is deprecating the internal converter configs 
and will instead always use schemaless JsonConverter.

> KafkaConfigBackingStore assumes configs will be stored as schemaless maps
> -
>
> Key: KAFKA-3988
> URL: https://issues.apache.org/jira/browse/KAFKA-3988
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 0.10.0.0
>Reporter: Ewen Cheslack-Postava
>Assignee: Ewen Cheslack-Postava
>Priority: Major
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> If you use an internal key/value converter that drops schema information (as 
> is the default in the config files we provide since we use JsonConverter with 
> schemas.enable=false), the schemas we use that are structs get converted to 
> maps since we don't know the structure to decode them to. Because our tests 
> run with these settings, we haven't validated that the code works if schemas 
> are preserved.
> When they are preserved, we'll hit an error message like this
> {quote}
> [2016-07-25 07:36:34,828] ERROR Found connector configuration 
> (connector-test-mysql-jdbc) in wrong format: class 
> org.apache.kafka.connect.data.Struct 
> (org.apache.kafka.connect.storage.KafkaConfigBackingStore:498)
> {quote}
> because the code currently checks that it is working with a map. We should 
> actually be checking for either a Struct or a Map. This same problem probably 
> affects a couple of other types of data in the same class as Connector 
> configs, Task configs, Connect task lists, and target states are all Structs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Release Plan for 1.0.1

[Ewen Cheslack-Postava]

Jeff,

The initial email was just to get people looking at JIRAs. We still have
one outstanding blocker, and Guozhang's PR has already been merged.
Otherwise we have no blockers, but there are a couple marked "Critical"
that are unresolved. I'll be following up on those JIRAs to get them either
moved to a later release or upgraded to blockers with folks working on
resolving them.

We'll probably do an RC either late this week or early next week. Since
this is a bug fix release, I am guessing it will only require 1 RC.

-Ewen

On Mon, Jan 22, 2018 at 11:23 AM, Jeff Widman  wrote:

> Any update on this release?
>
> Haven't seen anything other than Guozhang's email...
>
> Waiting for it to drop so we can upgrade to 1.0 series...
>
> On Tue, Jan 16, 2018 at 1:32 PM, Guozhang Wang  wrote:
>
> > Hello Ewen,
> >
> > Could you include one more notable changes in 1.0.1:
> > https://issues.apache.org/jira/browse/KAFKA-6398 ?
> >
> > My PR is ready for reviews and should be mergable at any time.
> >
> >
> > Guozhang
> >
> > On Tue, Jan 16, 2018 at 10:54 AM, Ewen Cheslack-Postava <
> e...@confluent.io
> > >
> > wrote:
> >
> > > Hi all,
> > >
> > > I'd like to start the process for doing a 1.0.1 bug fix release. 1.0.0
> > was
> > > released Nov 1, 2017, and about 2.5 mos have passed and 32 bug fixes
> have
> > > accumulated so far. A few of the more notable fixes that we've merged
> so
> > > far:
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-6269 - KTable restore
> fails
> > > after rebalance
> > > https://issues.apache.org/jira/browse/KAFKA-6185 - Selector memory
> leak
> > > with high likelihood of OOM in case of down conversion
> > > https://issues.apache.org/jira/browse/KAFKA-6167 - Timestamp on
> streams
> > > directory contains a colon, which is an illegal character
> > > https://issues.apache.org/jira/browse/KAFKA-6190 - GlobalKTable never
> > > finishes restoring when consuming transactional messages
> > > https://issues.apache.org/jira/browse/KAFKA-6252 - A fix for cleaning
> up
> > > new Connect metrics when a connector does not shut down properly
> > >
> > > These represent important fixes across all components -- core, Connect
> > and
> > > Streams.
> > >
> > > I've prepared
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.0.1
> to
> > > help track the fixes for 1.0.1. As you can see we have only 1 item
> > > currently marked as a blocker for 1.0.1. If folks with outstanding bugs
> > > could take a pass and a) make sure anything they think should go into
> > 1.0.1
> > > is marked as such, with the appropriate priority, and b) make sure
> > anything
> > > marked for 1.0.1 should really be there.
> > >
> > > Once people have taken a pass we can work on a VOTE thread and getting
> > any
> > > outstanding PRs reviewed.
> > >
> > > -Ewen
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
>
> *Jeff Widman*
> jeffwidman.com  | 740-WIDMAN-J (943-6265)
> <><
>

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

[Konstantine Karantasis]

Great addition!

+1 (non-binding)

Konstantine

On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava 
wrote:

> +1 (binding)
>
> Thanks for the work on this -- not a small upgrade to the Connect APIs!
>
> -Ewen
>
> On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch  wrote:
>
> > Hi everyone,
> >
> > I'd like to start the voting on this KIP to add support for headers in
> > Connect.:
> >
> > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 145+-+Expose+Record+Headers+in+Kafka+Connect
> >  > 145+-+Expose+Record+Headers+in+Kafka+Connect>*
> >
> > This does add a fair number of interfaces to our public API, and defines
> > some behavioral changes as well.
> >
> > Thanks! Your feedback is highly appreciated.
> >
> > Randall
> >
>

Re: [VOTE] KIP-208: Add SSL support to Kafka Connect REST interface

[Jakub Scholz]

Hi all,

I will close the vote for this KIP. It received 4 binding +1 votes and is
now approved. Thanks to everyone for voted and discussed this.

The PR for this KIP is at https://github.com/apache/kafka/pull/4429

Thanks & Regards
Jakub

On Sat, Jan 20, 2018 at 12:07 AM, Jakub Scholz  wrote:

> @Jason: Actually ... wouldn't it be better to name it only "
> rest.advertised.listener"? The "security" doesn't make much sense without
> the protocol. What do you think?
>
> On Fri, Jan 19, 2018 at 10:51 PM, Jakub Scholz  wrote:
>
>> Hi Jason,
>>
>> Thanks for the vote. Yeah, I think that renaming it to "
>> rest.advertised.security.listener" is good idea. Unless someone else
>> objects, I will update the KIP.
>>
>> Thanks & Regards
>> Jakub
>>
>> On Fri, Jan 19, 2018 at 6:09 PM, Jason Gustafson 
>> wrote:
>>
>>> +1 from me. I just had one suggestion. I saw in the discussion thread
>>> that
>>> you considered allowing multiple listeners for each protocol type, but
>>> ultimately rejected it. Maybe to leave the door a little more open in the
>>> future for this, we could rename the config
>>> "rest.advertised.security.protocol" to "rest.advertised.security.list
>>> ener"
>>> so that it will still make sense if we introduce a listener labeling
>>> approach similar to Kafka?
>>>
>>> Thanks,
>>> Jason
>>>
>>> On Fri, Jan 19, 2018 at 8:31 AM, Damian Guy 
>>> wrote:
>>>
>>> > Thanks Jakub!
>>> >
>>> > +1 (binding)
>>> >
>>> > On Thu, 18 Jan 2018 at 23:49 Jakub Scholz  wrote:
>>> >
>>> > > Hi all,
>>> > >
>>> > > We still need at least 2 more binding +1s. I think that the PR (
>>> > > https://github.com/apache/kafka/pull/4429) is shaping good. If we
>>> get
>>> > the
>>> > > votes, we should be able to make the 1.1.0 release.
>>> > >
>>> > > Thanks & Regards
>>> > > Jakub
>>> > >
>>> > > On Fri, Jan 5, 2018 at 4:30 AM, Ewen Cheslack-Postava <
>>> e...@confluent.io
>>> > >
>>> > > wrote:
>>> > >
>>> > > > Jakub,
>>> > > >
>>> > > > I left a few comments in the discuss thread, but I'll also reply
>>> here
>>> > > just
>>> > > > to bump the VOTE thread's visibility. I would like to resolve the
>>> few
>>> > > > comments I left, but I am effectively +1 on this, the comments I
>>> left
>>> > > were
>>> > > > mainly details.
>>> > > >
>>> > > > Committers that could help with the necessary votes would probably
>>> be
>>> > > Gwen
>>> > > > and Jason (but others more than welcome to help out too :)
>>> > > >
>>> > > > -Ewen
>>> > > >
>>> > > > On Mon, Nov 6, 2017 at 1:52 AM, Jakub Scholz 
>>> wrote:
>>> > > >
>>> > > > > Hi all,
>>> > > > >
>>> > > > > Just a reminder that htis is still up for vote. I think this is
>>> > > important
>>> > > > > featrue which would deserve your votes.
>>> > > > >
>>> > > > > Regards
>>> > > > > Jakub
>>> > > > >
>>> > > > > On Mon, Oct 30, 2017 at 9:24 PM, Jakub Scholz 
>>> > wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > It seems there are no more comments for this KIP, so I would
>>> like
>>> > to
>>> > > > > start
>>> > > > > > the voting .
>>> > > > > >
>>> > > > > > For more details about the KIP-208 go to
>>> > > > > > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> > > > > 208%3A+Add+SSL+support+to+Kafka+Connect+REST+interface
>>> > > > > > >> > > > > 208%3A+Add+SSL+support+to+Kafka+Connect+REST+interface>*
>>> > > > > >
>>> > > > > > Thanks & Regards
>>> > > > > > Jakub
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: [DISCUSS] Release Plan for 1.0.1

[Jeff Widman]

Any update on this release?

Haven't seen anything other than Guozhang's email...

Waiting for it to drop so we can upgrade to 1.0 series...

On Tue, Jan 16, 2018 at 1:32 PM, Guozhang Wang  wrote:

> Hello Ewen,
>
> Could you include one more notable changes in 1.0.1:
> https://issues.apache.org/jira/browse/KAFKA-6398 ?
>
> My PR is ready for reviews and should be mergable at any time.
>
>
> Guozhang
>
> On Tue, Jan 16, 2018 at 10:54 AM, Ewen Cheslack-Postava  >
> wrote:
>
> > Hi all,
> >
> > I'd like to start the process for doing a 1.0.1 bug fix release. 1.0.0
> was
> > released Nov 1, 2017, and about 2.5 mos have passed and 32 bug fixes have
> > accumulated so far. A few of the more notable fixes that we've merged so
> > far:
> >
> > https://issues.apache.org/jira/browse/KAFKA-6269 - KTable restore fails
> > after rebalance
> > https://issues.apache.org/jira/browse/KAFKA-6185 - Selector memory leak
> > with high likelihood of OOM in case of down conversion
> > https://issues.apache.org/jira/browse/KAFKA-6167 - Timestamp on streams
> > directory contains a colon, which is an illegal character
> > https://issues.apache.org/jira/browse/KAFKA-6190 - GlobalKTable never
> > finishes restoring when consuming transactional messages
> > https://issues.apache.org/jira/browse/KAFKA-6252 - A fix for cleaning up
> > new Connect metrics when a connector does not shut down properly
> >
> > These represent important fixes across all components -- core, Connect
> and
> > Streams.
> >
> > I've prepared
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.0.1 to
> > help track the fixes for 1.0.1. As you can see we have only 1 item
> > currently marked as a blocker for 1.0.1. If folks with outstanding bugs
> > could take a pass and a) make sure anything they think should go into
> 1.0.1
> > is marked as such, with the appropriate priority, and b) make sure
> anything
> > marked for 1.0.1 should really be there.
> >
> > Once people have taken a pass we can work on a VOTE thread and getting
> any
> > outstanding PRs reviewed.
> >
> > -Ewen
> >
>
>
>
> --
> -- Guozhang
>



-- 

*Jeff Widman*
jeffwidman.com  | 740-WIDMAN-J (943-6265)
<><

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

[Vahid S Hashemian]

And this one requires two more binding +1's :)

Thanks.



From:   Ewen Cheslack-Postava 
To: dev@kafka.apache.org
Date:   01/02/2018 09:50 PM
Subject:Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups 
API



+1 (binding)

Thanks for the KIP Vahid, nice improvement!

-Ewen

On Tue, Dec 19, 2017 at 11:30 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> I believe the concerns on this KIP have been addressed so far.
> Therefore, I'd like to start a vote.
>
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D=DwIBaQ=jf_iaSHvJObTbx-siA1ZOg=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc=qA7xTKn3gB0jf1lj1AQ3oFqsUZ9YooTz69QNqRGMs00=JNhAeUvZiILPhLdH011wge0zOn5TYMCpVHu2WoXefN8=

> 231%3A+Improve+the+Required+ACL+of+ListGroups+API
>
> Thanks.
> --Vahid
>
>

Re: [VOTE] KIP-229: DeleteGroups API

[Vahid S Hashemian]

Bumping this thread as this KIP requires only one more binding +1 to make 
it to 1.1.0.

Thanks.
--Vahid



From:   "Vahid S Hashemian" 
To: dev@kafka.apache.org
Date:   01/16/2018 10:41 AM
Subject:Re: [VOTE] KIP-229: DeleteGroups API



Thanks Colin.

Your suggestion is fair. I added a GROUP_ID_NOT_FOUND error to cover the 
group deletion case where the group id does not exist.
I hope this change is what you had in mind, and also hope it does not void 

the votes cast so far :)

--Vahid




From:   Colin McCabe 
To: dev@kafka.apache.org
Date:   01/16/2018 10:06 AM
Subject:Re: [VOTE] KIP-229: DeleteGroups API



Thanks for this KIP, Vahid.  +1 (non-binding).

How about creating a GROUP_ID_NOT_FOUND error, rather than re-using 
INVALID_GROUP_ID here?  INVALID_GROUP_ID is used to indicate that the 
group id itself is bad (contains invalid characters, is an empty string, 
etc.).  A group ID not being found on the server is different than the ID 
itself not being valid.  We should probably not combine these two error 
cases.

On Tue, Jan 16, 2018, at 09:31, Guozhang Wang wrote:
> Thanks Vahid, +1 (binding) from me.
> 
> A minor question as for tooling improvements you mentioned in the wiki, 
are
> we going to add a `--delete` option on the `kafka-consumer-groups.sh`
> script? I thought it is "yes" but the Proposed Changes section does not
> explicitly mention it.

+1 for a --delete option on kafka-consumer-groups.sh

regards,
Colin

> 
> 
> Guozhang
> 
> 
> On Tue, Jan 16, 2018 at 5:55 AM, Mickael Maison 

> wrote:
> 
> > +1 (non binding)
> > Thanks for the KIP
> >
> > On Tue, Jan 16, 2018 at 11:54 AM, Rajini Sivaram
> >  wrote:
> > > Hi Vahid,
> > >
> > > +1 (binding)
> > >
> > > Thanks for the KIP.
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > > On Tue, Jan 16, 2018 at 10:24 AM, Edoardo Comar 
> > wrote:
> > >
> > >> +1 (non binding) - thanks Vahid
> > >>
> > >> --
> > >>
> > >> Edoardo Comar
> > >>
> > >> IBM Message Hub
> > >>
> > >> IBM UK Ltd, Hursley Park, SO21 2JN
> > >>
> > >>
> > >>
> > >> From:   Ted Yu 
> > >> To: dev@kafka.apache.org
> > >> Date:   15/01/2018 20:33
> > >> Subject:Re: [VOTE] KIP-229: DeleteGroups API
> > >>
> > >>
> > >>
> > >> +1
> > >>
> > >> On Mon, Jan 15, 2018 at 12:22 PM, Jeff Widman 
> > wrote:
> > >>
> > >> > +1 (non-binding)
> > >> >
> > >> > On Jan 15, 2018 10:23 AM, "Vahid S Hashemian"
> > >> 
> > >> > wrote:
> > >> >
> > >> > > Happy Monday,
> > >> > >
> > >> > > I believe the concerns on this KIP have been addressed in the
> > current
> > >> > > version of the KIP:
> > >> > >
> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> > >> apache.org_confluence_display_KAFKA_KIP-2D=DwIBaQ=jf_
> > >> 
iaSHvJObTbx-siA1ZOg=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ=
> > >> gy9vDnDO2DIaWpZBxdO89w7Pxx5mZNKksKLbjB4_Yp4=XYsVEz3hq1Qyb8Oho_
> > >> z7iSlv88IVioUD4iIQ9gp_Trc=
> > >>
> > >> > > 229%3A+DeleteGroups+API
> > >> > > So I'd like to start a vote.
> > >> > >
> > >> > > Thanks.
> > >> > > --Vahid
> > >> > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> Unless stated otherwise above:
> > >> IBM United Kingdom Limited - Registered in England and Wales with 
number
> > >> 741598.
> > >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
PO6
> > 3AU
> > >>
> >
> 
> 
> 
> -- 
> -- Guozhang

Re: [VOTE] KIP-219 - Improve Quota Communication

[Becket Qin]

Thanks for the discussion and voting.

KIP-219 has passed with +3 binding votes (Becket, Jun, Rajini).

On Thu, Jan 18, 2018 at 1:32 AM, Rajini Sivaram 
wrote:

> Hi Becket,
>
> Thanks for the update. Yes, it does address my concern.
>
> +1 (binding)
>
> Regards,
>
> Rajini
>
> On Wed, Jan 17, 2018 at 5:24 PM, Becket Qin  wrote:
>
> > Actually returning an empty fetch request may still be useful to reduce
> the
> > throttle time due to request quota violation because the FetchResponse
> send
> > time will be less. I just updated the KIP.
> >
> > Rajini, does that address your concern?
> >
> > On Tue, Jan 16, 2018 at 7:01 PM, Becket Qin 
> wrote:
> >
> > > Thanks for the reply, Jun.
> > >
> > > Currently the byte rate quota does not apply to HeartbeatRequest,
> > > JoinGroupRequest/SyncGroupRequest. So the only case those requests are
> > > throttled is because the request quota is violated. In that case, the
> > > throttle time does not really matter whether we return a full
> > FetchResponse
> > > or an empty one. Would it be more consistent if we throttle based on
> the
> > > actual throttle time / channel mute time?
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> > > On Tue, Jan 16, 2018 at 3:45 PM, Jun Rao  wrote:
> > >
> > >> Hi, Jiangjie,
> > >>
> > >> You are right that the heartbeat is done in a channel different from
> the
> > >> fetch request. I think it's still useful to return an empty fetch
> > response
> > >> when the quota is violated. This way, the throttle time for the
> > heartbeat
> > >> request won't be large. I agree that we can just mute the channel for
> > the
> > >> fetch request for the throttle time computed based on a full fetch
> > >> response. This probably also partially addresses Rajini's #1 concern.
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Mon, Jan 15, 2018 at 9:27 PM, Becket Qin 
> > wrote:
> > >>
> > >> > Hi Rajini,
> > >> >
> > >> > Thanks for the comments. Pleas see the reply inline.
> > >> >
> > >> > Hi Jun,
> > >> >
> > >> > Thinking about the consumer rebalance case a bit more, I am not sure
> > if
> > >> we
> > >> > need to worry about the delayed rebalance due to quota violation or
> > not.
> > >> > The rebalance actually uses a separate channel to coordinator.
> > Therefore
> > >> > unless the request quota is hit, the rebalance won't be throttled.
> > Even
> > >> if
> > >> > request quota is hit, it seems unlikely to be delayed long. So it
> > looks
> > >> > that we don't need to unmute the channel earlier than needed. Does
> > this
> > >> > address your concern?
> > >> >
> > >> > Thanks,
> > >> >
> > >> > Jiangjie (Becket) Qin
> > >> >
> > >> > On Mon, Jan 15, 2018 at 4:22 AM, Rajini Sivaram <
> > >> rajinisiva...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Becket,
> > >> > >
> > >> > > A few questions:
> > >> > >
> > >> > > 1. KIP says: *Although older client implementations (prior to
> > >> knowledge
> > >> > of
> > >> > > this KIP) will immediately send the next request after the broker
> > >> > responds
> > >> > > without paying attention to the throttle time field, the broker is
> > >> > > protected by virtue of muting the channel for time X. i.e., the
> next
> > >> > > request will not be processed until the channel is unmuted. *
> > >> > > For fetch requests, the response is sent immediately and the mute
> > >> time on
> > >> > > the broker based on empty fetch response will often be zero
> (unlike
> > >> the
> > >> > > throttle time returned to the client based on non-empty response).
> > >> Won't
> > >> > > that lead to a tight loop of fetch requests from old clients
> > >> > (particularly
> > >> > > expensive with SSL)? Wouldn't it be better to retain current
> > behaviour
> > >> > for
> > >> > > old clients? Also, if we change the behaviour for old clients,
> > client
> > >> > > metrics that track throttle time will report a lot of throttle
> > >> unrelated
> > >> > to
> > >> > >  actual throttle time.
> > >> > >
> > >> > For consumers, if quota is violated, the throttle time on the broker
> > >> will
> > >> > not be 0. It is just that the throttle time will not be increasing
> > >> because
> > >> > the consumer will return an empty response in this case. So there
> > should
> > >> > not be a tight loop.
> > >> >
> > >> >
> > >> > > 2. KIP says: *The usual idle timeout i.e.,
> connections.max.idle.ms
> > >> > >  will still be honored during the
> > >> > throttle
> > >> > > time X. This makes sure that the brokers will detect client
> > connection
> > >> > > closure in a bounded time.*
> > >> > > Wouldn't it be better to bound maximum throttle time to
> > >> > > *connections.max.idle.ms
> > >> > > *? If we mute for a time greater
> > than
> > >> > > *connections.max.idle.ms
> > >> > > * and then close a client
> > 

Re: [VOTE] KIP-248: Create New ConfigCommand That Uses The New AdminClient

[Viktor Somogyi]

Hi all,

I'd like to ask the dev community to vote on this KIP as the KIP freeze is
very close (tomorrow) and this would be a relevant feature (mostly from the
usability side) for KIP-226: Dynamic Broker Configuration
,
which will be included in 1.1.

Thank you,
Viktor

On Fri, Jan 19, 2018 at 2:08 PM, Viktor Somogyi 
wrote:

> Hi all,
>
> I'd like to start the vote on KIP-248: Create New ConfigCommand That Uses
> The New AdminClient.
>
> The KIP can be read here: https://cwiki.apache.
> org/confluence/display/KAFKA/KIP-248+-+Create+New+
> ConfigCommand+That+Uses+The+New+AdminClient
>
> The discussion thread is at https://lists.apache.org/thread.html/
> 6c0b347a52b51f618d7b86b44d0dce65a6079764358fef619d4807c9@%
> 3Cdev.kafka.apache.org%3E.
>
> Thank you,
> Viktor
>

Re: offsetsForTimes API performance

[Andrew Otto]

Speaking of, has there been any talk of combining those two requests into a
single API call?  I’d assume that offsetForTimes + consumer seek is
probably the most common use case of offsetForTimes.  Maybe a round trip
could be avoided if the broker could just auto-assign the consumer to the
offset for a timestamp.


On Mon, Jan 22, 2018 at 9:59 AM, srimugunthan dhandapani <
srimugunthan.dhandap...@gmail.com> wrote:

>  Hi all,
>
> We use kafka as our store and  every one of our record is associated with a
> timeStamp. We pull data from kafka by seeking to a timeStamp offset
> everytime and then get the records by polling. We use KafkaConsumer's
> offsetsForTimes (
> https://kafka.apache.org/0110/javadoc/org/apache/kafka/clients/consumer/
> KafkaConsumer.html#offsetsForTimes(java.util.Map)
>
> ) API to  find offset and seek to a particular time offset.
>
>  We see that using the offsetsForTimes API and the subsequent seek to the
> offset takes anything from 17 milliseconds to 500millisec per iteration.
>
> I would like to know if anybody has done any performance testing of the
> offsetsForTimes API and what does the performance of the API depend on?
> Will the API be slower if there is more data in the kafka?
>
>
> thanks,
> mugunthan
>

offsetsForTimes API performance

[srimugunthan dhandapani]

 Hi all,

We use kafka as our store and  every one of our record is associated with a
timeStamp. We pull data from kafka by seeking to a timeStamp offset
everytime and then get the records by polling. We use KafkaConsumer's
offsetsForTimes (
https://kafka.apache.org/0110/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#offsetsForTimes(java.util.Map)

) API to  find offset and seek to a particular time offset.

 We see that using the offsetsForTimes API and the subsequent seek to the
offset takes anything from 17 milliseconds to 500millisec per iteration.

I would like to know if anybody has done any performance testing of the
offsetsForTimes API and what does the performance of the API depend on?
Will the API be slower if there is more data in the kafka?


thanks,
mugunthan

[jira] [Created] (KAFKA-6466) Kafka connect task sometimes fails on start-up

[Alexander Koval (JIRA)]

Alexander Koval created KAFKA-6466:
--

 Summary: Kafka connect task sometimes fails on start-up
 Key: KAFKA-6466
 URL: https://issues.apache.org/jira/browse/KAFKA-6466
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Affects Versions: 1.0.0
Reporter: Alexander Koval


We use kafka connect for indexing into Elasticsearch. Sometimes when we 
updating our kafka-connect application one or several tasks get {{FAILED}} 
status. Restarting connector with failed task usually helps.

 

Below is an example of the failed task's stacktrace:
{code:java}
java.lang.IllegalArgumentException: A metric named 'MetricName 
[name=offset-commit-max-time-ms, group=connector-task-metrics, description=The 
maximum time in milliseconds taken by this task to commit offsets., 
tags={connector=prom-ua-catalog-product, task=2}]' already exists, can't 
register another one.
at org.apache.kafka.common.metrics.Metrics.registerMetric(Metrics.java:532)
at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:256)
at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:241)
at 
org.apache.kafka.connect.runtime.WorkerTask$TaskMetricsGroup.(WorkerTask.java:328)
at org.apache.kafka.connect.runtime.WorkerTask.(WorkerTask.java:69)
at 
org.apache.kafka.connect.runtime.WorkerSinkTask.(WorkerSinkTask.java:98)
at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:449)
at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:404)
at 
org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:852)
at 
org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:108)
at 
org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:866)
at 
org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:862)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

[Jorge Esteban Quilcate Otoya]

My bad, KIP is updated:

```
public class MemberDescription {
private final String consumerId;
private final String clientId;
private final String host;
private final MemberAssignment assignment;
}
public class MemberAssignment {
private final List assignment;
}
```

Cheers,
Jorge.

El lun., 22 ene. 2018 a las 6:46, Jun Rao () escribió:

> Hi, Jorge,
>
> For #3, I wasn't suggesting using the internal Assignment. We can just
> introduce a new public type that wraps List. We can call it
> sth like MemberAssignment to distinguish it from the internal one. This
> makes extending the type in the future easier.
>
> Thanks,
>
> Jun
>
> On Sun, Jan 21, 2018 at 3:19 PM, Jorge Esteban Quilcate Otoya <
> quilcate.jo...@gmail.com> wrote:
>
> > Hi all,
> >
> > Thanks all for your votes and approving this KIP :)
> >
> > @Jun Rao:
> >
> > 1. Yes, KIP is updated with MemberDescription.
> > 2. Changed:
> > ```
> > public class ListGroupOffsetsResult {
> > final KafkaFuture> future;
> > ```
> > 3. Not sure about this one as Assignment type is part of
> > o.a.k.clients.consumer.internals. Will we be breaking encapsulation if we
> > expose it as part of AdminClient?
> > Currently is defined as:
> > ```
> > public class MemberDescription {
> > private final String consumerId;
> > private final String clientId;
> > private final String host;
> > private final List assignment;
> > }
> > ```
> >
> > BTW: I've created a PR with the work in progress:
> > https://github.com/apache/kafka/pull/4454
> >
> > Cheers,
> > Jorge.
> >
> > El vie., 19 ene. 2018 a las 23:52, Jun Rao ()
> escribió:
> >
> > > Hi, Jorge,
> > >
> > > Thanks for the KIP. Looks good to me overall. A few comments below.
> > >
> > > 1. It seems that ConsumerDescription should be MemberDescription?
> > >
> > > 2. Each offset can have an optional metadata. So, in
> > > ListGroupOffsetsResult, perhaps it's better to have
> > > KafkaFuture>, where
> > > OffsetAndMetadata contains an offset and a metadata of String.
> > >
> > > 3. As Jason mentioned in the discussion, it would be nice to extend
> this
> > > api to support general group management, instead of just the consumer
> > group
> > > in the future. For that, it might be better for MemberDescription to
> have
> > > assignment of type Assignment, which consists of a list of partitions.
> > > Then, in the future, we can add other fields to Assignment.
> > >
> > > Jun
> > >
> > >
> > > On Thu, Jan 18, 2018 at 9:45 AM, Mickael Maison <
> > mickael.mai...@gmail.com>
> > > wrote:
> > >
> > > > +1 (non binding), thanks
> > > >
> > > > On Thu, Jan 18, 2018 at 5:41 PM, Colin McCabe 
> > > wrote:
> > > > > +1 (non-binding)
> > > > >
> > > > > Colin
> > > > >
> > > > >
> > > > > On Thu, Jan 18, 2018, at 07:36, Ted Yu wrote:
> > > > >> +1
> > > > >>  Original message From: Bill Bejeck <
> > > bbej...@gmail.com>
> > > > >> Date: 1/18/18  6:59 AM  (GMT-08:00) To: dev@kafka.apache.org
> > Subject:
> > > > >> Re: [VOTE] KIP-222 - Add "describe consumer group" to
> > KafkaAdminClient
> > > > >> Thanks for the KIP
> > > > >>
> > > > >> +1
> > > > >>
> > > > >> Bill
> > > > >>
> > > > >> On Thu, Jan 18, 2018 at 4:24 AM, Rajini Sivaram <
> > > > rajinisiva...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > +1 (binding)
> > > > >> >
> > > > >> > Thanks for the KIP, Jorge.
> > > > >> >
> > > > >> > Regards,
> > > > >> >
> > > > >> > Rajini
> > > > >> >
> > > > >> > On Wed, Jan 17, 2018 at 9:04 PM, Guozhang Wang <
> > wangg...@gmail.com>
> > > > wrote:
> > > > >> >
> > > > >> > > +1 (binding). Thanks Jorge.
> > > > >> > >
> > > > >> > >
> > > > >> > > Guozhang
> > > > >> > >
> > > > >> > > On Wed, Jan 17, 2018 at 11:29 AM, Gwen Shapira <
> > g...@confluent.io
> > > >
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > Hey, since there were no additional comments in the
> > discussion,
> > > > I'd
> > > > >> > like
> > > > >> > > to
> > > > >> > > > resume the voting.
> > > > >> > > >
> > > > >> > > > +1 (binding)
> > > > >> > > >
> > > > >> > > > On Fri, Nov 17, 2017 at 9:15 AM Guozhang Wang <
> > > wangg...@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > > >
> > > > >> > > > > Hello Jorge,
> > > > >> > > > >
> > > > >> > > > > I left some comments on the discuss thread. The wiki page
> > > itself
> > > > >> > looks
> > > > >> > > > good
> > > > >> > > > > overall.
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > Guozhang
> > > > >> > > > >
> > > > >> > > > > On Tue, Nov 14, 2017 at 10:02 AM, Jorge Esteban Quilcate
> > > Otoya <
> > > > >> > > > > quilcate.jo...@gmail.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Added.
> > > > >> > > > > >
> > > > >> > > > > > El mar., 14 nov. 2017 a las 19:00, Ted Yu (<
> > > > yuzhih...@gmail.com>)
> > > > >> > > > > > escribió:
> > > > >> > > > > >
> > > > >> > > > > > > Please fill in