[jira] [Commented] (KAFKA-3549) Close consumers instantiated in consumer tests

[ https://issues.apache.org/jira/browse/KAFKA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242488#comment-15242488 ] Guozhang Wang commented on KAFKA-3549: -- This reminds me of KAFKA-1501 again.. BTW for all those

[jira] [Commented] (KAFKA-3549) Close consumers instantiated in consumer tests

[ https://issues.apache.org/jira/browse/KAFKA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242461#comment-15242461 ] Ewen Cheslack-Postava commented on KAFKA-3549: -- [~granthenke] [~ijuma] So, I verified that

[jira] [Commented] (KAFKA-3549) Close consumers instantiated in consumer tests

[ https://issues.apache.org/jira/browse/KAFKA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242443#comment-15242443 ] ASF GitHub Bot commented on KAFKA-3549: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-3549) Close consumers instantiated in consumer tests

[ https://issues.apache.org/jira/browse/KAFKA-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3549: - Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved

[GitHub] kafka pull request: KAFKA-3549: Close consumers instantiated in co...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1217 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242428#comment-15242428 ] Jason Rosenberg commented on KAFKA-2143: After looking at KAFKA-2477, I think that is the actual

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242412#comment-15242412 ] Jason Rosenberg commented on KAFKA-2143: Hello, I recently experienced this issue, as described in

[RELEASE] Anyone minds if we push the next RC another week away?

Hi Team, As we got more time, we merrily expended the scope of all our in-progress KIPs :) I want to be able to at least get some of them in. Do you feel that pushing the next RC from Apr 22 (next week) to Apr 29 (week after!) will be helpful? Any objections to this plan? Gwen

Re: [GitHub] kafka pull request: KAFKA-3420 Allow users to set key in KTable.to...

Sorry - This PR is actually for KAFKA-3430, I entered the wrong number, I have updated the title on the PR On Thu, Apr 14, 2016 at 10:41 PM, bbejeck wrote: > GitHub user bbejeck opened a pull request: > > https://github.com/apache/kafka/pull/1222 > > KAFKA-3420

Re: [jira] [Commented] (KAFKA-3420) Transient failure in OffsetCommitTest.testNonExistingTopicOffsetCommit

Sorry - this message is in error, I entered the wrong Jira number when issuing the pull request, I have updated the title on the PR. On Thu, Apr 14, 2016 at 10:42 PM, ASF GitHub Bot (JIRA) wrote: > > [ >

[jira] [Updated] (KAFKA-3430) Allow users to set key in KTable.toStream() and KStream

[ https://issues.apache.org/jira/browse/KAFKA-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-3430: --- Status: Patch Available (was: Open) Submitted PR for KAFKA-3430 Allow users to set key in

[jira] [Work stopped] (KAFKA-3430) Allow users to set key in KTable.toStream() and KStream

[ https://issues.apache.org/jira/browse/KAFKA-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3430 stopped by Bill Bejeck. -- > Allow users to set key in KTable.toStream() and KStream >

[jira] [Assigned] (KAFKA-3421) Update docs with new connector features

[ https://issues.apache.org/jira/browse/KAFKA-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liquan Pei reassigned KAFKA-3421: - Assignee: Liquan Pei (was: Ewen Cheslack-Postava) > Update docs with new connector features >

[jira] [Commented] (KAFKA-3420) Transient failure in OffsetCommitTest.testNonExistingTopicOffsetCommit

[ https://issues.apache.org/jira/browse/KAFKA-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242318#comment-15242318 ] ASF GitHub Bot commented on KAFKA-3420: --- GitHub user bbejeck opened a pull request:

[GitHub] kafka pull request: KAFKA-3420 Allow users to set key in KTable.to...

GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/1222 KAFKA-3420 Allow users to set key in KTable.toStream and in KStream. … … With KStream the method selectKey was added to enable getting a key from values before perfoming aggregation-by-key

please add me

Re: Scala 2.12 support

Hi Josh, We haven't started looking into that yet. We do intend to support it, but the fact that Scala 2.12 requires Java 8 introduces a bit of complexity. If we could drop support for Java 7 at the same time as adding support for Scala 2.12, things would be easier. If no-one else beats me to

Scala 2.12 support

Hello, Are there any plans to support Scala 2.12 in Kafka? I'm helping to coordinate the effort to support Scala 2.12 in Spark ( https://issues.apache.org/jira/browse/SPARK-14220) and am currently testing against the 2.12.0-M4 milestone. Our spark-streaming-kafka module depends directly on Kafka

Re: KStreams Group Rebalance Commit Error

Hi Mike, FYI, support for "max.poll.records" was added in https://github.com/apache/kafka/pull/931 (KAFKA-3007) which was not present in the streams tech preview release. It will however be in 0.10 Cheers, Geoff On Thu, Apr 7, 2016 at 4:58 AM, Michael D. Coon wrote:

[jira] [Commented] (KAFKA-2832) support exclude.internal.topics in new consumer

[ https://issues.apache.org/jira/browse/KAFKA-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242012#comment-15242012 ] ASF GitHub Bot commented on KAFKA-2832: --- Github user vahidhashemian closed the pull request at:

[GitHub] kafka pull request: KAFKA-2832: Add a consumer config option to ex...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/932 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: [VOTE] KIP-33 - Add a time based log index

Hi Jun, 11. Yes, that sounds a reasonable solution. In the latest patch I am doing the following in order: a. Create an empty time index for a log segment if there isn't one. b. For all non-active log segments, append an entry of (last_modification_time -> next_base_offset) into the index. The

Re: best way to duplicate a KStream?

great, thanks again! On Thu, Apr 14, 2016 at 12:07 PM, Guozhang Wang wrote: > Hi Greg, > > You can apply multiple operations on the same streams object in the Kafka > Streams DSL, for example: > > stream1 = builder.stream("topic1"); > > stream2 = stream1.map(/*set

Build failed in Jenkins: kafka-trunk-jdk8 #527

See Changes: [wangguoz] KAFKA-3557; Update rocksdb to 4.4.1 and patch updates to snappy and -- [...truncated 975 lines...] kafka.coordinator.GroupCoordinatorResponseTest > testInvalidGroupId

Build failed in Jenkins: kafka-trunk-jdk7 #1197

See Changes: [wangguoz] KAFKA-3557; Update rocksdb to 4.4.1 and patch updates to snappy and -- [...truncated 1604 lines...] kafka.log.OffsetIndexTest > appendTooMany PASSED

[jira] [Commented] (KAFKA-1464) Add a throttling option to the Kafka replication tool

[ https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241774#comment-15241774 ] Jason Ruckman commented on KAFKA-1464: -- Hello Neha, One problem we've run into, is we run a system

Re: best way to duplicate a KStream?

Hi Greg, You can apply multiple operations on the same streams object in the Kafka Streams DSL, for example: stream1 = builder.stream("topic1"); stream2 = stream1.map(/*set key*/).through("topic2").aggregateByKey(...); stream3 = stream1.map(/*a different

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Hi Vahid, Thanks for the proposal. I think one of the advantages of having sticky assignment would be reduce the need to cleanup local partition state between rebalances. Do you have any thoughts on how the user would take advantage of this assignor in the consumer to do this? Maybe one approach

[DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Hi all, I have started a new KIP under https://cwiki.apache.org/confluence/display/KAFKA/KIP-54+-+Sticky+Partition+Assignment+Strategy The corresponding JIRA is at https://issues.apache.org/jira/browse/KAFKA-2273 The corresponding PR is at https://github.com/apache/kafka/pull/1020 Your

[jira] [Commented] (KAFKA-3557) Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

[ https://issues.apache.org/jira/browse/KAFKA-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241582#comment-15241582 ] ASF GitHub Bot commented on KAFKA-3557: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-3557) Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

[ https://issues.apache.org/jira/browse/KAFKA-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3557: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request: KAFKA-3557; Update rocksdb to 4.4.1 and patch ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3522) Consider adding version information into rocksDB storage format

[ https://issues.apache.org/jira/browse/KAFKA-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3522: - Labels: architecture (was: semantics) > Consider adding version information into rocksDB storage

[jira] [Updated] (KAFKA-3185) Allow users to cleanup internal data

[ https://issues.apache.org/jira/browse/KAFKA-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3185: - Labels: user-experience (was: developer-experience) > Allow users to cleanup internal data >

[jira] [Updated] (KAFKA-3514) Stream timestamp computation needs some further thoughts

[ https://issues.apache.org/jira/browse/KAFKA-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3514: - Labels: architecture (was: semantics) > Stream timestamp computation needs some further thoughts

[jira] [Updated] (KAFKA-3101) Optimize Aggregation Outputs

[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3101: - Labels: architecture (was: semantics) > Optimize Aggregation Outputs >

Reg: Kafka With Kerberos/SSL [Enhancement to add option, Need suggestions on this]

Hi All, When Kafka is running on kerberoized cluster/ SSL. Can we add an option security.protocol. So, that user can given PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. This will be helpful during running console producer and console consumer. ./bin/kafka-console-producer.sh --broker-list --topic

[jira] [Created] (KAFKA-3561) Auto create through topic for KStream aggregation and join

Guozhang Wang created KAFKA-3561: Summary: Auto create through topic for KStream aggregation and join Key: KAFKA-3561 URL: https://issues.apache.org/jira/browse/KAFKA-3561 Project: Kafka

best way to duplicate a KStream?

Hi all, I'm working on a Kafka Streaming job and I've hit a case I didn't see straightforward support for. I have an input stream topic I'd like to perform two separate aggregations on, but building up two parallel sub-topologies with the same source topic at the source results in an error: "Topic

Re: [VOTE] KIP-33 - Add a time based log index

Hi, Jiangjie, 11. Rebuilding all missing time indexes will make the upgrade process longer since the log segments pre 0.10.0 don't have the time index. Could we only rebuild the missing indexes after the last flush offset? For other segments missing the time index, we just assume lastTimestamp to

[jira] [Work started] (KAFKA-3559) Task creation time taking too long in rebalance callback

[ https://issues.apache.org/jira/browse/KAFKA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3559 started by Eno Thereska. --- > Task creation time taking too long in rebalance callback >

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241488#comment-15241488 ] Jun Rao commented on KAFKA-3042: [~fpj], I don't think we log the liveBrokers in the leaderAndIsrRequest

[jira] [Work started] (KAFKA-3337) Extract selector as a separate groupBy operator for KTable aggregations

[ https://issues.apache.org/jira/browse/KAFKA-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3337 started by Matthias J. Sax. -- > Extract selector as a separate groupBy operator for KTable aggregations >

[DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

Hi to all, any news about KIP-48? Giampaolo ps: I could not reply to original mail ( http://mail-archives.apache.org/mod_mbox/kafka-dev/201602.mbox/%3cd2f60a7c.61f2c%25pbrahmbh...@hortonworks.com%3E) because I've subscribed after and it's not on my inbox. Please, reply to the original instead

[jira] [Commented] (KAFKA-3560) Kafka is not working on Solaris

[ https://issues.apache.org/jira/browse/KAFKA-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241383#comment-15241383 ] Marcus Gründler commented on KAFKA-3560: Everything is working fine with the same version on Linux

[jira] [Commented] (KAFKA-3560) Kafka is not working on Solaris

[ https://issues.apache.org/jira/browse/KAFKA-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241380#comment-15241380 ] Marcus Gründler commented on KAFKA-3560: The same is happening when using the multi broker setup

[jira] [Commented] (KAFKA-3558) Add compression_type parameter to benchmarks in benchmark_test.py

[ https://issues.apache.org/jira/browse/KAFKA-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241356#comment-15241356 ] ASF GitHub Bot commented on KAFKA-3558: --- Github user ijuma closed the pull request at:

[GitHub] kafka pull request: KAFKA-3558; Add compression_type parameter to ...

Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/1221 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3560) Kafka is not working on Solaris

[ https://issues.apache.org/jira/browse/KAFKA-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Gründler updated KAFKA-3560: --- Attachment: state-change.log server.log log-cleaner.log

[jira] [Assigned] (KAFKA-3559) Task creation time taking too long in rebalance callback

[ https://issues.apache.org/jira/browse/KAFKA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3559: --- Assignee: Eno Thereska > Task creation time taking too long in rebalance callback >

[jira] [Created] (KAFKA-3560) Kafka is not working on Solaris

Marcus Gründler created KAFKA-3560: -- Summary: Kafka is not working on Solaris Key: KAFKA-3560 URL: https://issues.apache.org/jira/browse/KAFKA-3560 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-3559) Task creation time taking too long in rebalance callback

Guozhang Wang created KAFKA-3559: Summary: Task creation time taking too long in rebalance callback Key: KAFKA-3559 URL: https://issues.apache.org/jira/browse/KAFKA-3559 Project: Kafka Issue

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241314#comment-15241314 ] Flavio Junqueira commented on KAFKA-3042: - [~junrao] In this comment:

[jira] [Updated] (KAFKA-3558) Add compression_type parameter to benchmarks in benchmark_test.py

[ https://issues.apache.org/jira/browse/KAFKA-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3558: --- Status: Patch Available (was: Open) > Add compression_type parameter to benchmarks in

[jira] [Commented] (KAFKA-3554) Generate actual data with specific compression ratio in the ProducerPerformance tool.

[ https://issues.apache.org/jira/browse/KAFKA-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241270#comment-15241270 ] Ismael Juma commented on KAFKA-3554: Thanks for filing this [~becket_qin], I agree that this is

[jira] [Commented] (KAFKA-3558) Add compression_type parameter to benchmarks in benchmark_test.py

[ https://issues.apache.org/jira/browse/KAFKA-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241259#comment-15241259 ] ASF GitHub Bot commented on KAFKA-3558: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: KAFKA-3558; Add compression_type parameter to ...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1221 KAFKA-3558; Add compression_type parameter to benchmarks in benchmark_test.py * Accept an optional `compress_type` argument in `EndToEndLatency.scala` * Use a fixed `Random` seed in

[jira] [Assigned] (KAFKA-3183) Add metrics for persistent store caching layer

[ https://issues.apache.org/jira/browse/KAFKA-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3183: --- Assignee: Eno Thereska > Add metrics for persistent store caching layer >

[jira] [Updated] (KAFKA-3548) Locale is not handled properly in kafka-consumer

[ https://issues.apache.org/jira/browse/KAFKA-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-3548: -- Status: Patch Available (was: Open) > Locale is not handled properly in kafka-consumer >

[jira] [Commented] (KAFKA-3548) Locale is not handled properly in kafka-consumer

[ https://issues.apache.org/jira/browse/KAFKA-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240838#comment-15240838 ] ASF GitHub Bot commented on KAFKA-3548: --- GitHub user rajinisivaram opened a pull request:

[GitHub] kafka pull request: KAFKA-3548: Use root locale for case transform...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/1220 KAFKA-3548: Use root locale for case transformation of constant strings For enums and other constant strings, use locale independent case conversions to enable comparisons to work regardless

[jira] [Created] (KAFKA-3558) Add compression_type parameter to benchmarks in benchmark_test.py

Ismael Juma created KAFKA-3558: -- Summary: Add compression_type parameter to benchmarks in benchmark_test.py Key: KAFKA-3558 URL: https://issues.apache.org/jira/browse/KAFKA-3558 Project: Kafka

[jira] [Resolved] (KAFKA-3544) Missing topics on startup

[ https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Fodor resolved KAFKA-3544. --- Resolution: Not A Problem > Missing topics on startup > - > >

[jira] [Commented] (KAFKA-3544) Missing topics on startup

[ https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240723#comment-15240723 ] Greg Fodor commented on KAFKA-3544: --- Ah this makes sense. Fortunately thanks to the other responses

[jira] [Updated] (KAFKA-3557) Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

[ https://issues.apache.org/jira/browse/KAFKA-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3557: --- Status: Patch Available (was: Open) > Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

[jira] [Commented] (KAFKA-3557) Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

[ https://issues.apache.org/jira/browse/KAFKA-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240689#comment-15240689 ] ASF GitHub Bot commented on KAFKA-3557: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: MINOR: Patch version updates for snappy and sl...

Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/1218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-3557; Update rocksdb to 4.4.1 and patch ...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1219 KAFKA-3557; Update rocksdb to 4.4.1 and patch updates to snappy and slf4j * The hope is that RocksDb 4.4.1 is more stable than 4.1.0 (occasional segfaults) and 4.2.0 (very frequent segfaults),

[jira] [Created] (KAFKA-3557) Update rocksdb to 4.4.1 and patch updates to snappy and slf4j

Ismael Juma created KAFKA-3557: -- Summary: Update rocksdb to 4.4.1 and patch updates to snappy and slf4j Key: KAFKA-3557 URL: https://issues.apache.org/jira/browse/KAFKA-3557 Project: Kafka