[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352387#comment-15352387 ] Henry Cai commented on KAFKA-3904: -- Please take a look at this PR for the fix:

[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352386#comment-15352386 ] ASF GitHub Bot commented on KAFKA-3904: --- GitHub user HenryCaiHaiying opened a pull request:

[GitHub] kafka pull request #1563: KAFKA-3904: File descriptor leaking (Too many open...

GitHub user HenryCaiHaiying opened a pull request: https://github.com/apache/kafka/pull/1563 KAFKA-3904: File descriptor leaking (Too many open files) for long ru… …nning stream process I noticed when my application was running for more than one day, I will get 'Too

[jira] [Comment Edited] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352378#comment-15352378 ] Xing Huang edited comment on KAFKA-3905 at 6/28/16 5:22 AM: I think

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352378#comment-15352378 ] Xing Huang commented on KAFKA-3905: --- I think IllegalArgumentException with an error message will be

[jira] [Commented] (KAFKA-3089) VerifiableProducer should do a clean shutdown in stop_node()

[ https://issues.apache.org/jira/browse/KAFKA-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352205#comment-15352205 ] ASF GitHub Bot commented on KAFKA-3089: --- Github user lindong28 closed the pull request at:

[GitHub] kafka pull request #755: KAFKA-3089; VerifiableProducer should do a clean sh...

Github user lindong28 closed the pull request at: https://github.com/apache/kafka/pull/755 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3522) Consider adding version information into rocksDB storage format

[ https://issues.apache.org/jira/browse/KAFKA-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352172#comment-15352172 ] Ishita Mandhan commented on KAFKA-3522: --- I'd like to work on this once a consensus has been reached

[jira] [Commented] (KAFKA-3885) Kafka new producer cannot failover

[ https://issues.apache.org/jira/browse/KAFKA-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352163#comment-15352163 ] wateray commented on KAFKA-3885: [~junrao] Is it a defect? > Kafka new producer cannot failover >

[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352126#comment-15352126 ] Henry Cai commented on KAFKA-3904: -- I took a look at FileChannel.open, looks like it will still create a

[jira] [Commented] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352103#comment-15352103 ] Jan Filipiak commented on KAFKA-3705: - I will just shoot a quick reply now, time somehow became sparse

Re: KTable.filter usage, memory consumption and materialized view semantics

The boolean flag can be reset by a child operator which requires the source to be materialized, more details can be found in this design wiki: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65143671 But just to give you a concrete idea, if your topology is defined as: KTable

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Hi Jason, Thanks for the thoughtful comments. Please see my response below. BTW, I have been trying to update the KIP with some of the recent discussions on the mailing list. Regards, --Vahid From: Jason Gustafson To: dev@kafka.apache.org Date: 06/27/2016

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352034#comment-15352034 ] Ismael Juma commented on KAFKA-3905: [~wushujames], I agree. > remove null from subscribed topics in

[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352025#comment-15352025 ] Ismael Juma commented on KAFKA-3904: By the way, there's no need to create a `RandomAccessFile` just

[jira] [Commented] (KAFKA-3908) Set SendBufferSize for socket used by Processor

[ https://issues.apache.org/jira/browse/KAFKA-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351977#comment-15351977 ] ASF GitHub Bot commented on KAFKA-3908: --- GitHub user lindong28 opened a pull request:

[GitHub] kafka pull request #1562: KAFKA-3908; Set SendBufferSize for socket used by ...

GitHub user lindong28 opened a pull request: https://github.com/apache/kafka/pull/1562 KAFKA-3908; Set SendBufferSize for socket used by Processor You can merge this pull request into a Git repository by running: $ git pull https://github.com/lindong28/kafka KAFKA-3908

[jira] [Created] (KAFKA-3908) Set SendBufferSize for socket used by Processor

Dong Lin created KAFKA-3908: --- Summary: Set SendBufferSize for socket used by Processor Key: KAFKA-3908 URL: https://issues.apache.org/jira/browse/KAFKA-3908 Project: Kafka Issue Type: Bug

Re: KTable.filter usage, memory consumption and materialized view semantics

Then I don't see any simple solution here at least for a novice, especially since I don't know what can trigger the boolean flag to true. On 27 Jun 2016 5:38 p.m., "Guozhang Wang" wrote: > My concern is that, the overhead of requesting the source KTable to be > materialized

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351906#comment-15351906 ] James Cheng commented on KAFKA-3905: I think it's better to throw an error, instead of silently

[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351901#comment-15351901 ] Guozhang Wang commented on KAFKA-3904: -- [~h...@pinterest.com] This is a great find, I think we have

[jira] [Updated] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3904: - Labels: architecture newbie (was: api newbie) > File descriptor leaking (Too many open files)

Re: KTable.filter usage, memory consumption and materialized view semantics

My concern is that, the overhead of requesting the source KTable to be materialized (i.e. creating a state store, and sending the {old -> new} pair instead of the new value only) may be over-whelming compared with its potential benefits of reducing the downstream traffic. Guozhang On Sun, Jun

[jira] [Created] (KAFKA-3907) Better support for user-specific committing in the Streams DSL

Guozhang Wang created KAFKA-3907: Summary: Better support for user-specific committing in the Streams DSL Key: KAFKA-3907 URL: https://issues.apache.org/jira/browse/KAFKA-3907 Project: Kafka

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Rajini, Thanks for the update. Looks good to me. My only comment is that instead of /config/users//clients, would it be better to represent it as /config/users//clients/ so that it's more consistent? Jun On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: >

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351785#comment-15351785 ] ASF GitHub Bot commented on KAFKA-3905: --- GitHub user rekhajoshm opened a pull request:

[GitHub] kafka pull request #1561: KAFKA-3905:remove null from subscribed topics in K...

GitHub user rekhajoshm opened a pull request: https://github.com/apache/kafka/pull/1561 KAFKA-3905:remove null from subscribed topics in KafkaConsumer#subscribe KAFKA-3905:remove null from subscribed topics in KafkaConsumer#subscribe You can merge this pull request into a Git

Jenkins build is back to normal : kafka-trunk-jdk8 #722

See

Re: [jira] [Commented] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

Never mind sorry. I should research this on my own. On 27 Jun 2016 3:29 p.m., "Philippe Derome" wrote: > Guozhang, all on this thread, > > I've been spending a fair amount looking at this problem very indirectly > with my PR for KAFKA-3902. I had 3 commits on the PR and the

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Hey Vahid, Comments below: I'm not very clear on the first part of this paragraph. You could clarify > it for me, but in general balancing out the partitions across consumers in > a group as much as possible would normally mean balancing the load within > the cluster, and that's something a user

Re: [jira] [Commented] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

Guozhang, all on this thread, I've been spending a fair amount looking at this problem very indirectly with my PR for KAFKA-3902. I had 3 commits on the PR and the last one fails on Jenkins for this unit test here: https://github.com/apache/kafka/pull/1556 Additionally, that particular file

[jira] [Commented] (KAFKA-3906) Connect logical types do not support nulls.

[ https://issues.apache.org/jira/browse/KAFKA-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351622#comment-15351622 ] ASF GitHub Bot commented on KAFKA-3906: --- GitHub user jcustenborder opened a pull request:

[GitHub] kafka pull request #1560: KAFKA-3906 - Connect logical types do not support ...

GitHub user jcustenborder opened a pull request: https://github.com/apache/kafka/pull/1560 KAFKA-3906 - Connect logical types do not support nulls. Initial commit with failing unit tests for proposed functionality. You can merge this pull request into a Git repository by running:

[jira] [Created] (KAFKA-3906) Connect logical types do not support nulls.

Jeremy Custenborder created KAFKA-3906: -- Summary: Connect logical types do not support nulls. Key: KAFKA-3906 URL: https://issues.apache.org/jira/browse/KAFKA-3906 Project: Kafka Issue

[jira] [Resolved] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

[ https://issues.apache.org/jira/browse/KAFKA-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3896. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request 1549

[jira] [Commented] (KAFKA-3896) Unstable test KStreamRepartitionJoinTest.shouldCorrectlyRepartitionOnJoinOperations

[ https://issues.apache.org/jira/browse/KAFKA-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351603#comment-15351603 ] ASF GitHub Bot commented on KAFKA-3896: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1549: KAFKA-3896: Unstable test KStreamRepartitionJoinTe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1549 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3822) Kafka Consumer close() hangs indefinitely if Kafka Broker shutdown while connected

[ https://issues.apache.org/jira/browse/KAFKA-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351574#comment-15351574 ] Ashish K Singh commented on KAFKA-3822: --- [~hachikuji] yeah makes sense. Will have a KIP up for

Re: [DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

Hello Ismael, That is a fair suggestion. As of now, I do not see how Authorizer will be used by clients and as such I am happy to adopt your suggested change. Given that this KIP already went through multiple rounds of discussions and votes, do we want to do another round of vote for this change,

[jira] [Assigned] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-3905: -- Assignee: Rekha Joshi > remove null from subscribed topics in KafkaConsumer#subscribe >

[jira] [Assigned] (KAFKA-3844) Sort configuration items in log

[ https://issues.apache.org/jira/browse/KAFKA-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-3844: -- Assignee: Rekha Joshi > Sort configuration items in log > --- > >

[jira] [Commented] (KAFKA-3844) Sort configuration items in log

[ https://issues.apache.org/jira/browse/KAFKA-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351494#comment-15351494 ] ASF GitHub Bot commented on KAFKA-3844: --- GitHub user rekhajoshm opened a pull request:

[GitHub] kafka pull request #1559: KAFKA-3844; Sort configuration items in log

GitHub user rekhajoshm opened a pull request: https://github.com/apache/kafka/pull/1559 KAFKA-3844; Sort configuration items in log KAFKA-3844; Sort configuration items in log You can merge this pull request into a Git repository by running: $ git pull

Re: delay of producer and consumer in kafka 0.9 is too big to be accepted

can someone please explain why latency is so big for me？thanks > 在 2016年6月25日，下午11:16，Jay Kreps 写道： > > Can you sanity check this with the end-to-end latency test that ships with > Kafka in the tools package? > >

[GitHub] kafka pull request #1558: org.apache.kafka.streams.errors.StreamsException: ...

GitHub user stathmarxis opened a pull request: https://github.com/apache/kafka/pull/1558 org.apache.kafka.streams.errors.StreamsException: Failed to rebalance I am running the below simple example in kafka streams and i got a weird exception which i cannot handle Properties

[jira] [Commented] (KAFKA-3900) High CPU util on broker

[ https://issues.apache.org/jira/browse/KAFKA-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350909#comment-15350909 ] Andrey Konyaev commented on KAFKA-3900: --- Any comments? > High CPU util on broker >

[jira] [Commented] (KAFKA-1889) Refactor shell wrapper scripts

[ https://issues.apache.org/jira/browse/KAFKA-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350715#comment-15350715 ] Carlo Alberto Ferraris commented on KAFKA-1889: --- An additional note about {{-loggc}}: GC

[jira] [Created] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

Xing Huang created KAFKA-3905: - Summary: remove null from subscribed topics in KafkaConsumer#subscribe Key: KAFKA-3905 URL: https://issues.apache.org/jira/browse/KAFKA-3905 Project: Kafka

[GitHub] kafka pull request #1557: Improved warning to be clear on first read.

GitHub user LuboVarga opened a pull request: https://github.com/apache/kafka/pull/1557 Improved warning to be clear on first read. I hope it will make searching for reason it is actually shown a bit cleaner. Changed just text. Also fixed typo (server parameter is not