Re: Reg. beginner issues for a new contributor

Just an add on: Also Connect API and Streams API are written in Java. Not just producer/consumer/admin clients. Thus, Java code base is quite big and not just a small part. -Matthias On 12/14/17 7:28 PM, Gwen Shapira wrote: > Just to clarify, since I wrote the quora answer: Apache Kafka core

Kafka 1.0 process of security vulnerabilities

All: We are deploying Kafka 1.0 as microservice. I want to understand the process of security vulnerabilities in the Kafka project. How the vulnerabilities are identified in addition to reported by users. Are any tools used for static and dynamic scan? Can the scan results be shared?

Re: Queries on Kafka Capacity

Adding Users list to this email for help on below queries. please help us. Regards,Ajay chaudhary On Thursday 14 December 2017, 1:07:56 PM IST, ajay chaudhary wrote: Hi Team, This is Ajay working with Yodlee India. We are trying to setup Kafka cluster for

[jira] [Created] (KAFKA-6370) MirrorMakerIntegrationTest#testCommaSeparatedRegex may fail due to NullPointerException

Ted Yu created KAFKA-6370: - Summary: MirrorMakerIntegrationTest#testCommaSeparatedRegex may fail due to NullPointerException Key: KAFKA-6370 URL: https://issues.apache.org/jira/browse/KAFKA-6370 Project:

Build failed in Jenkins: kafka-trunk-jdk8 #2277

See Changes: [jason] KAFKA-6308; Connect Struct should use deepEquals/deepHashCode -- [...truncated 3.38 MB...] kafka.utils.CoreUtilsTest > testReadInt STARTED

Re: Reg. beginner issues for a new contributor

Just to clarify, since I wrote the quora answer: Apache Kafka core code (brokers) is not transitioning to Java. It is written in Scala and I didn't see much interest in rewriting it. The *client API* was replaced over the last 3-4 years and the latest versions are in Java. On Thu, Dec 14, 2017 at

[GitHub] kafka pull request #4326: KAFKA-6362: maybeAutoCommitOffsetsAsync should try...

GitHub user huxihx opened a pull request: https://github.com/apache/kafka/pull/4326 KAFKA-6362: maybeAutoCommitOffsetsAsync should try to discover coordinator Currently, `maybeAutoCommitOffsetsAsync` may not retry to find out coordinator even after the coordinator goes back to

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi, Dong, Thanks for the update. A few more comments below. 10. It seems that we need to return the leader epoch in the fetch response as well When fetching data, we could be fetching data from a leader epoch older than what's returned in the metadata response. So, we want to use the leader

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

Hi all, I think the KIP has progressed a lot and is ready for a vote soon. I'll call a vote tomorrow if there are no more comments. best, Colin On Thu, Dec 14, 2017, at 17:22, Colin McCabe wrote: > On Tue, Dec 12, 2017, at 11:48, Becket Qin wrote: > > Hi Colin, > > > > I am not completely

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

On Tue, Dec 12, 2017, at 11:48, Becket Qin wrote: > Hi Colin, > > I am not completely sure, but I am hoping that when we do > FileChannel.transferTo() the OS will just use a fixed buffer to transfer > the data to the socket channel without polluting the page cache. But this > might not be true if

Jenkins build is back to normal : kafka-trunk-jdk8 #2276

See

Jenkins build is back to normal : kafka-trunk-jdk7 #3038

See

[jira] [Created] (KAFKA-6369) General wildcard support for ACL's in kafka

Antony Stubbs created KAFKA-6369: Summary: General wildcard support for ACL's in kafka Key: KAFKA-6369 URL: https://issues.apache.org/jira/browse/KAFKA-6369 Project: Kafka Issue Type: New

[GitHub] kafka pull request #4293: KAFKA-6308: Connect Struct should use deepEquals/d...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4293 ---

[jira] [Resolved] (KAFKA-6308) Connect: Struct equals/hashCode method should use Arrays#deep* methods

[ https://issues.apache.org/jira/browse/KAFKA-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6308. Resolution: Fixed Fix Version/s: 1.0.1 Issue resolved by pull request 4293

[jira] [Created] (KAFKA-6368) AdminClient should contact multiple nodes before timing out a call

Colin P. McCabe created KAFKA-6368: -- Summary: AdminClient should contact multiple nodes before timing out a call Key: KAFKA-6368 URL: https://issues.apache.org/jira/browse/KAFKA-6368 Project: Kafka

[GitHub] kafka pull request #4105: KAFKA-6102: Consolidate MockTime implementations b...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4105 ---

[jira] [Resolved] (KAFKA-6102) Consolidate MockTime implementations between connect and clients

[ https://issues.apache.org/jira/browse/KAFKA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6102. Resolution: Fixed Fix Version/s: 1.0.1 Issue resolved by pull request 4105

Re: [DISCUSS] KIP-238: Expose Kafka cluster ID in Connect REST API

Thanks, Ewen. I think the KIP is clear enough about the intent and the changed behavior. On Tue, Dec 12, 2017 at 12:22 AM, Ewen Cheslack-Postava wrote: > And to clarify a bit further: the goal is for both standalone and > distributed mode to display the same basic

[jira] [Created] (KAFKA-6367) Fix StateRestoreListener To Use Correct Ending Offset

Bill Bejeck created KAFKA-6367: -- Summary: Fix StateRestoreListener To Use Correct Ending Offset Key: KAFKA-6367 URL: https://issues.apache.org/jira/browse/KAFKA-6367 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk7 #3037

See Changes: [wangguoz] KAFKA-6360: Clear RocksDB Segments when store is closed -- [...truncated 229.46 KB...] kafka.api.SslEndToEndAuthorizationTest >

[jira] [Resolved] (KAFKA-6365) How to add a client to list of available clients?

[ https://issues.apache.org/jira/browse/KAFKA-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6365. Resolution: Resolved > How to add a client to list of available clients? >

[jira] [Created] (KAFKA-6366) StackOverflowError in kafka-coordinator-heartbeat-thread

Joerg Heinicke created KAFKA-6366: - Summary: StackOverflowError in kafka-coordinator-heartbeat-thread Key: KAFKA-6366 URL: https://issues.apache.org/jira/browse/KAFKA-6366 Project: Kafka

[jira] [Created] (KAFKA-6365) How to add a client to list of available clients?

Lev Gorodinski created KAFKA-6365: - Summary: How to add a client to list of available clients? Key: KAFKA-6365 URL: https://issues.apache.org/jira/browse/KAFKA-6365 Project: Kafka Issue

Build failed in Jenkins: kafka-trunk-jdk8 #2275

See Changes: [wangguoz] KAFKA-6360: Clear RocksDB Segments when store is closed -- [...truncated 1.43 MB...] org.apache.kafka.common.config.ConfigDefTest >

[jira] [Created] (KAFKA-6364) Add Second Check for End Offset During Restore

Bill Bejeck created KAFKA-6364: -- Summary: Add Second Check for End Offset During Restore Key: KAFKA-6364 URL: https://issues.apache.org/jira/browse/KAFKA-6364 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-6363) Use MockAdminClient for any unit tests that depend on AdminClient

Guozhang Wang created KAFKA-6363: Summary: Use MockAdminClient for any unit tests that depend on AdminClient Key: KAFKA-6363 URL: https://issues.apache.org/jira/browse/KAFKA-6363 Project: Kafka

[GitHub] kafka pull request #4324: KAFKA-6360: Clear RocksDB Segments when store is c...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4324 ---

[jira] [Resolved] (KAFKA-6360) RocksDB segments not removed when store is closed causes re-initialization to fail

[ https://issues.apache.org/jira/browse/KAFKA-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-6360. -- Resolution: Fixed Issue resolved by pull request 4324

Re: Queries on Kafka Capacity

It seems this should help (coming in 1.1 release): https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories On Wed, Dec 13, 2017 at 11:37 PM, ajay chaudhary < juniora...@yahoo.com.invalid> wrote: > Hi Team, > This is Ajay working with Yodlee

Re: Reg. beginner issues for a new contributor

Have you seen this ? http://search-hadoop.com/m/Kafka/uyzND14PPsT5THhN1?subj=Re+Contributing+to+Kafka On Thu, Dec 14, 2017 at 8:55 AM, Abhinav Koppula wrote: > Hi Team, > I wanted to get started with contributing to Kafka and learn more about its > internals. Can

Reg. beginner issues for a new contributor

Hi Team, I wanted to get started with contributing to Kafka and learn more about its internals. Can anyone please suggest me some "good-first-bugs" which I can look at? Also, I had another basic doubt. I see that Kafka core is written in Scala but I have read from some sources

Re: [DISCUSS] KIP 145 - Expose Record Headers in Kafka Connect

Hi, Michael. Yeah, I liked your PR a lot, and there definitely are a lot of similarities. But here are the more significant differences from my perspective (none of which are really that big): First, your `SubjectConverter` and my `HeaderConverter` are pretty similar -- mine is just more closely

Queries on Kafka Capacity

Hi Team, This is Ajay working with Yodlee India. We are trying to setup Kafka cluster for streaming and we need some clarifications.   Could you please help us to understand this behavior. How do we add a capacity to existing Kafka cluster. Let us assume, we have Kafka cluster with 3 brokers and

[GitHub] kafka pull request #4325: Check null message

GitHub user lisa2lisa opened a pull request: https://github.com/apache/kafka/pull/4325 Check null message When enable the trace level log in mirror maker, the message could contain null value, and this will throw null pointer exception. *Summary of testing strategy

Re: Flaky healthcheck when trying to check Kafka Stream processing app status

Hi Patrice, Sorry for the delay in getting back to you. I cloned your repo and ran the example. Shutting down the broker does not stop the streams app, so streams state remains RUNNING and it still has its state store. Hence the SUCCESS status when running curl -XGET

[jira] [Resolved] (KAFKA-6350) File descriptors leak with persistent KeyValueStore

[ https://issues.apache.org/jira/browse/KAFKA-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alin Gheorghe resolved KAFKA-6350. -- Resolution: Not A Bug > File descriptors leak with persistent KeyValueStore >

[jira] [Created] (KAFKA-6362) auto commit not work since coordinatorUnknown() is always true.

Renkai Ge created KAFKA-6362: Summary: auto commit not work since coordinatorUnknown() is always true. Key: KAFKA-6362 URL: https://issues.apache.org/jira/browse/KAFKA-6362 Project: Kafka Issue

[jira] [Resolved] (KAFKA-5123) Refactor ZkUtils readData* methods

[ https://issues.apache.org/jira/browse/KAFKA-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balint Molnar resolved KAFKA-5123. -- Resolution: Won't Fix > Refactor ZkUtils readData* methods >

[GitHub] kafka pull request #3554: KAFKA-5123 Refactor ZkUtils readData* methods

Github user baluchicken closed the pull request at: https://github.com/apache/kafka/pull/3554 ---

[GitHub] kafka pull request #4324: KAFKA-6360: Clear RocksDB Segments when store is c...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/4324 KAFKA-6360: Clear RocksDB Segments when store is closed Now that we support re-initializing state stores, we need to clear the segments when the store is closed so that they can be re-opened.

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Thanks Ted, now fixed. On 13 December 2017 at 18:38, Ted Yu wrote: > Tom: > bq. create a znode /admin/reassignments/$topic-$partition > > Looks like the tree structure above should be: > > /admin/reassignments/$topic/$partition > > bq. The controller removes

Re: [VOTE] Allowing write access to GitHub repositories (aka GitBox)

+1 On 12 December 2017 at 20:38, Sriram Subramanian wrote: > +1 > > On Tue, Dec 12, 2017 at 8:22 AM, Manikumar > wrote: > > > +1 > > > > On Tue, Dec 12, 2017 at 9:49 PM, Rajini Sivaram > > > wrote: > > > > > +1 > > > > >