Build failed in Jenkins: kafka-trunk-jdk8 #2207

See Changes: [ismael] MINOR: Use Scala Future in CoreUtils test -- [...truncated 381.89 KB...] kafka.server.epoch.LeaderEpochFileCacheTest >

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Features for 2.0.0 will be known after 1.1.0 is released in February 2018. We are still doing the usual time-based release process[1]. I am raising this well ahead of time because of the potential impact of removing the old Scala clients (particularly the old high-level consumer) and dropping

[GitHub] kafka pull request #4142: MINOR: Use Scala Future in CoreUtils test

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4142 ---

[GitHub] kafka pull request #4201: MINOR: Exclude PULL_REQUEST_TEMPLATE.md from rat c...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4201 ---

Build failed in Jenkins: kafka-trunk-jdk9 #190

See Changes: [ismael] MINOR: Use Scala Future in CoreUtils test -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-4

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Hi Paolo, The KIP looks good, I have a couple of comments: 1. partitionsAndOffsets could perhaps be `recordsToDelete`. 2. It seems a bit inconsistent that the argument is `RecordsToDelete`, but the result is just a `Long`. Should the result be `DeleteRecords` or something like that? It could

[GitHub] kafka pull request #551: KAFKA-2858; Introduce `SimplePrincipal` and use it ...

Github user ijuma closed the pull request at: https://github.com/apache/kafka/pull/551 ---

Build failed in Jenkins: kafka-trunk-jdk8 #2206

See Changes: [ismael] MINOR: Add pull request template -- [...truncated 383.91 KB...] kafka.security.auth.ResourceTypeTest > testJavaConversions PASSED

Build failed in Jenkins: kafka-trunk-jdk7 #2967

See Changes: [ismael] MINOR: Use Scala Future in CoreUtils test -- [...truncated 1.84 MB...] org.apache.kafka.streams.KafkaStreamsTest > testStateGlobalThreadClose PASSED

[GitHub] kafka pull request #4202: MINOR: Exclude Committer Checklist section from co...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/4202 MINOR: Exclude Committer Checklist section from commit message ### Committer Checklist - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

+1 (binding) - thanks On Tue, 7 Nov 2017 at 15:14 Paolo Patierno wrote: > Because Guozhang and Colin are doing a great job on reviewing the related > PR and we are really close to have it in a decent/final shape, what do the > other committers think about this KIP ? > > We

0.11 git tags

I just noticed that although every other release is tagged in git there are no tags for 0.11.0.0 and 0.11.0.1. Did something go wrong? Please can they be tagged retrospectively? Cheers, Tom

Re: 0.11 git tags

Not when I run `git tag` locally though. On 10 November 2017 at 12:44, Ismael Juma wrote: > They seem to be there: > > https://github.com/apache/kafka/tree/0.11.0.0 > https://github.com/apache/kafka/tree/0.11.0.1 > > Ismael > > On Fri, Nov 10, 2017 at 12:41 PM, Tom Bentley

[GitHub] kafka pull request #4174: MINOR: Add pull request template

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4174 ---

Re: 0.11 git tags

They seem to be there: https://github.com/apache/kafka/tree/0.11.0.0 https://github.com/apache/kafka/tree/0.11.0.1 Ismael On Fri, Nov 10, 2017 at 12:41 PM, Tom Bentley wrote: > I just noticed that although every other release is tagged in git there are > no tags for

Build failed in Jenkins: kafka-trunk-jdk9 #188

See Changes: [wangguoz] HOTFIX: GlobalStateManagerImpl in trunk has renamed the consumer field -- [...truncated 1.41 MB...] kafka.server.epoch.LeaderEpochFileCacheTest >

Build failed in Jenkins: kafka-trunk-jdk8 #2205

See Changes: [wangguoz] HOTFIX: GlobalStateManagerImpl in trunk has renamed the consumer field -- [...truncated 3.32 MB...] kafka.security.auth.ResourceTypeTest >

[GitHub] kafka pull request #4201: MINOR: Exclude PULL_REQUEST_TEMPLATE.md from rat c...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/4201 MINOR: Exclude PULL_REQUEST_TEMPLATE.md from rat checks ### Committer Checklist - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify

[jira] [Created] (KAFKA-6198) kerberos login fails

Ronald van de Kuil created KAFKA-6198: - Summary: kerberos login fails Key: KAFKA-6198 URL: https://issues.apache.org/jira/browse/KAFKA-6198 Project: Kafka Issue Type: Test

[GitHub] kafka pull request #4202: MINOR: Exclude Committer Checklist section from co...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4202 ---

Build failed in Jenkins: kafka-trunk-jdk7 #2966

See Changes: [ismael] MINOR: Add pull request template -- [...truncated 382.50 KB...] kafka.server.epoch.LeaderEpochFileCacheTest >

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Hi Ismael, 1. yes it sounds better. Agree. 2. We are using a wrapper class like RecordsToDelete in order to allow future operations that won't be based only on specifying an offset. At same time with this wrapper class and static methods (i.e. beforeOffset) the API is really clear to

Build failed in Jenkins: kafka-trunk-jdk9 #189

See Changes: [ismael] MINOR: Add pull request template -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H18 (ubuntu

Jenkins build is back to normal : kafka-trunk-jdk7 #2965

See

Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

Hi Randall, I have set aside some time to work on this next week. The fix itself is quite simple, but I've yet to write tests to properly catch this, which turns out to be a bit more complex, as it needs a running restserver which is mocked in the tests I've looked at so far. Should I withdraw

Re: [DISCUSS] 0.11.0.2 bug fix release

Update on 0.11.0.2 release: I moved some of the JIRAs out of this release. If anyone thinks they should be included, please let me know. There are still a couple left (and a 3rd one which is a duplicate). They are not blockers, so I will move them out of 0.11.0.2 if there are no objections.

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

Sounds like a good middle ground to me. What do you think Steven? On Mon, Nov 6, 2017 at 8:18 PM Colin McCabe wrote: > It would definitely be nice to use the jdk8 CompletableFuture. I think > that's a bit of a separate discussion, though, since it has such heavy >

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Hi Ismael, Are there any new features other than the language specific changes that are being planned for 2.0.0? Also, when 2.x gets released, will the 1.x series see continued bug fixes and releases in the community or is the plan to have one single main version that gets continuous updates

Re: 0.11 git tags

That was it, thanks! But I still don't understand how I had tags for 1.0.0, but not 0.11.0.0. On 10 November 2017 at 13:10, Kamal wrote: > Try, `git fetch upstream --tags` > > On Fri, Nov 10, 2017 at 6:32 PM, Mickael Maison > wrote: > >

Jenkins build is back to normal : kafka-trunk-jdk7 #2968

See

Re: 0.11 git tags

Try, `git fetch upstream --tags` On Fri, Nov 10, 2017 at 6:32 PM, Mickael Maison wrote: > Have you tried: > git fetch --tags ? > > On Fri, Nov 10, 2017 at 12:54 PM, Tom Bentley > wrote: > > Not when I run `git tag` locally though. > > > > On 10

[jira] [Resolved] (KAFKA-6198) kerberos login fails

[ https://issues.apache.org/jira/browse/KAFKA-6198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald van de Kuil resolved KAFKA-6198. --- Resolution: Fixed > kerberos login fails > > >

Re: 0.11 git tags

Have you tried: git fetch --tags ? On Fri, Nov 10, 2017 at 12:54 PM, Tom Bentley wrote: > Not when I run `git tag` locally though. > > On 10 November 2017 at 12:44, Ismael Juma wrote: > >> They seem to be there: >> >>

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Paolo, You might return the timestamp if the user did a delete by timestamp for example. But let's maybe hear what others think before we change the KIP. Ismael On Fri, Nov 10, 2017 at 12:48 PM, Paolo Patierno wrote: > Hi Ismael, > > > 1. yes it sounds better. Agree. >

Jenkins build is back to normal : kafka-trunk-jdk9 #191

See

Re: [DISCUSS] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

Collin, Ben, Thanks for the input. I will work out this proposa, so I get an idea on the impact. Do you think it is a good idea to line up the new method names with those of CompletableFuture? Thanks, Steven Op vr 10 nov. 2017 om 12:12 schreef Ben Stopford : > Sounds

[GitHub] kafka-site pull request #108: Add Rajini's keys

Github user asfgit closed the pull request at: https://github.com/apache/kafka-site/pull/108 ---

[jira] [Created] (KAFKA-6200) 00000000000000000015.timeindex: The process cannot access the file because it is being used by another process.

yassine bsf created KAFKA-6200: -- Summary: 0015.timeindex: The process cannot access the file because it is being used by another process. Key: KAFKA-6200 URL:

[GitHub] kafka-site pull request #108: Add Rajini's keys

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka-site/pull/108 Add Rajini's keys You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka-site add-key Alternatively you can review and

[jira] [Created] (KAFKA-6199) Single broker with fast growing heap usage

Robin Tweedie created KAFKA-6199: Summary: Single broker with fast growing heap usage Key: KAFKA-6199 URL: https://issues.apache.org/jira/browse/KAFKA-6199 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #4205: KAFKA-4827: Correctly encode special chars while c...

GitHub user wicknicks opened a pull request: https://github.com/apache/kafka/pull/4205 KAFKA-4827: Correctly encode special chars while creating URI objects Signed-off-by: Arjun Satish You can merge this pull request into a Git repository by running: $ git

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Sounds good to me for returning an object than a Long type. Guozhang On Fri, Nov 10, 2017 at 9:26 AM, Paolo Patierno wrote: > Ismael, > > > yes it makes sense. Taking a look to the other methods in the Admin > Client, there is no use case returning a "simple" type : most

[jira] [Created] (KAFKA-6201) Provide more context in Kafka Connect REST error messages

Randall Hauch created KAFKA-6201: Summary: Provide more context in Kafka Connect REST error messages Key: KAFKA-6201 URL: https://issues.apache.org/jira/browse/KAFKA-6201 Project: Kafka

[GitHub] kafka pull request #4203: MINOR: Update version in docs for 0.11.0.2 release

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/4203 MINOR: Update version in docs for 0.11.0.2 release You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka

[GitHub] kafka pull request #4204: KAFKA-5238: BrokerTopicMetrics can be recreated af...

GitHub user edoardocomar opened a pull request: https://github.com/apache/kafka/pull/4204 KAFKA-5238: BrokerTopicMetrics can be recreated after topic is deleted Avoiding a DelayedFetch recreate the metrics when a topic has been deleted developed with @mimaison

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Hi Gwen, A KIP has been proposed, but it is stalled: https://cwiki.apache.org/confluence/display/KAFKA/KIP-125%3A+ ZookeeperConsumerConnector+to+KafkaConsumer+Migration+and+Rollback Unless the interested parties pick that up, we would drop support without a rolling upgrade path. Users would be

Build failed in Jenkins: kafka-trunk-jdk8 #2208

See Changes: [ismael] MINOR: Exclude PULL_REQUEST_TEMPLATE.md from rat checks [ismael] MINOR: Exclude Committer Checklist section from commit message -- [...truncated

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Thanks Ismael, the proposal looks good to me. A side note regarding: https://issues.apache.org/jira/browse/KAFKA-5637, could we resolve this ticket sooner than later to make clear about the code deprecation and support duration when moving from 1.0.x to 2.0.x? Guozhang On Fri, Nov 10, 2017 at

Re: Add to contributors list

Hi, Arjun, Thanks for your interest. Added you to the contributor list. Jun On Fri, Nov 10, 2017 at 10:33 AM, Arjun Satish wrote: > Hi, > > Could you please add me to the contributors list for Apache Kafka? > > My Apache username is wicknicks, and the profile page is

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Ismael, yes it makes sense. Taking a look to the other methods in the Admin Client, there is no use case returning a "simple" type : most of them are Void or complex result represented by classes. In order to support future extensibility I like your idea. Let's see what's the others opinions

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Last time we tried deprecating the Scala consumer, there were concerns about a lack of upgrade path. There is no rolling upgrade, and migrating offsets is not trivial (and not documented). Did anything change in that regard? Or are we planning on dropping support without an upgrade path? On

[jira] [Resolved] (KAFKA-6175) AbstractIndex should cache index file to avoid unnecessary disk access during resize()

[ https://issues.apache.org/jira/browse/KAFKA-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-6175. Resolution: Fixed > AbstractIndex should cache index file to avoid unnecessary disk access during >

Add to contributors list

Hi, Could you please add me to the contributors list for Apache Kafka? My Apache username is wicknicks, and the profile page is located here ( https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wicknicks). Thanks very much, Arjun

[jira] [Created] (KAFKA-6202) Classes OffsetsMessageFormatter and GroupMetadataMessageFormatter shall be used by kafka tools, but in the last releases lost visibility

Seweryn Habdank-Wojewodzki created KAFKA-6202: - Summary: Classes OffsetsMessageFormatter and GroupMetadataMessageFormatter shall be used by kafka tools, but in the last releases lost visibility Key: KAFKA-6202

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

+1 for returning a named object rather than Long. C. On Fri, Nov 10, 2017, at 10:07, Guozhang Wang wrote: > Sounds good to me for returning an object than a Long type. > > > Guozhang > > On Fri, Nov 10, 2017 at 9:26 AM, Paolo Patierno > wrote: > > > Ismael, > > > > > > yes

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Re: migrating offsets for old Scala consumers. I work in the python world, so haven't directly used the old high level consumer, but from what I understand the underlying problem remains the migration of zookeeper offsets to the __consumer_offsets topic. We've used a slightly modified version of

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

I'm leaning towards option 3, although option 2 is a reasonable tradeoff between the two. Overall I'm leaning towards option 3 because: 1. As Guozhang has said we are failing "fast enough" with an Exception from the first rebalance. 2. Less complexity/maintenance cost by not having a

[GitHub] kafka pull request #4206: KAFKA-6122: Global Consumer should handle TimeoutE...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4206 KAFKA-6122: Global Consumer should handle TimeoutException Implements KIP-224: - adding new StreamsConfig `retires` - uses `retires` and `retry.backoff.ms` to handle TimeoutException in

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

Thanks for the feedback. Typos fixed. Damian explained already why we need the new strategy. @Kamal: many users don't want to retry but want to fail the Kafka Stream instance in case of an error. All default parameters are chosen to follow this pattern (similar to consumer/producer/broker

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

bq. it would be worth to reuse both parameters for those I agree. On Fri, Nov 10, 2017 at 1:14 PM, Matthias J. Sax wrote: > Thanks for the feedback. Typos fixed. > > Damian explained already why we need the new strategy. > > @Kamal: many users don't want to retry but

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

One more change: as parameter is called "retries", default value should be zero (instead of one). -Matthias On 11/10/17 1:14 PM, Matthias J. Sax wrote: > Thanks for the feedback. Typos fixed. > > Damian explained already why we need the new strategy. > > @Kamal: many users don't want to retry

Build failed in Jenkins: kafka-0.11.0-jdk7 #334

See Changes: [rajinisivaram] MINOR: Remove note in docs that is no longer required -- [...truncated 977.69 KB...] kafka.utils.UtilsTest > testReadInt PASSED

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Hey everyone. Regarding the status of KIP-125, just a heads up: I have an implementation of KIP-125 (KAFKA-4513) here: https://github.com/onurkaraman/kafka/commit/3b5448006ab70ba2b0b5e177853d191d0f777452 The code might need to be rebased. The steps described in the KIP are a bit involved. Other

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

Overall I'd agree with re-using the parameter for state lock retries. Would there ever be a case where you'd need to have them be different values? On Fri, Nov 10, 2017 at 4:17 PM, Ted Yu wrote: > bq. it would be worth to reuse both parameters for those > > I agree. > > On

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

I was thinking about the source stream/table idea once more and it seems it would not be too hard to implement: We add two new classes SourceKStream extends KStream and SourceKTable extend KTable and return both from StreamsBuilder#stream and StreamsBuilder#table As both are sub-classes,

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

I just review the PR, and there is one thing we should discuss. There are different types of exceptions that could occur. Some apply to all records (like Authorization exception) while others are for single records only (like record too large). For the first category, it seems to not make sense

Re: [DISCUSS] KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods

I updated this KIP as discarded. On 10/13/17 10:34 AM, Matthias J. Sax wrote: > Hi, > > with KIP-182 being implemented in 1.0 that will be released shortly, I > wanted to follow up on this KIP. It seems, that we don't need it any > longer, as KIP-182 resolves those issues. > > If you agree,

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

Just catching up on this KIP. One tiny comment: I would prefer to remove the "Always" from the handler implementation names -- it sounds "cleaner" to me without it. -Matthias On 11/5/17 12:57 PM, Matt Farmer wrote: > It is agreed, then. I've updated the pull request. I'm trying to also >

[VOTE] 0.11.0.2 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 0.11.0.2. This is a bug fix release and it includes fixes and improvements from 16 JIRAs, including a few critical bugs. Release notes for the 0.11.0.2 release:

Re: [VOTE] 0.11.0.2 RC0

Resending to include kafka-clients. On Sat, Nov 11, 2017 at 12:37 AM, Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers, > > > This is the first candidate for release of Apache Kafka 0.11.0.2. > > > This is a bug fix release and it includes

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

For state lock retries, I realized that for both global state stores and local state stores we are using hard-coded retries today (different values though: 5 and MAX). Should we only use retries for global state, and use 0 for local state store to fallback to the main loop? Guozhang On Fri,

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

Hi, i think this is the better way. Naming is always tricky Source is kinda taken I had TopicBackedK[Source|Table] in mind but for the user its way better already IMHO Thank you for reconsideration Best Jan On 10.11.2017 22:48, Matthias J. Sax wrote: I was thinking about the source

Re: Experimenting with Kafka and OpenSSL

I ran these same tests with Java 9 runtime today and have updated the blog to include these numbers[1]. I'm pasting the summary for Java 9 here: - Both for producer and consumer, there's a *drastic improvement in the JRE shipped SSLEngine numbers, in almost all metrics, in Java 9 as compared