Build failed in Jenkins: kafka-trunk-jdk8 #1093

See Changes: [wangguoz] KAFKA-4486: Don't commit offsets on exception -- [...truncated 8012 lines...] kafka.integration.SaslPlaintextTopicMetadataTest > testIsrAfterBrokerShutDownAndJoinsBack

[jira] [Created] (KAFKA-4520) Kafka broker fails with not so user-friendly error msg when log.dirs is not set

Buchi Reddy B created KAFKA-4520: Summary: Kafka broker fails with not so user-friendly error msg when log.dirs is not set Key: KAFKA-4520 URL: https://issues.apache.org/jira/browse/KAFKA-4520

Build failed in Jenkins: kafka-trunk-jdk8 #1092

See Changes: [jason] KAFKA-4375; Reset interrupt state in a few places where [jason] KAFKA-4272; Add missing 'connect' Windows batch scripts [jason] KAFKA-4000; Collect and record per-topic consumer metrics [wangguoz] KAFKA-4393:

Re: [DISCUSS] KIP-95: Incremental Batch Processing for Kafka Streams

About using offset-topic metadata field: Even with KIP-98, I think this would not work (or maybe in a weird way). If we have the following topology: topic1 -> subTopologyA -> topic2 -> subTopologyB If producer of subTopologyA commits, it will commit its input offsets from topic1. Thus, the

[jira] [Resolved] (KAFKA-4486) Kafka Streams - exception in process still commits offsets

[ https://issues.apache.org/jira/browse/KAFKA-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4486. -- Resolution: Fixed Issue resolved by pull request 2225

[jira] [Commented] (KAFKA-4486) Kafka Streams - exception in process still commits offsets

[ https://issues.apache.org/jira/browse/KAFKA-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736912#comment-15736912 ] ASF GitHub Bot commented on KAFKA-4486: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2225: KAFKA-4486: Don't commit offsets on exception

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2225 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2239: Docs templates

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka/pull/2239 Docs templates - Move Streams documentation out to it's own page - Render docs with Handlebars.js so we quickly set repeated items like doc version numbers You can merge this pull request

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736891#comment-15736891 ] James Cheng commented on KAFKA-4477: [~tcrayford-heroku] told me that he's seen a deadlock bug that

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736810#comment-15736810 ] Vahid Hashemian commented on KAFKA-3853: Sounds good. The KIP needs one more binding vote to pass

Build failed in Jenkins: kafka-trunk-jdk8 #1091

See Changes: [jason] MINOR: Fix typos in KafkaConsumer docs -- [...truncated 27781 lines...] org.apache.kafka.connect.runtime.WorkerSinkTaskTest > testWakeupInCommitSyncCausesRetry PASSED

[jira] [Commented] (KAFKA-4393) Improve invalid/negative TS handling

[ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736779#comment-15736779 ] ASF GitHub Bot commented on KAFKA-4393: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-4393) Improve invalid/negative TS handling

[ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4393: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request #2117: KAFKA-4393: Improve invalid/negative TS handling

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2117 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736775#comment-15736775 ] Ismael Juma commented on KAFKA-3853: Let's try and include it in 0.10.2.0. :) > Report offsets for

[jira] [Updated] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3853: --- Fix Version/s: 0.10.2.0 > Report offsets for empty groups in ConsumerGroupCommand >

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736755#comment-15736755 ] Vahid Hashemian commented on KAFKA-3853: [~jeffwidman] This is currently waiting for

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736740#comment-15736740 ] Jeff Widman commented on KAFKA-3853: Any idea if this will land in 0.10.2? > Report offsets for empty

[jira] [Updated] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4497: --- Priority: Critical (was: Major) > log cleaner breaks on timeindex > ---

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736646#comment-15736646 ] Ismael Juma commented on KAFKA-4497: There's no way to disable the time index. [~becket_qin], any

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736628#comment-15736628 ] Robert Schumann commented on KAFKA-4497: Catching the right exception and continuing is a valid

[jira] [Updated] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4000: --- Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736627#comment-15736627 ] ASF GitHub Bot commented on KAFKA-4000: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1684: KAFKA-4000: Collect and record per-topic consumer ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1684 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Onur, I understand your question now. So it is indeed possible that after commitTxn() returned the messages could still be lost permanently if all replicas failed before the data was flushed to disk. This is the virtue of Kafka's design to reply on replication (probably in memory) for high

Re: [DISCUSS] Deprecating the old consumers in trunk

Looks good to me too. On Fri, Dec 9, 2016 at 1:02 PM, Jason Gustafson wrote: > Hey Ismael, that sounds fair to me. I'm +1. > > -Jason > > On Thu, Dec 8, 2016 at 8:01 AM, Ismael Juma wrote: > > > Thanks Onur and Jason. I filed a JIRA to track this: > > > >

Re: [DISCUSS] KIP-81: Max in-flight fetches

Hi Mickael, I think the approach looks good, just a few minor questions: 1. The KIP doesn't say what the default value of `buffer.memory` will be. Looks like we use 50MB as the default for `fetch.max.bytes`, so perhaps it makes sense to set the default based on that. Might also be worth

[GitHub] kafka pull request #2238: MINOR: Sync up 'kafka-run-class.bat' with 'kafka-r...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/2238 MINOR: Sync up 'kafka-run-class.bat' with 'kafka-run-class.sh' Some of the recent changes to `kafka-run-class.sh` have not been applied to `kafka-run-class.bat`. These recent changes

[jira] [Assigned] (KAFKA-3209) Support single message transforms in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3209: -- Assignee: Shikhar Bhushan > Support single message transforms in Kafka Connect >

Re: [DISCUSS] Deprecating the old consumers in trunk

Hey Ismael, that sounds fair to me. I'm +1. -Jason On Thu, Dec 8, 2016 at 8:01 AM, Ismael Juma wrote: > Thanks Onur and Jason. I filed a JIRA to track this: > > https://issues.apache.org/jira/browse/KAFKA-4513 > > My take is that this would be good to have and one could

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

In other words, we can see inconsistency when the transaction log reports the transaction as COMMITTED while the markers and data corresponding to the transaction itself on the user partitions may have been partially lost after-the-fact because of kafka's durability guarantees. On Fri, Dec 9,

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

@Guozhang no I actually meant durability concerns over COMMIT/ABORT markers (and a subset of the user's data produced in the transaction for that matter) getting lost from the delta between the write and flush. KIP-98 relies on replicas writing to logs, so transaction durability is effectively

[GitHub] kafka pull request #2237: support scala 2.12 build

GitHub user pjfanning opened a pull request: https://github.com/apache/kafka/pull/2237 support scala 2.12 build I have a scenario where we have a 0.8 Kafka Broker managed by a different org but I would like to upgrade my application to Scala 2.12. I know the 0.8 branch is no

Jenkins build is back to normal : kafka-trunk-jdk7 #1742

See

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Apologies on the spelling. *Hi Jay, From: Michael Pearce Sent: Friday, December 9, 2016 7:52:25 PM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging Hi Jey 1) I agree, these

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Jey 1) I agree, these should be used to add this in a future kip if ever was enough of a case. As stated for us I think for these systems we will keep our JMS solutions there. I think maybe in the docs when this feature is written up, one should redirect users to alternative options such

[jira] [Commented] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736139#comment-15736139 ] Ismael Juma commented on KAFKA-4517: Since we didn't remove it in 0.10.0.0, we now have to wait until

[jira] [Created] (KAFKA-4519) Delete old unused branches in git repo

Jeff Widman created KAFKA-4519: -- Summary: Delete old unused branches in git repo Key: KAFKA-4519 URL: https://issues.apache.org/jira/browse/KAFKA-4519 Project: Kafka Issue Type: Task

[jira] [Commented] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736132#comment-15736132 ] ASF GitHub Bot commented on KAFKA-4517: --- GitHub user jeffwidman opened a pull request:

[GitHub] kafka pull request #2236: KAFKA-4517: Remove deprecated shell script

GitHub user jeffwidman opened a pull request: https://github.com/apache/kafka/pull/2236 KAFKA-4517: Remove deprecated shell script Remove the kafka-consumer-offset-checker.sh script completely since it was already deprecated in Kafka9 Currently it's quite confusing to

[jira] [Commented] (KAFKA-4272) Kafka Connect batch scripts are missing under `bin/windows/`

[ https://issues.apache.org/jira/browse/KAFKA-4272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736123#comment-15736123 ] ASF GitHub Bot commented on KAFKA-4272: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2146: KAFKA-4272: Add missing 'connect' Windows batch sc...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2146 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-4272) Kafka Connect batch scripts are missing under `bin/windows/`

[ https://issues.apache.org/jira/browse/KAFKA-4272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4272: --- Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch

[jira] [Created] (KAFKA-4518) Kafka-connect Not starting after DB restarts

Akshath Patkar created KAFKA-4518: - Summary: Kafka-connect Not starting after DB restarts Key: KAFKA-4518 URL: https://issues.apache.org/jira/browse/KAFKA-4518 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4517: --- Summary: Remove kafka-consumer-offset-checker.sh script since already deprecated in Kafka 9 (was:

[jira] [Updated] (KAFKA-4517) Remove kafka-consumer-offset-checker.sh script since deprecated in Kafka 9

[ https://issues.apache.org/jira/browse/KAFKA-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4517: --- Summary: Remove kafka-consumer-offset-checker.sh script since deprecated in Kafka 9 (was: Remove

[jira] [Commented] (KAFKA-4375) Kafka consumer may swallow some interrupts meant for the calling thread

[ https://issues.apache.org/jira/browse/KAFKA-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736108#comment-15736108 ] ASF GitHub Bot commented on KAFKA-4375: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2100: KAFKA-4375: Reset thread interrupted state in a fe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-4375) Kafka consumer may swallow some interrupts meant for the calling thread

[ https://issues.apache.org/jira/browse/KAFKA-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4375: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull

[jira] [Created] (KAFKA-4517) Remove shell scripts deprecated in Kafka 9

Jeff Widman created KAFKA-4517: -- Summary: Remove shell scripts deprecated in Kafka 9 Key: KAFKA-4517 URL: https://issues.apache.org/jira/browse/KAFKA-4517 Project: Kafka Issue Type: Task

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hey Michael, Yeah, I don't think you need to go into the details of whatever you guys have. I think several people in the thread said "let's do XA transactions too!" Obviously in a world where features were free and always worked perfectly we would! I've probably talked to about 100 people about

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

@Onur I think you are asking about this bullet point right? https://docs.google.com/document/d/11Jqy_GjUGtdXJK94XGsEIK7CP1SnQGdp2eF0wSw9ra8/edit#bookmark=id.ia8lrje8xifh To me "durability" means that once a transaction has been acknowledged committed, it will never be lost. In this sense the

Re: Reg: ACLS

Ismael, thanks for the correction. I assumed the question was targeted for without any security enabled, but yea even then IP based auth is possible. On Fri, Dec 9, 2016 at 11:01 AM, Ismael Juma wrote: > It is possible to use ACLs with IPs or other SASL mechanisms (PLAIN for

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

With respect to the naming, I think it makes sense for two reasons. 1. The begin/commit/rollback semantics are what people expect from transactions, i.e. it passes the intuitive "gut check" meaning people have. If you make up a new name it will likely not convey the expectation. 2.

Re: Reg: ACLS

It is possible to use ACLs with IPs or other SASL mechanisms (PLAIN for example). So Kerberos and SSL are not required (although commonly used). Ismael On Fri, Dec 9, 2016 at 6:59 PM, Ashish Singh wrote: > Hey, > > No it does not. Without kerberos or ssl, all requests will

Re: Reg: ACLS

Hey, No it does not. Without kerberos or ssl, all requests will appear to come from anonymous user, and as long as a user is not identified it is not possible to do authorization on. On Fri, Dec 9, 2016 at 10:40 AM, BigData dev wrote: > Hi All, > I have a question

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

I had a similar comment to Andrew's in terms of the system's safety. He may have stated this scenario in a slightly different way in his first failure scenario. Specifically I'm wondering if the transaction can appear complete and then later go into a bad state. A transaction can involve messages

Re: [DISCUSS] KIP-95: Incremental Batch Processing for Kafka Streams

I will read through the KIP doc once again to provide more detailed feedbacks, but let me through my two cents just for the above email. There are a few motivations to have a "consistent" stop-point across tasks belonging to different sub-topologies. One of them is for interactive queries: say

Reg: ACLS

Hi All, I have a question here, Does Kafka support ACL's with out kerberos/SSL? Any info on this would be greatly helpful. Thanks

[VOTE] KIP-100 - Relax Type constraints in Kafka Streams API

Hi everyone, I would like to start the vote for KIP-100 unless there are any more comments. https://cwiki.apache.org/confluence/display/KAFKA/KIP-100+-+Relax+Type+constraints+in+Kafka+Streams+API corresponding PR here https://github.com/apache/kafka/pull/2205 Thanks, Xavier

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Jay, I can't go too deep into exact implantation due to no NDA. So apologies here. Essentially we have multiple processes each owning selection of accounts so on general flows an action for an account just needs to be managed local to the owning node, happy days ever change is handled as a

[GitHub] kafka-site issue #36: Streams standalone docs

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/36 Thanks @derrickdoo ! Please add me to reviewers list when you created the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] kafka-site issue #36: Streams standalone docs

Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/36 @guozhangwang yes, 2 things need to happen to new copies of stuff in side the versioned docs folder. 1. streams.html will need to have the full shell around it, so the includes at

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hey Michael, Doesn't that example have more to do with applying the update against two rows in a single transaction? That is, clearly the write to Kafka needs to be "transactional" and the write to the destination needs to be transactional, but it's not clear to me that you need isolation that

[GitHub] kafka pull request #2178: Minor: Fix typos in KafkaConsumer docs

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2178 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-4516) When a CachingStateStore is closed it should clear its associated NamedCache. Subsequent queries should throw InvalidStateStoreException

[ https://issues.apache.org/jira/browse/KAFKA-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4516: -- Status: Patch Available (was: Open) > When a CachingStateStore is closed it should clear its

[jira] [Comment Edited] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735863#comment-15735863 ] Tom DeVoe edited comment on KAFKA-4477 at 12/9/16 5:39 PM: --- File limit is set to

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735863#comment-15735863 ] Tom DeVoe commented on KAFKA-4477: -- File limit is set to {{Max open files1048576

[jira] [Commented] (KAFKA-4516) When a CachingStateStore is closed it should clear its associated NamedCache. Subsequent queries should throw InvalidStateStoreException

[ https://issues.apache.org/jira/browse/KAFKA-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735855#comment-15735855 ] ASF GitHub Bot commented on KAFKA-4516: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #2235: KAFKA-4516: When a CachingStateStore is closed it ...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2235 KAFKA-4516: When a CachingStateStore is closed it should clear its associated NamedCache Clear and remove the NamedCache from the ThreadCache when a CachingKeyValueStore or CachingWindowStore is

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735852#comment-15735852 ] Jason Gustafson commented on KAFKA-4477: One other thing to check is whether we're hitting the

[jira] [Updated] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom DeVoe updated KAFKA-4477: - Attachment: issue_node_1001.log issue_node_1003.log issue_node_1002.log I

[jira] [Resolved] (KAFKA-4492) java.lang.IllegalStateException: Attempting to put a clean entry for key... into NamedCache

[ https://issues.apache.org/jira/browse/KAFKA-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4492. --- Resolution: Fixed > java.lang.IllegalStateException: Attempting to put a clean entry for key... >

[jira] [Created] (KAFKA-4516) When a CachingStateStore is closed it should clear its associated NamedCache. Subsequent queries should throw InvalidStateStoreException

Damian Guy created KAFKA-4516: - Summary: When a CachingStateStore is closed it should clear its associated NamedCache. Subsequent queries should throw InvalidStateStoreException Key: KAFKA-4516 URL:

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Should the word transaction therefor be avoided here and another way to describe the feature found. My worry is that by naming this transaction there instantly becomes a tie and expectation to traditional transaction properties. From: Edoardo Comar

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Hi Michael, I don't think KIP-98 should be considered a half-way house. It would provide transaction with the level of durability that Kafka provides. Which is as Andrew suggests, different to other transactional systems. BTW, I am convinced that Andrew has answered the 3rd of my previous 3

[jira] [Commented] (KAFKA-3071) Kafka Server 0.8.2 ERROR OOME with siz

[ https://issues.apache.org/jira/browse/KAFKA-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735600#comment-15735600 ] Anish Khanzode commented on KAFKA-3071: --- Is this an issue that needs attention. I see my consumer

[jira] [Resolved] (KAFKA-4286) metric reporter may hit NullPointerException during shutdown

[ https://issues.apache.org/jira/browse/KAFKA-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4286. Resolution: Fixed Assignee: Xavier Léauté This was fixed by

[jira] [Updated] (KAFKA-4286) metric reporter may hit NullPointerException during shutdown

[ https://issues.apache.org/jira/browse/KAFKA-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4286: --- Fix Version/s: 0.10.2.0 > metric reporter may hit NullPointerException during shutdown >

[jira] [Commented] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735393#comment-15735393 ] ASF GitHub Bot commented on KAFKA-4431: --- GitHub user rajinisivaram opened a pull request:

[GitHub] kafka pull request #2234: KAFKA-4431: Make consumer heartbeat thread a daemo...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/2234 KAFKA-4431: Make consumer heartbeat thread a daemon thread You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka KAFKA-4431

[jira] [Assigned] (KAFKA-4431) HeartbeatThread should be a daemon thread

[ https://issues.apache.org/jira/browse/KAFKA-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram reassigned KAFKA-4431: - Assignee: Rajini Sivaram > HeartbeatThread should be a daemon thread >

[jira] [Commented] (KAFKA-3355) GetOffsetShell command doesn't work with SASL enabled Kafka

[ https://issues.apache.org/jira/browse/KAFKA-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735361#comment-15735361 ] Mohammed amine GARMES commented on KAFKA-3355: -- Hello [~singhashish], [~gwenshap], I want to

[jira] [Commented] (KAFKA-4515) Async producer send not retrying on TimeoutException: Batch Expired

[ https://issues.apache.org/jira/browse/KAFKA-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735355#comment-15735355 ] Rajini Sivaram commented on KAFKA-4515: --- This is being addressed in KIP-91:

Re: [VOTE] KIP-94: Session Windows

Hi, The vote for KIP-94 has now concluded. The KIP has been accepted with: 3 binding votes (Ewen, Sriram, Guozhang) and 4 non-binding (Michael, Matthias, Eno, Bill). Thanks everyone. Damian On Wed, 7 Dec 2016 at 10:39 Michael Noll wrote: > +1 (non-binding) > > On Wed,

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Agreed, I think it is better to either support transactions properly and fully to the level or expectation normally associated with a transactional system. Or don't provide it at all. Having a half way house can be more dangerous. As I said earlier the issue of message duplication can be

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

Thanks for the update Michael. I just wanted to add that there is one crucial piece of information that i've failed to add (I apologise). To me, the join between 2 Global Tables just produces a view on top of the underlying tables (this is the same as it works for KTables today). So that means

[jira] [Commented] (KAFKA-4463) Setup travis-ci integration for ducktape tests

[ https://issues.apache.org/jira/browse/KAFKA-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735136#comment-15735136 ] Ismael Juma commented on KAFKA-4463: When this is done, we should update the following wiki page

[jira] [Updated] (KAFKA-3715) Higher granularity streams metrics

[ https://issues.apache.org/jira/browse/KAFKA-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3715: Priority: Major (was: Minor) > Higher granularity streams metrics >

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

I've been pondering this question of coordinating other resource managers with Kafka transactions for a while and I'm not convinced it's a good idea. My reservations come down to the guarantees that it would provide in failure scenarios. I don't think KIP-98 gives proper ACID semantics in the

[jira] [Commented] (KAFKA-4505) Cannot get topic lag since kafka upgrade from 0.8.1.0 to 0.10.1.0

[ https://issues.apache.org/jira/browse/KAFKA-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734892#comment-15734892 ] Romaric Parmentier commented on KAFKA-4505: --- I'm not sure, although we are using the new driver

Re: [DISCUSS] KIP-99: Add Global Tables to Kafka Streams

Damian and I briefly chatted offline (thanks, Damian!), and here's the summary of my thoughts and conclusion. TL;DR: Let's skip outer join support for global tables. In more detail: - We agreed that, technically, we can add OUTER JOIN support. However, outer joins only work if certain

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734770#comment-15734770 ] Jan Omar commented on KAFKA-4477: - Nothing interesting there in our case. Looked at 30 minutes before

[jira] [Commented] (KAFKA-4515) Async producer send not retrying on TimeoutException: Batch Expired

[ https://issues.apache.org/jira/browse/KAFKA-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734702#comment-15734702 ] huxi commented on KAFKA-4515: - Did you see lots of "Got error produce response with correlation id {} on

[jira] [Commented] (KAFKA-4497) log cleaner breaks on timeindex

[ https://issues.apache.org/jira/browse/KAFKA-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734657#comment-15734657 ] huxi commented on KAFKA-4497: - A safe but naive solution is to reject this invalid index entry and captures