[jira] [Resolved] (KAFKA-6132) KafkaProducer.initTransactions dead locks
[
https://issues.apache.org/jira/browse/KAFKA-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski resolved KAFKA-6132.
---
Resolution: Duplicate
Fix Version/s: 1.0.0
This is probably a duplicate of KAFKA-6042. Fixed in 1.0.0-rc3
> KafkaProducer.initTransactions dead locks
> -
>
> Key: KAFKA-6132
> URL: https://issues.apache.org/jira/browse/KAFKA-6132
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 0.11.0.0, 0.11.0.1
> Environment: Travis
>Reporter: Piotr Nowojski
>Priority: Critical
> Fix For: 1.0.0
>
>
> I have found some intermittent failures on travis when using Kafka 0.11
> transactions for writing. One of them is a apparent deadlock with the
> following stack trace:
> {code:java}
> "KafkaTestThread[19, false]" #166 prio=5 os_prio=0 tid=0x7f4b64c29800
> nid=0x1260 waiting on condition [0x7f4b10fa4000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x947048a8> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> at
> org.apache.kafka.clients.producer.internals.TransactionalRequestResult.await(TransactionalRequestResult.java:50)
> at
> org.apache.kafka.clients.producer.KafkaProducer.initTransactions(KafkaProducer.java:537)
> {code}
> I was unsuccessful to reproduce it locally, however I think I can semi
> reliably reproduce it on Travis. Scenario includes simultaneous sequence of
> instantiating new producers, calling {{KafkaProducer.initTransactions}},
> closing them interleaved with writing. I have created a stripped down version
> of this scenario as a github project:
> https://github.com/pnowojski/kafka-init-deadlock
> The code for the test scenario is here:
> https://github.com/pnowojski/kafka-init-deadlock/blob/master/src/test/java/pl/nowojski/KafkaInitDeadLockTest.java
> I have defined 30 build profiles that run this test and in case of detecting
> a dead lock (5 minutes period of inactivity), stack trace of all threads is
> being printed out. Example travis run:
> https://travis-ci.org/pnowojski/kafka-init-deadlock/builds/293256284
> as you can see deadlock occurred in 7 out of 30 builds. It seems like in this
> scenario all of them are failing/dead locking in exactly same way.
> I have observed this issue both on 0.11.0.0 and 0.11.0.1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KAFKA-6132) KafkaProducer.initTransactions dead locks
Piotr Nowojski created KAFKA-6132:
-
Summary: KafkaProducer.initTransactions dead locks
Key: KAFKA-6132
URL: https://issues.apache.org/jira/browse/KAFKA-6132
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 0.11.0.0, 0.11.0.1
Environment: Travis
Reporter: Piotr Nowojski
Priority: Critical
I have found some intermittent failures on travis when using Kafka 0.11
transactions for writing. One of them is a apparent deadlock with the following
stack trace:
{code:java}
"KafkaTestThread[19, false]" #166 prio=5 os_prio=0 tid=0x7f4b64c29800
nid=0x1260 waiting on condition [0x7f4b10fa4000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x947048a8> (a
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at
org.apache.kafka.clients.producer.internals.TransactionalRequestResult.await(TransactionalRequestResult.java:50)
at
org.apache.kafka.clients.producer.KafkaProducer.initTransactions(KafkaProducer.java:537)
{code}
I was unsuccessful to reproduce it locally, however I think I can semi reliably
reproduce it on Travis. Scenario includes simultaneous sequence of
instantiating new producers, calling {{KafkaProducer.initTransactions}},
closing them interleaved with writing. I have created a stripped down version
of this scenario as a github project:
https://github.com/pnowojski/kafka-init-deadlock
The code for test scenario is here:
https://github.com/pnowojski/kafka-init-deadlock/blob/master/src/test/java/pl/nowojski/KafkaInitDeadLockTest.java
I have defined 30 build profiles that run this test and in case of detecting a
dead lock (5 minutes period of inactivity) stack trace of all threads is being
printed out. Example travis run:
https://travis-ci.org/pnowojski/kafka-init-deadlock/builds/293256284
as you can see deadlock occurred in 7 out of 30 builds. It seems like in this
scenario all of them are failing/dead locking in exactly same way.
I have observed this issue both on 0.11.0.0 and 0.11.0.1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KAFKA-5589) Bump dependency of Kafka 0.10.x to the latest one
[ https://issues.apache.org/jira/browse/KAFKA-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski resolved KAFKA-5589. --- Resolution: Not A Problem Sorry, created an issue in wrong project. > Bump dependency of Kafka 0.10.x to the latest one > - > > Key: KAFKA-5589 > URL: https://issues.apache.org/jira/browse/KAFKA-5589 > Project: Kafka > Issue Type: Improvement > Components: KafkaConnect >Reporter: Piotr Nowojski > > We are using pretty old Kafka version for 0.10. Besides any bug fixes and > improvements that were made between 0.10.0.1 and 0.10.2.1, it 0.10.2.1 > version is more similar to 0.11.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KAFKA-5589) Bump dependency of Kafka 0.10.x to the latest one
Piotr Nowojski created KAFKA-5589: - Summary: Bump dependency of Kafka 0.10.x to the latest one Key: KAFKA-5589 URL: https://issues.apache.org/jira/browse/KAFKA-5589 Project: Kafka Issue Type: Improvement Components: KafkaConnect Reporter: Piotr Nowojski We are using pretty old Kafka version for 0.10. Besides any bug fixes and improvements that were made between 0.10.0.1 and 0.10.2.1, it 0.10.2.1 version is more similar to 0.11.0. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
