[jira] [Updated] (KAFKA-3996) ByteBufferMessageSet.writeTo() should be non-blocking
[ https://issues.apache.org/jira/browse/KAFKA-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3996: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 1669 [https://github.com/apache/kafka/pull/1669] > ByteBufferMessageSet.writeTo() should be non-blocking > - > > Key: KAFKA-3996 > URL: https://issues.apache.org/jira/browse/KAFKA-3996 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.10.0.0 >Reporter: Jun Rao >Assignee: Ismael Juma >Priority: Blocker > Fix For: 0.10.0.1 > > > Currently, in ByteBufferMessageSet.writeTo(), we try to finish writing all > bytes in the buffer in a single call. The code has been like that since 0.8. > This hasn't been a problem historically since the broker uses zero-copy to > send fetch responses and only use ByteBufferMessageSet to send produce > responses, which are small. However, in 0.10.0, if a consumer is before > 0.10.0, the broker has to down convert the message and use > ByteBufferMessageSet to send a fetch response to the consumer. If the client > is slow and there are lots of bytes in the ByteBufferMessageSet, we may not > be able to completely send all the bytes in the buffer for a long period of > time. When this happens, the Processor will be blocked and can't handle other > connections, which is bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3996) ByteBufferMessageSet.writeTo() should be non-blocking
[ https://issues.apache.org/jira/browse/KAFKA-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3996: --- Reviewer: Jun Rao Status: Patch Available (was: In Progress) > ByteBufferMessageSet.writeTo() should be non-blocking > - > > Key: KAFKA-3996 > URL: https://issues.apache.org/jira/browse/KAFKA-3996 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.10.0.0 >Reporter: Jun Rao >Assignee: Ismael Juma >Priority: Blocker > Fix For: 0.10.0.1 > > > Currently, in ByteBufferMessageSet.writeTo(), we try to finish writing all > bytes in the buffer in a single call. The code has been like that since 0.8. > This hasn't been a problem historically since the broker uses zero-copy to > send fetch responses and only use ByteBufferMessageSet to send produce > responses, which are small. However, in 0.10.0, if a consumer is before > 0.10.0, the broker has to down convert the message and use > ByteBufferMessageSet to send a fetch response to the consumer. If the client > is slow and there are lots of bytes in the ByteBufferMessageSet, we may not > be able to completely send all the bytes in the buffer for a long period of > time. When this happens, the Processor will be blocked and can't handle other > connections, which is bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3996) ByteBufferMessageSet.writeTo() should be non-blocking
[ https://issues.apache.org/jira/browse/KAFKA-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3996: --- Priority: Blocker (was: Major) > ByteBufferMessageSet.writeTo() should be non-blocking > - > > Key: KAFKA-3996 > URL: https://issues.apache.org/jira/browse/KAFKA-3996 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.10.0.0 >Reporter: Jun Rao >Assignee: Ismael Juma >Priority: Blocker > Fix For: 0.10.0.1 > > > Currently, in ByteBufferMessageSet.writeTo(), we try to finish writing all > bytes in the buffer in a single call. The code has been like that since 0.8. > This hasn't been a problem historically since the broker uses zero-copy to > send fetch responses and only use ByteBufferMessageSet to send produce > responses, which are small. However, in 0.10.0, if a consumer is before > 0.10.0, the broker has to down convert the message and use > ByteBufferMessageSet to send a fetch response to the consumer. If the client > is slow and there are lots of bytes in the ByteBufferMessageSet, we may not > be able to completely send all the bytes in the buffer for a long period of > time. When this happens, the Processor will be blocked and can't handle other > connections, which is bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3996) ByteBufferMessageSet.writeTo() should be non-blocking
[ https://issues.apache.org/jira/browse/KAFKA-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3996: --- Fix Version/s: 0.10.0.1 > ByteBufferMessageSet.writeTo() should be non-blocking > - > > Key: KAFKA-3996 > URL: https://issues.apache.org/jira/browse/KAFKA-3996 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.10.0.0 >Reporter: Jun Rao >Assignee: Ismael Juma > Fix For: 0.10.0.1 > > > Currently, in ByteBufferMessageSet.writeTo(), we try to finish writing all > bytes in the buffer in a single call. The code has been like that since 0.8. > This hasn't been a problem historically since the broker uses zero-copy to > send fetch responses and only use ByteBufferMessageSet to send produce > responses, which are small. However, in 0.10.0, if a consumer is before > 0.10.0, the broker has to down convert the message and use > ByteBufferMessageSet to send a fetch response to the consumer. If the client > is slow and there are lots of bytes in the ByteBufferMessageSet, we may not > be able to completely send all the bytes in the buffer for a long period of > time. When this happens, the Processor will be blocked and can't handle other > connections, which is bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
