[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15269599#comment-15269599
 ] 

ASF GitHub Bot commented on KAFKA-3383:
---

Github user zhuchen1018 closed the pull request at:

https://github.com/apache/kafka/pull/1050


> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.1.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200965#comment-15200965
 ] 

ASF GitHub Bot commented on KAFKA-3383:
---

GitHub user zhuchen1018 reopened a pull request:

https://github.com/apache/kafka/pull/1050

KAFKA-3383: remove in flight request only after response parsing succeeds

@becketqin, could you take a look at the patch?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhuchen1018/kafka KAFKA-3383

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1050.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1050


commit 84372bbe06bd121e758348b54d2d79d5fb4fd095
Author: Chen Zhu 
Date:   2016-03-11T03:43:52Z

KAFKA-3383: remove in flight request only after response parsing succeeds




> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[Jun Rao (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200493#comment-15200493
 ] 

Jun Rao commented on KAFKA-3383:


[~becket_qin], do you know why the response parsing will fail?

> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[Jun Rao (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201566#comment-15201566
 ] 

Jun Rao commented on KAFKA-3383:


[~becket_qin], if this is due to a bug that the broker sends the wrong 
response, it's more important to fix the cause than the symptom, right?

> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[Jiangjie Qin (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201777#comment-15201777
 ] 

Jiangjie Qin commented on KAFKA-3383:
-

[~junrao] I think the broker side bug has already been fixed in the trunk. 

The reason broker threw quota sensor exception was due to some mistake I made 
when cherry pick KAFKA-3310. Supposedly the broker should return an 
ProduceResponse with error code -1 using the version that the request was 
using. But the broker running at that time does not have KAFKA-2071 yet so it 
is still using the scala Produce Request/Response code, which seems always 
forming an error response with the latest version, in this case it is v1. 
Because the producer was actually sending v0 and don't understand the throttle 
time in the response yet, the producer parse failed.

After KAFKA-2071, I think the bug has been fixed because the broker will return 
the ProduceResponse in the same version as ProduceRequest.

> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[Jiangjie Qin (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200588#comment-15200588
 ] 

Jiangjie Qin commented on KAFKA-3383:
-

[~junrao] It was due to some quota sensor exception. The broker also has some 
issue that it did not give back the error response in correct format. I haven't 
got a chance to dig into that yet. I will create another ticket for that after 
the investigation.

> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200964#comment-15200964
 ] 

ASF GitHub Bot commented on KAFKA-3383:
---

Github user zhuchen1018 closed the pull request at:

https://github.com/apache/kafka/pull/1050


> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
>Priority: Critical
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3383) Producer should not remove an in flight request before successfully parsing the response.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190441#comment-15190441
 ] 

ASF GitHub Bot commented on KAFKA-3383:
---

GitHub user zhuchen1018 opened a pull request:

https://github.com/apache/kafka/pull/1050

KAFKA-3383: remove in flight request only after response parsing succeeds

@becketqin, could you take a look at the patch?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhuchen1018/kafka KAFKA-3383

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1050.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1050


commit 84372bbe06bd121e758348b54d2d79d5fb4fd095
Author: Chen Zhu 
Date:   2016-03-11T03:43:52Z

KAFKA-3383: remove in flight request only after response parsing succeeds




> Producer should not remove an in flight request before successfully parsing 
> the response.
> -
>
> Key: KAFKA-3383
> URL: https://issues.apache.org/jira/browse/KAFKA-3383
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.9.0.0
>Reporter: Jiangjie Qin
>Assignee: chen zhu
> Fix For: 0.10.0.0
>
>
> In the NetworkClient, we remove the in flight request before we successfully 
> parse the response. If the response parse failed, the request will not be 
> fulfilled but just lost. For a producer request, that means the callback of 
> the messages won't be fired forever.
> We should only remove the in flight request after response parsing succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)