[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HADOOP-10940: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.0 Status: Resolved (was: Patch Available) I've committed this to trunk, branch-2 and branch-2.8. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-10940.1.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.trunk.1.patch, HADOOP-10940.trunk.2.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.trunk.2.patch Updated core-default as requested. Thanks. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.1.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.trunk.1.patch, HADOOP-10940.trunk.2.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.trunk.1.patch The former HADOOP-10940.1.patch applies to branch-2. HADOOP-10940.trunk.1.patch applies to trunk. The conflict is due to another jira randomly removing leading whitespace on blank lines not in the vicinity of the change. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.1.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.trunk.1.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.1.patch > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.1.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HADOOP-10940: Issue Type: Sub-task (was: Bug) Parent: HADOOP-13425 > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HADOOP-10940: - Target Version/s: (was: 2.8.0) Not much going on here for a long time, dropping from 2.8.0. Not putting any target-version either anymore, let's target this depending on when there is patch activity. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0-alpha1 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.patch Addressed style warnings. Tests pass for me, pre-commit has become so unreliable. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.patch Patch is up to date. Added flush as request by [~cmccabe] and created a distinct config key for the client vs. server. > RPC client does no bounds checking of responses > --- > > Key: HADOOP-10940 > URL: https://issues.apache.org/jira/browse/HADOOP-10940 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Labels: BB2015-05-TBR > Attachments: HADOOP-10940.patch, HADOOP-10940.patch, > HADOOP-10940.patch, HADOOP-10940.patch > > > The rpc client does no bounds checking of server responses. In the case of > communicating with an older and incompatible RPC, this may lead to OOM issues > and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-10940: -- Labels: BB2015-05-TBR (was: ) RPC client does no bounds checking of responses --- Key: HADOOP-10940 URL: https://issues.apache.org/jira/browse/HADOOP-10940 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical Labels: BB2015-05-TBR Attachments: HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch The rpc client does no bounds checking of server responses. In the case of communicating with an older and incompatible RPC, this may lead to OOM issues and leaking of resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.patch Mark max length as final, update comments to clarify how flushing must be handled. RPC client does no bounds checking of responses --- Key: HADOOP-10940 URL: https://issues.apache.org/jira/browse/HADOOP-10940 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical Attachments: HADOOP-10940.patch, HADOOP-10940.patch, HADOOP-10940.patch The rpc client does no bounds checking of server responses. In the case of communicating with an older and incompatible RPC, this may lead to OOM issues and leaking of resources. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Attachment: HADOOP-10940.patch Findbugs already had an synch exclusion for the in/out streams. I changed it to exclude ipcStreams instead. RPC client does no bounds checking of responses --- Key: HADOOP-10940 URL: https://issues.apache.org/jira/browse/HADOOP-10940 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical Attachments: HADOOP-10940.patch, HADOOP-10940.patch The rpc client does no bounds checking of server responses. In the case of communicating with an older and incompatible RPC, this may lead to OOM issues and leaking of resources. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Summary: RPC client does no bounds checking of responses (was: RPC client does poor bounds checking of responses) RPC client does no bounds checking of responses --- Key: HADOOP-10940 URL: https://issues.apache.org/jira/browse/HADOOP-10940 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical The rpc client does no bounds checking of server responses. In the case of communicating with an older and incompatible RPC, this may lead to OOM issues and leaking of resources. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[ https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-10940: - Status: Patch Available (was: Open) RPC client does no bounds checking of responses --- Key: HADOOP-10940 URL: https://issues.apache.org/jira/browse/HADOOP-10940 Project: Hadoop Common Issue Type: Bug Components: ipc Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical Attachments: HADOOP-10940.patch The rpc client does no bounds checking of server responses. In the case of communicating with an older and incompatible RPC, this may lead to OOM issues and leaking of resources. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HADOOP-10940) RPC client does no bounds checking of responses
[
https://issues.apache.org/jira/browse/HADOOP-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-10940:
-
Attachment: HADOOP-10940.patch
The problem causes invalid rpc responses to cause the client to go OOM. This
is killing oozie servers when users try to use a 2.x client to 0.23. The same
applies for 2.x to 1.x.
Added a {{IpcStreams}} object to manage the rpc encoding/decoding. Response
size must be 0 and data data length used by the rpc server. Request
decoding is simpler and more efficient.
If the first response has length -1, then it's assumed to be a pre-rpcv9 error
response. Pre-rpcv9 responses began with the callId, not a length, and the
callId for error was -1.
This patch also fixes flushing issues. Namely the multiple-send before reading
a response. This occurs in two cases:
# insecure: connection header+context+call
# secure: connection header+sasl negotiate
W/o the fix to control flushing, unit tests to verify invalid rpc version
always failed with broken pipe. When the server reads the connection header
for an incompatible client, it sends an error response and immediately closes
the socket. The client may still be in the process of sending multiple
messages as listed above and cause a broken pipe.
I believe the flushing issue may also solve the sporadic unit tests failing
under windows about the remote end closing the connection.
RPC client does no bounds checking of responses
---
Key: HADOOP-10940
URL: https://issues.apache.org/jira/browse/HADOOP-10940
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Attachments: HADOOP-10940.patch
The rpc client does no bounds checking of server responses. In the case of
communicating with an older and incompatible RPC, this may lead to OOM issues
and leaking of resources.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
