[
https://issues.apache.org/jira/browse/HDFS-16535?focusedWorklogId=755077&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-755077
]
ASF GitHub Bot logged work on HDFS-16535:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/22 03:44
Start Date: 11/Apr/22 03:44
Worklog Time Spent: 10m
Work Description: stiga-huang opened a new pull request, #4158:
URL: https://github.com/apache/hadoop/pull/4158
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
HDFS-13639 improves the performance of short-circuit shm slot releasing by
reusing the domain socket that the client previously used to send release
request to the DataNode.
This is good when there are only one DataNode locates with the client (truth
in most of the production environment). However, if we launch multiple
DataNodes on a machine (usually for testing, e.g. Impala's end-to-end tests),
the request could be sent to the wrong DataNode. See an example in IMPALA-11234.
We should only reuse the domain socket when it corresponds to the same
socket path.
### How was this patch tested?
I deploy the fix in my Impala minicluster with multiple DataNodes launched
on my machine. Verified that the error logs of slot release failures
disappeared.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
Issue Time Tracking
-------------------
Worklog Id: (was: 755077)
Remaining Estimate: 0h
Time Spent: 10m
> SlotReleaser should reuse the domain socket based on socket paths
> -----------------------------------------------------------------
>
> Key: HDFS-16535
> URL: https://issues.apache.org/jira/browse/HDFS-16535
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.3.1, 3.4.0
> Reporter: Quanlong Huang
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HDFS-13639 improves the performance of short-circuit shm slot releasing by
> reusing the domain socket that the client previously used to send release
> request to the DataNode.
> This is good when there are only one DataNode locates with the client (truth
> in most of the production environment). However, if we launch multiple
> DataNodes on a machine (usually for testing, e.g. Impala's end-to-end tests),
> the request could be sent to the wrong DataNode. See an example in
> IMPALA-11234.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
