[
https://issues.apache.org/jira/browse/SPARK-39353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17545254#comment-17545254
]
Jinpeng Chi commented on SPARK-39353:
-------------------------------------
I enabled short-circuit reading on HDFS deployed on Kubernetes and mounted it
on the physical machine. The Spark Pods (including Executor and Driver) I
submitted to Kubernetes mount this Socket, but when I query, all Task Loading
data on HDFS is very slow, and the UI shows that the Locality Level is Any, so
I want to ask how to open Node Local, instead of going through the network,
directly from the disk
> Cannot fetch hdfs data node local
> ---------------------------------
>
> Key: SPARK-39353
> URL: https://issues.apache.org/jira/browse/SPARK-39353
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 3.2.1
> Environment: HDFS on Kubernetes 3.3.1
> Spark on Kubernetes 3.2.1
> Reporter: Jinpeng Chi
> Priority: Major
>
> When i use hdfs short circuit read, the local level always any
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
