[
https://issues.apache.org/jira/browse/HDFS-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782533#comment-17782533
]
Gautham Banasandra edited comment on HDFS-17246 at 11/3/23 12:01 PM:
---------------------------------------------------------------------
Thanks for catching this [~ayushtkn], I apologize for causing the regression.
In MiniYARNCluster.java, a local temp path is used. Please refer to -
#
https://github.com/apache/hadoop/blob/2b1378c99e78e5cc13c03f80acea5a501cbad683/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java#L338-L341
#
https://github.com/apache/hadoop/blob/2b1378c99e78e5cc13c03f80acea5a501cbad683/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java#L190-L223
Hence, while spinning up the MiniYARNCluster, it goes through the
*DFSUtil.isValidName* check.
!image-2023-11-03-17-31-14-990.png!
was (Author: gautham):
Thanks for catching this [~ayushtkn], I apologize for causing the regression.
In MiniYARNCluster.java, a local temp path is used. Please refer to -
#
https://github.com/apache/hadoop/blob/2b1378c99e78e5cc13c03f80acea5a501cbad683/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java#L338-L341
#
https://github.com/apache/hadoop/blob/2b1378c99e78e5cc13c03f80acea5a501cbad683/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java#L190-L223
Hence, while spinning up the MiniYARNCluster, it goes through the
*DFSUtil.isValidName* check.
> Fix shaded client for building Hadoop on Windows
> ------------------------------------------------
>
> Key: HDFS-17246
> URL: https://issues.apache.org/jira/browse/HDFS-17246
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.4.0
> Environment: Windows 10
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2023-11-03-17-31-14-990.png
>
>
> Currently, the *shaded client* Yetus personality in Hadoop fails to build on
> Windows -
> https://github.com/apache/hadoop/blob/4c04a6768c0cb3d5081cfa5d84ffb389d92f5805/dev-support/bin/hadoop.sh#L541-L615.
> This happens due to the integration test failures in Hadoop client modules -
> https://github.com/apache/hadoop/tree/4c04a6768c0cb3d5081cfa5d84ffb389d92f5805/hadoop-client-modules/hadoop-client-integration-tests.
> There are several issues that need to be addressed in order to get the
> integration tests working -
> # Set the HADOOP_HOME, needed by the Mini DFS and YARN clusters spawned by
> the integration tests.
> # Add Hadoop binaries to PATH, so that winutils.exe can be located.
> # Create a new user with Symlink privilege in the Docker image. This is
> needed for the proper working of Mini YARN cluster, spawned by the
> integration tests.
> # Fix a bug in DFSUtilClient.java that prevents colon ( *:* ) in the path.
> The colon is used a delimiter for the PATH variable while specifying multiple
> paths. However, this isn't a delimiter in the case of Windows and must be
> handled appropriately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
