[
https://issues.apache.org/jira/browse/HDFS-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782602#comment-17782602
]
Ayush Saxena commented on HDFS-17246:
-------------------------------------
[~gaurava] If the path to be referenced in on {{LocalFileSystem}} then these
things will surface and behaviour differences will kick in wrt Unix.
If the path is on HDFS, we can have any path, that need not be on LFS. If you
check HDFS-13296, it also solved a similar problem, if you read the description.
For WebHDFS, it was having a path like
{{"webhdfs://127.0.0.1:18334/D:/target/test/data/vUqZkOrBZa/test"}}, it has
D:/, but this wasn't required & it got fixed by that util.
If you check this
method(https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeAttributeTestUtils.java#L45-L50),
it is used by several other tests & If those tests are passing on Windows,
then it should fix.
Unless that {{getCanonicalPath()}} screws up something, In that case we may try
{code:java}
conf.set(YarnConfiguration.FS_NODE_ATTRIBUTE_STORE_ROOT_DIR,GenericTestUtils.getRandomizedTestDir().getAbsolutePath());
{code}
Let me know if it still fails & the test you ran, if no luck we can go ahead
and just change the DFSUtils as you initially proposed
> Fix DFSUtilClient.ValidName ERROR
> ---------------------------------
>
> Key: HDFS-17246
> URL: https://issues.apache.org/jira/browse/HDFS-17246
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.4.0
> Environment: Windows 10
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2023-11-03-17-31-14-990.png
>
>
> Currently, the *shaded client* Yetus personality in Hadoop fails to build on
> Windows -
> https://github.com/apache/hadoop/blob/4c04a6768c0cb3d5081cfa5d84ffb389d92f5805/dev-support/bin/hadoop.sh#L541-L615.
> This happens due to the integration test failures in Hadoop client modules -
> https://github.com/apache/hadoop/tree/4c04a6768c0cb3d5081cfa5d84ffb389d92f5805/hadoop-client-modules/hadoop-client-integration-tests.
> There are several issues that need to be addressed in order to get the
> integration tests working -
> # Set the HADOOP_HOME, needed by the Mini DFS and YARN clusters spawned by
> the integration tests.
> # Add Hadoop binaries to PATH, so that winutils.exe can be located.
> # Create a new user with Symlink privilege in the Docker image. This is
> needed for the proper working of Mini YARN cluster, spawned by the
> integration tests.
> # Fix a bug in DFSUtilClient.java that prevents colon ( *:* ) in the path.
> The colon is used a delimiter for the PATH variable while specifying multiple
> paths. However, this isn't a delimiter in the case of Windows and must be
> handled appropriately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
