deepakpanda93 commented on code in PR #19747: URL: https://github.com/apache/hudi/pull/19747#discussion_r4013343867
########## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/TestSparkClientFunctionalTestHarness.java: ########## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.hudi.testutils; + +import org.apache.hudi.common.model.HoodieTableType; +import org.apache.hudi.common.table.HoodieTableMetaClient; +import org.apache.hudi.common.testutils.HoodieTestTable; +import org.apache.hudi.storage.StoragePath; + +import org.junit.jupiter.api.Test; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Covers the path contract of {@link SparkClientFunctionalTestHarness} (HUDI-6042). + */ +public class TestSparkClientFunctionalTestHarness extends SparkClientFunctionalTestHarness { + + @Test + public void basePathHasNoSchemeAndBaseUriDoes() { + assertFalse(basePath().startsWith("file:"), + "basePath() must be unqualified so java.nio.file.Paths can consume it, but was " + basePath()); + assertTrue(Paths.get(basePath()).isAbsolute(), + "basePath() must resolve to an absolute path, but was " + basePath()); + assertEquals("file", baseUri().getScheme(), "baseUri() must keep the scheme"); + assertEquals(basePath(), Paths.get(baseUri()).toString(), + "baseUri() and basePath() must point at the same directory"); + } + + @Test + public void nioResolvesPartitionsUnderTheTableDirectory() { + // A scheme-qualified basePath() silently yields a relative path here, placing partitions under + // the working directory rather than the table. This is the failure mode FileCreateUtils hits. + java.nio.file.Path partition = Paths.get(basePath(), "2016/03/15"); + assertTrue(partition.isAbsolute(), "partition path must be absolute, but was " + partition); + assertTrue(partition.startsWith(Paths.get(basePath())), + "partition path must sit under the table directory, but was " + partition); + } + + @Test Review Comment: Applied, thanks. There is no competing `Path` on this file's imports — only `java.nio.file.Files`, `java.nio.file.Paths` and `org.apache.hudi.storage.StoragePath`, which is a distinct name — so the qualification was buying nothing: ```java import java.nio.file.Path; ... Path partition = Paths.get(basePath(), "2016/03/15"); ``` I left the one on the line above alone, since that occurrence is prose inside an assertion message rather than a type reference: ```java "basePath() must be unqualified so java.nio.file.Paths can consume it, but was " + basePath() ``` Also rebased onto latest master in the same push. The branch was 100 commits behind, and one of those commits, `34b4850421eb`, touched `TestSparkSampleWritesUtils`, which is the file this PR deletes the `HUDI-6042` override from. It auto-merged; I checked the result rather than trusting it, and both sides survived: the new parameterised table-version tests are present and the override is gone. Its new `assertSampleWritesShadowTableVersion` helper builds `new Path(basePath(), SAMPLE_WRITES_FOLDER_PATH)`, which the two-argument Hadoop constructor handles with an unqualified path. Local test runs are not available to me on this machine right now — `hadoop-client-api` shadows `hadoop-common`'s `UserGroupInformation` and the shaded `commons-configuration2` is missing from the classpath, so anything touching `HoodieStorage` dies in class init. It is not related to this change: `TestHoodieSparkCopyOnWriteTableArchiveWithReplace`, which this PR does not touch, fails identically. The change itself is compile-verified (the module installs and `checkstyle:check` is clean), and CI covers the rest — it was 23/23 on the previous head and is re-running on the rebased one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
