vt921102 commented on PR #8692:
URL: https://github.com/apache/hadoop/pull/8692#issuecomment-6116784601
Thanks @anujmodi2021 — the `branch-3.5` validations are complete.
I ran the full `hadoop-azure` integration suite **on Windows**, which
matters for
this particular change: the defect only reproduces where `File.separator` is
a
backslash, so the earlier trunk run (macOS) could not exercise the failing
platform. This is the first integration validation of the fix on the OS
where the
bug actually occurs.
### Environment
- Windows 11 Pro 24H2, 4 vCPU / 8 GiB
- JDK 17 (Microsoft Build of OpenJDK 17.0.20.1), Maven via the project
`mvnw.cmd`
- Visual Studio 2022 Build Tools, MSVC v143, Windows SDK 10.0.22621
- ADLS Gen2 account in Azure **Sweden Central**, **HNS enabled**,
auth = **OAuth 2.0 client credentials** (`ClientCredsTokenProvider`)
The storage account belongs to my employer, so I have not named it here. I
am happy
to share the endpoint with a committer privately if the project requires the
specific account name on record.
### Commits tested
| Role | Commit |
|---|---|
| Patched | `da0ada979471ced5a1f64c514998f65c206b2276` (this PR) |
| Baseline | `e7d3cb3e3fec8ca8431bf53cfb5edc6cb99febdf` (this PR's parent) |
### Command
```
mvnw.cmd -B -ntp -Duse.platformToolsetVersion=v143 \
-pl hadoop-tools/hadoop-azure -Dparallel-tests=abfs -Dscale verify
```
### Results — patched vs baseline, same VM and same account
| | Patched | Baseline |
|---|---|---|
| Tests run | **1494** | 1489 |
| Failures | **10** | 9 |
| Errors | **22** | 23 |
| Skipped | 474 | 474 |
| Duration | 28:10 | 28:24 |
Both runs exited 0, with 343 ABFS test classes started in each.
**No regression.** The failing sets are identical in both runs except for two
single, retry-flaky entries that cancel each other out:
| Test | Patched | Baseline |
|---|---|---|
| `TestAbfsInputStream.testAdaptiveInputStream` | fails | passes |
| `ITestAzureBlobFileSystemRenameUnicode.testRenameFileUsingUnicode` (retry
5) | passes | fails |
Neither relates to list-status path construction.
### The fix is actually exercised
```
TestAbfsClient: patched 3 tests, 0 failures
baseline 2 tests, 0 failures
```
The additional test is `testGetVersionedFileStatusFromEntryUsesForwardSlash`,
passing on Windows. And the directly relevant integration suite is clean:
```
ITestAzureBlobFileSystemListStatus: Tests run: 20, Failures: 0, Errors: 0,
Skipped: 7
```
### Failures present in both runs (pre-existing / environmental)
| Cause | Tests |
|---|---|
| `There is no primary group for UGI SYSTEM` — tests ran in a service
context without a primary group | `ITestAbfsIdentityTransformer`,
`ITestAzureBlobFileSystemBasics.testWorkingDirectory` |
| 403 `AuthorizationPermissionMismatch` on `setAccessControl` — the service
principal lacks container-root ACL rights |
`ITestAzureBlobFilesystemAcl.testEnsureAclOperationWorksForRoot` |
| OAuth-only account; test expects SharedKey / user-bound SAS to be
configured | `ITestAbfsClient.testAuthTypeProviderSetup` (×3) |
| Pre-existing, unchanged between runs |
`ITestAzureBlobFileSystemDelete.testDeleteIdempotencyTriggerHttp404`,
`ITestAzureBlobFileSystemE2E.testHttpReadTimeout`, `ITestAbfsWrappedIO`
teardown NPE |
| Unit tests (surefire), unchanged between runs, unrelated to ABFS listing |
`TestAggregateMetricsManager` (×3), `TestReadBufferManagerV2` (×2),
`TestShellDecryptionKeyProvider` |
### Separate finding, not related to this PR
Building `hadoop-common` on Windows fails against **Windows SDK 10.0.26100**:
```
winutils.h(129,7): error C2733: 'GetFileInformationByName':
you cannot overload a function with 'extern "C"' linkage
```
Hadoop declares its own `GetFileInformationByName` in
`hadoop-common/src/main/winutils/include/winutils.h`, and a function of the
same
name now exists in the newer SDK. `libwinutils.vcxproj` does not pin
`WindowsTargetPlatformVersion`, so msbuild selects the newest SDK installed.
I
worked around it by pinning SDK 10.0.22621. Happy to raise this as its own
JIRA if
it is not already tracked.
Also worth noting for anyone reproducing this on Build Tools rather than the
full
Visual Studio IDE: `convert-ms-winutils` requires `devenv`, so
`-Duse.platformToolsetVersion=v143` is needed to route through the
`compile-ms-winutils-using-build-tools` path.
### Caveats
1. The suite ran in a service account context, which causes the two "primary
group"
failures above; running as an interactive user would remove them.
2. The account is OAuth-only, so SharedKey and user-bound SAS code paths
were not
covered by this run.
3. The Windows SDK pin affects the build harness only, not the code under
test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
