[
https://issues.apache.org/jira/browse/HADOOP-19977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18111577#comment-18111577
]
ASF GitHub Bot commented on HADOOP-19977:
-----------------------------------------
pan3793 commented on PR #8713:
URL: https://github.com/apache/hadoop/pull/8713#issuecomment-5541960039
@GauthamBanasandra, happy to see you are back.
> Built on Windows 11 with ..., and JDK 8 ...
can not be true - `trunk` requires JDK 17+ now
Jenkins is not stable, so don't be surprised if it fails. The community
recently spent effort moving the CI to GitHub Actions, and the Linux part
should be mostly working. Since you might be the only committer who is
proficient in both Hadoop native code and Windows, would you be interested in
adding Windows coverage to the GitHub Actions CI?
As for `winutils` and `libhadoop`: they date back to an era when Hadoop ran
on a very old JDK that lacked many OS APIs Hadoop needs. Modern JDKs have
changed the picture — they now provide much of that functionality, so migrating
the native code to JDK-provided APIs would significantly reduce its maintenance
burden. I'm currently exploring what that looks like for the Linux/Unix side.
For the Windows portion, though — would you be interested in investigating the
migration there?
> winutils: GetFileInformationByName collides with Windows SDK 10.0.26100
> -----------------------------------------------------------------------
>
> Key: HADOOP-19977
> URL: https://issues.apache.org/jira/browse/HADOOP-19977
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 3.5.0
> Environment: Windows 10/11
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> h3. Symptom
> Building {{hadoop-common}} with {{-Pnative-win}} on Windows fails to compile
> winutils against a current Windows 11 SDK:
> {noformat}
> winutils.h(129,7): error C2733: 'GetFileInformationByName': you cannot
> overload
> a function with 'extern "C"' linkage
> [hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.vcxproj]
> {noformat}
> The build stops at the {{compile-ms-winutils}} step, so no {{winutils.exe}} or
> {{hadoop.dll}} is produced and the whole native Windows build is blocked.
> h3. Root cause
> Windows SDK 10.0.26100.0 introduced its own {{GetFileInformationByName}} in
> {{WinBase.h}} (line 9397), guarded by {{#if (NTDDI_VERSION >=
> NTDDI_WIN11_ZN)}}.
> It collides with the winutils helper of the same name that Hadoop has carried
> for years:
> {code:c}
> /* Windows SDK 10.0.26100.0 - WinBase.h */
> BOOL WINAPI GetFileInformationByName(PCWSTR FileName,
> FILE_INFO_BY_NAME_CLASS
> FileInformationClass,
> PVOID FileInfoBuffer,
> ULONG FileInfoBufferSize);
> /* hadoop - winutils/include/winutils.h:129 */
> DWORD GetFileInformationByName(LPCWSTR pathName, BOOL followLink,
> LPBY_HANDLE_FILE_INFORMATION
> lpFileInformation);
> {code}
> The two differ in signature, so in C++ they would ordinarily overload. But
> overloading requires C++ linkage and both of these have *C* linkage --
> {{winutils.h}} wraps its declarations in {{extern "C"}} (winutils.h:33) and
> the
> SDK header does the same. The compiler therefore rejects the pair outright.
> It surfaces in {{libwinutils.vcxproj}}, which compiles {{config.cpp}} as C++.
> Windows SDK 10.0.22621.0 does not declare the symbol, which is why this has
> only
> started breaking recently -- it is triggered by the installed SDK version,
> not by
> any change in Hadoop.
> h3. Affected code
> {{GetFileInformationByName}} is referenced in 7 places across 5 files, all
> under
> {{hadoop-common-project/hadoop-common/src/main/winutils/}}:
> * {{include/winutils.h}} -- declaration
> * {{libwinutils.c}} -- definition, plus one call in
> {{FindFileOwnerAndPermission}}
> * {{chmod.c}}, {{hardlink.c}}, {{ls.c}} -- call sites
> h3. Proposed fix
> The SDK name cannot be changed, so rename the Hadoop helper. Suggested name is
> {{GetFileInformationByPathName}}, which also reads as the by-path counterpart
> to
> the SDK's existing {{GetFileInformationByHandle}}.
> This is a pure rename of one internal static-library helper and its call
> sites.
> There is no behaviour change, and no public or JNI-visible surface is
> affected.
> h3. To reproduce
> On Windows with Visual Studio 2022 and Windows SDK 10.0.26100.0 installed,
> from
> an x64 Native Tools Command Prompt:
> {noformat}
> mvn clean package -DskipTests -Pnative-win -Duse.platformToolsetVersion=v143 ^
> -Drequire.openssl -Dopenssl.prefix=<vcpkg>\installed\x64-windows ^
> -Dcmake.prefix.path=<vcpkg>\installed\x64-windows ^
> -Dwindows.cmake.toolchain.file=<vcpkg>\scripts\buildsystems\vcpkg.cmake ^
> -Dwindows.cmake.build.type=RelWithDebInfo -Dwindows.build.hdfspp.dll=off ^
> -Dwindows.no.sasl=on -pl hadoop-common-project/hadoop-common -am
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
