[
https://issues.apache.org/jira/browse/HDFS-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359695#comment-17359695
]
Gautham Banasandra edited comment on HDFS-15971 at 6/9/21, 5:41 AM:
--------------------------------------------------------------------
Hi [~Jim_Brennan],
I'm sorry for the inconvenience that you're facing. Please find my answers
below -
*_It looks like HDFS-15740 caused a need for the rhel8 build instructions to be
updated_*
I'm working on a bigger change HADOOP-17193 - this requires the latest C++
compiler to be used. At least the one which supports the C++17 standard. Hence,
the upgrade from GCC 4.8.1 to 9.3.0. Now, if we think about the possibility of
having the compiler upgraded only for Windows and let GCC 4.8.1 stay for Linux
systems, this would lead to a huge dichotomy in the codebase, which would be
extremely difficult to maintain and to make further improvements/fixes.
*_The Cmake update seems reasonable, but do these changes really justify
updating the GCC from 4.8 to 9.3 - that's a pretty significant change. I don't
think we usually upgrade dependencies that aggressively._*
I believe you had HDFS-15740 in mind when you wrote this, but thinking about
HADOOP-17193, it certainly is huge. Resolving HADOOP-17193 will enable Hadoop
to be used on Windows platform. It will also get rid of the age old winutils
issues HADOOP-13223. This'll not just benefit Hadoop, but also the ecosystem
that runs on top of Hadoop - Spark on Windows still needs winutils.exe. I've
also fixed a lot of bugs on my journey through HADOOP-17193, please take a look
its children and related JIRAs - this would've been extremely difficult without
this upgrade. With all this in mind, I believe the compiler upgrade is
justified.
*_[~elgoiri] mentioned fuse dependencies. Does this change require Linux FUSE?
I think that used to be optional._*
No, this change has no relation with Linux FUSE. I think [~elgoiri]
[mentioned|https://issues.apache.org/jira/browse/HDFS-15971?focusedCommentId=17352081&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17352081]
about FUSE because of the CMake stack trace seen in [~ebadger]'s
[comment|https://issues.apache.org/jira/browse/HDFS-15971?focusedCommentId=17322535&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17322535].
{code}
[WARNING] -- Checking for module 'fuse'
[WARNING] -- No package 'fuse' found
[WARNING] -- Failed to find Linux FUSE libraries or include files. Will not
build FUSE client.
{code}
What [~goiri] was saying is that the build setup used by [~ebadger] doesn't
contain FUSE, however, the [^Dockerfile_centos_7] has FUSE installed and was
suggesting to use this.
*_I'm also puzzled by this failing check. This did not fail on 4.8.5 before
this commit (and the warning message hasn't been updated either):_*
This is coming from
[here|https://github.com/apache/hadoop/blob/bcaeb1ac8cd67a14bd332d62bdec1a90ce1c9549/hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt#L159-L178].
Let me take a look at this and get back to you. Nevertheless, we don't face
this issue when GCC is upgraded to 9.3.0.
[~Jim_Brennan] have you considered upgrading GCC to 9.3.0 in your system? I've
also added the
[Dockerfile|https://github.com/apache/hadoop/blob/trunk/dev-support/docker/Dockerfile_centos_7]
for convenience of those who wish to run Hadoop on Centos 7.
was (Author: gautham):
Hi [~Jim_Brennan],
I'm sorry for the inconvenience that you're facing. Please find my answers
below -
*_It looks like HDFS-15740 caused a need for the rhel8 build instructions to be
updated_*
I'm working on a bigger change HADOOP-17193 - this requires the latest C++
compiler to be used. At least the one which supports the C++17 standard. Hence,
the upgrade from GCC 4.8.1 to 9.3.0. Now, if we think about the possibility of
having the compiler upgraded only for Windows and let GCC 4.8.1 stay for Linux
systems, this would lead to a huge dichotomy in the codebase, which would be
extremely difficult to maintain and to make further improvements/fixes.
*_The Cmake update seems reasonable, but do these changes really justify
updating the GCC from 4.8 to 9.3 - that's a pretty significant change. I don't
think we usually upgrade dependencies that aggressively._*
I believe you had HDFS-15740 in mind when you wrote this, but thinking about
HADOOP-17193, it certainly is huge. Resolving HADOOP-17193 will enable Hadoop
to be used on Windows platform. It will also get rid of the age old winutils
issues HADOOP-13223. This'll not just benefit Hadoop, but also the ecosystem
that runs on top of Hadoop - Spark on Windows still needs winutils.exe. I've
also fixed a lot of bugs on my journey through HADOOP-17193, please take a look
its children and related JIRAs. With all this in mind, I believe the compiler
upgrade is justified.
*_[~elgoiri] mentioned fuse dependencies. Does this change require Linux FUSE?
I think that used to be optional._*
No, this change has no relation with Linux FUSE. I think [~elgoiri]
[mentioned|https://issues.apache.org/jira/browse/HDFS-15971?focusedCommentId=17352081&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17352081]
about FUSE because of the CMake stack trace seen in [~ebadger]'s
[comment|https://issues.apache.org/jira/browse/HDFS-15971?focusedCommentId=17322535&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17322535].
{code}
[WARNING] -- Checking for module 'fuse'
[WARNING] -- No package 'fuse' found
[WARNING] -- Failed to find Linux FUSE libraries or include files. Will not
build FUSE client.
{code}
What [~goiri] was saying is that the build setup used by [~ebadger] doesn't
contain FUSE, however, the [^Dockerfile_centos_7] has FUSE installed and was
suggesting to use this.
*_I'm also puzzled by this failing check. This did not fail on 4.8.5 before
this commit (and the warning message hasn't been updated either):_*
This is coming from
[here|https://github.com/apache/hadoop/blob/bcaeb1ac8cd67a14bd332d62bdec1a90ce1c9549/hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt#L159-L178].
Let me take a look at this and get back to you. Nevertheless, we don't face
this issue when GCC is upgraded to 9.3.0.
[~Jim_Brennan] have you considered upgrading GCC to 9.3.0 in your system? I've
also added the
[Dockerfile|https://github.com/apache/hadoop/blob/trunk/dev-support/docker/Dockerfile_centos_7]
for convenience of those who wish to run Hadoop on Centos 7.
> Make mkstemp cross platform
> ---------------------------
>
> Key: HDFS-15971
> URL: https://issues.apache.org/jira/browse/HDFS-15971
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: libhdfs++
> Affects Versions: 3.4.0
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: Dockerfile_centos_7, build-log.zip, commit-details.txt
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> mkstemp isn't available in Visual C++. Need to make it cross platform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
