[
https://issues.apache.org/jira/browse/YETUS-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248692#comment-15248692
]
Allen Wittenauer commented on YETUS-222:
----------------------------------------
bq. Surely it will perform best on x86 architectures, but will fall back and
still perform better than pure Java implementations.
My understanding of this code is that it may also utilize OpenSSL. Then
question then becomes if the OpenSSL implementation performs better than ISAL
in 'compatibility' mode. Given that the vast majority of people are more likely
to have OpenSSL installed, we may actually be hurting our end users by testing
with ISAL as a higher priority...
bq. I may misunderstand YETUS and the task, but I guess it's only for project
development, building and testing?
Apache Yetus is used for patch testing by a significant numbers of contributors
in Apache Hadoop on a wide variety of diverse platforms. We need to be *very*
careful here otherwise people will break in unexpected and surprising ways.
bq. I'm not sure about we're going to run the scripts in specific
architectures like AMD, ARM and etc. as mentioned above?
Yes, people do. Apache Yetus was tested on Windows+cygwin very very early on.
bq. The users should be developers as we are, and the running environment
should be limited to ubuntu, centos and mac?
Um, no. Definitely not. Don't confuse commercial support with what is happening
in open source. Apache Hadoop has never stated a platform compatibility
requirement. The project has people regularly contributing fixes for Windows,
Linux/ppc, FreeBSD, Solaris/SPARC, Solaris/x86, Linux/ARM, ... and probably
others, from quite a few large, well known companies who are known to have
deployments on those platforms. (Heck, I personally ran it on Solaris for
LinkedIn for two years!)
bq. I'm not a hardware person, but we're surely not developing a library like
ISA-L and we'd better not couple with the specifics in the high level.
Of course not, but we should not do things that purposefully break those other
platforms. For example, lots of people have been spending a lot of time
cleaning up the mess that is Hadoop's native code to be POSIX compliant or at
least work in a portable way so that it does work out of the box. In the case
of Apache Yetus, this means making sure that project's personalities work on
more platforms than not.
bq. It looks good to support more options other than ISA-L, but I don't see
any other better option it supports actually.
Your vendor hat is showing.
bq. Furthermore, how it will solve the concern and problem we're discussing
about here? With any real back end is used, we'll still need to worry about the
packaging and testing stuffs.
IMO:
a) liberasurecode is known to be cross-platform across a variety of OSes and
chipsets. Making it required becomes a much easier decision.
b) We should have greatly reduced unit testing requirements given that
liberasurecode would be responsible for large parts of it rather than Hadoop
doing all of the major lifting.
bq. 1) for test and developer, need to provide RPM like package installations
on developer friendly and Jenkins target platforms;
People shouldn't be building platform-level code so that it specifically passes
precommit. All we have to do is turn on builds for FreeBSD, Windows, Solaris,
Mac OS X, and Linux/ARM (all of which Jenkins has access to) and the world
falls apart. (One of the big advantages of using YETUS-156 to drive nightly
builds instead of the ad hoc mvn command list Hadoop has now would be
consistent testing regardless of the platform.)
bq. For this issue, if we'd like to close it as won't fix, maybe it won't be
bad to have it pending more time.
That's what I'm thinking too.
Thanks.
> hadoop: add ISA-L support
> -------------------------
>
> Key: YETUS-222
> URL: https://issues.apache.org/jira/browse/YETUS-222
> Project: Yetus
> Issue Type: Improvement
> Components: Test Patch
> Reporter: Allen Wittenauer
> Assignee: Kai Zheng
> Attachments: YETUS-222-v1.patch, YETUS-222-v2.patch
>
>
> Hadoop personality should support building the native code under Jenkins when
> Docker is invoked.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
