[
https://issues.apache.org/jira/browse/HDDS-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758410#comment-16758410
]
Hadoop QA commented on HDDS-922:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 44s{color}
| {color:red} hadoop-ozone in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
44s{color} | {color:green} hadoop-hdds in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.client.rpc.TestOzoneRpcClient |
| | hadoop.ozone.client.rpc.TestFailureHandlingByClient |
| | hadoop.ozone.om.TestScmChillMode |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-922 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957267/HDDS-922.004.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle
shellcheck |
| uname | Linux 6f13c5d0740b 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HDDS-Build/ozone.sh |
| git revision | trunk / 13aa939 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| shellcheck | v0.4.6 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDDS-Build/2166/artifact/out/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDDS-Build/2166/artifact/out/patch-unit-hadoop-ozone.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/2166/testReport/ |
| Max. process+thread count | 1120 (vs. ulimit of 10000) |
| modules | C: hadoop-hdds/common hadoop-hdds/docs hadoop-ozone
hadoop-ozone/dist hadoop-ozone/ozonefs hadoop-ozone/ozonefs-lib
hadoop-ozone/ozonefs-lib-legacy U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/2166/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Create isolated classloder to use ozonefs with any older hadoop versions
> ------------------------------------------------------------------------
>
> Key: HDDS-922
> URL: https://issues.apache.org/jira/browse/HDDS-922
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Filesystem
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Labels: pull-request-available
> Attachments: HDDS-922.001.patch, HDDS-922.002.patch,
> HDDS-922.003.patch, HDDS-922.004.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As of now we create a shaded ozonefs artifact which includes all the required
> class files to use ozonefs (Hadoop compatible file system for Ozone)
> But the shading process of this artifact is very easy, it includes all the
> class files but no relocation rules (package name renaming) are configured.
> With this approach ozonefs can be used from the compatible hadoop version
> (this is hadoop 3.1 only, I guess) but can't be used with any older hadoop
> version as it requires the newer version of hadoop-common.
> I tried to configure a full shading (with relocation) but it's not a simple
> task. For example a pure (non-relocated) Configuration is required by the
> ozonefs itself, but an other, newer Configuration class is required by the
> ozone client code which is a dependency of OzoneFileSystem So we need a
> relocated and a non-relocated class in the same time.
> I tried out a different approach: I moved out all of the ozone specific
> classes from the OzoneFileSystem to an adapter class (OzoneClientAdapter). In
> case of an older hadoop version the adapter class itself can be loaded with
> an isolated classloader. The isolated classloader can load all the required
> classes from the jar file from a specific path. It doesn't require any
> specific package relocation as the default class loader doesn't load these
> classes.
> The OzoneFileSystem (in case of older hadoop version) can load the adapter
> with the isolated classloader and only a few classes should be shared between
> the normal and isolated classloader (the interface of the adapter and the
> types in the method signatures). All of the other ozone classes and the newer
> hadoop dependencies will be hidden by the isolated classloader.
> This patch is more like a proof of concept, I would like to start a
> discussion about this approach. I successfully used the generated artifact to
> use ozonefs from spark 2.4 default distribution (which includes hadoop 2.7).
> For a final patch I would add some check to use the ozonefs without any
> classpath separation by default. (could be configured or chosen by
> automatically)
> For using spark (+ hadoop 2.7 + kubernetes scheduler) together with ozone,
> you can check this screencast:
> https://www.youtube.com/watch?v=cpRJcSHIEdM&t=8s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
