Hi Kang Sen:
I reseted kylin branch to 2.5.1 (commit id 4dc4410d) and wanted to
reproduce your compile error but failed. The compiling process successed
without any modify in Windows dev env with jdk 1.8.0_181.
The exception you descripted in previous mails said '[Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
diamond operator is not supported in -source 1.6' etc. It's mean some operator
in source code can't be compiled by JDK1.6. Diamond Operator has been
introduced from JDK1.7.
I think there may be an problem about maven plugin or unmatched complie
env. Now that you solve this problem with some additional config, it's doesn't
matter for you to continue study kylin source code, I think.
And try higher version of kylin may be another way.
Best regards
yuzhang
| |
yuzhang
|
|
[email protected]
|
签名由网易邮箱大师定制
On 4/15/2019 20:00，Lu, Kang-Sen<[email protected]> wrote：
Hi, Yuzhang:
Thanks for your reply. Yes, your answer is correctly indicating that
javaVersion was configured properly in the root pom.xml. I missed that point.
However, In my sandbox, I have updated two pom.xml with the following change:
[root@titanic kylin.2.5.1.new]# git diff engine-spark/pom.xml
diff --git a/engine-spark/pom.xml b/engine-spark/pom.xml
index 4455517f8..35c805122 100644
--- a/engine-spark/pom.xml
+++ b/engine-spark/pom.xml
@@ -125,6 +125,10 @@
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
+ <configuration>
+ <source>1.8</source>
+ <target>1.8</target>
+ </configuration>
<executions>
<execution>
<id>scala-compile-first</id>
Do you know why?
W/o these change, the build would fail.
Kang-sen
From: yuzhang <[email protected]>
Sent: Sunday, April 14, 2019 8:40 PM
To: Lu, Kang-Sen <[email protected]>
Cc:[email protected]
Subject: Re: question about compiling kylin 2.5.1 source error. diamond
operator is not supported in -source 1.6
NOTICE: This email was received from an EXTERNAL sender
Hi, Kang sen:
the code at 42 line of root pom.xml set the "javaVersion" variable like
```
<properties>
<!-- General Properties -->
<javaVersion>1.7</javaVersion>
<maven-model.version>3.3.9</maven-model.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<!-- Hadoop versions -->
<hadoop2.version>2.7.1</hadoop2.version>
<yarn.version>2.7.1</yarn.version>
```
maven will resolve those configuratioin to set dependency version include jdk.
Maybe you should check this config.
Best regard
yuzhang
|
|
yuzhang
|
|
[email protected]
|
签名由 网易邮箱大师 定制
On 4/11/2019 20:34，Lu, Kang-Sen<[email protected]> wrote：
Hi, Yuzhang:
What I meant was on my development machine, the overall environment including
OS, tool chain setup, somehow does not provide definition of “javaVersion”.
Since the kylin release uses javaVersion variable, and the release does not
come with an explicit definition. Then it must depend on some of its
environment to provide the value for that variable, right?
Kang-sen
From: yuzhang <[email protected]>
Sent: Thursday, April 11, 2019 8:24 AM
To: Lu, Kang-Sen <[email protected]>
Cc:[email protected]
Subject: Re: question about compiling kylin 2.5.1 source error. diamond
operator is not supported in -source 1.6
NOTICE: This email was received from an EXTERNAL sender
Yes, but what's the 'But my environment was not able to resolve that variable.
' mean? There have any obvious error about resolve 'javaVersion' during complie?
|
|
yuzhang
|
|
Email:[email protected]
|
Signature is customized by Netease Mail Master
Hi, Yuzhang:
In the root pom.xml, I saw the following:
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>${javaVersion}</source>
<target>${javaVersion}</target>
</configuration>
</plugin>
And my coworker did not have problem to resolve “${javaVersion}”. But my
environment was not able to resolve that variable. I am wondering from where
was this variable supposed to be configured. I could manually modify it to
become 1.8. But that is not what the code was meant to do, isn’t it?
Kang-sen
From: yuzhang <[email protected]>
Sent: Wednesday, April 10, 2019 8:55 PM
To: Lu, Kang-Sen <[email protected]>
Cc:[email protected]
Subject: Re: question about compiling kylin 2.5.1 source error. diamond
operator is not supported in -source 1.6
NOTICE: This email was received from an EXTERNAL sender
Hi, Kang-sen:
Kylin 2.5.x use jdk1.7. Maybe you could modify “javaVersion”to 1.8 in root
pom.xml to solve your problem.
If you have found any mistake, please feel free to correct it.
Best regards
yuzhang
|
|
yuzhang
|
|
[email protected]
|
签名由 网易邮箱大师 定制
On 4/10/2019 20:05，Lu, Kang-Sen<[email protected]> wrote：
Hi, Yuzhang:
I found that if I update some pom.xml with java version, the build will work.
Here is the diff:
[root@titanic kylin.2.5.1.new]# git diff engine-spark/pom.xml
diff --git a/engine-spark/pom.xml b/engine-spark/pom.xml
index 4455517f8..35c805122 100644
--- a/engine-spark/pom.xml
+++ b/engine-spark/pom.xml
@@ -125,6 +125,10 @@
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
+ <configuration>
+ <source>1.8</source>
+ <target>1.8</target>
+ </configuration>
<executions>
<execution>
<id>scala-compile-first</id>
[root@titanic kylin.2.5.1.new]#
Also need to update kylin-it/pom.xml.
A co-worker reported that in his development environment, the kylin release
2.5.3 would build as is. So it must be the environment difference that can
cause the problem. I was not able to figure out what has caused my problem.
Kang-sen
From: Lu, Kang-Sen
Sent: Tuesday, April 09, 2019 11:49 AM
To: 'yuzhang' <[email protected]>
Cc:[email protected]
Subject: RE: question about compiling kylin 2.5.1 source error. diamond
operator is not supported in -source 1.6
[root@titanic kylin.2.5.1.new]# javac -version
javac 1.8.0_191
[root@titanic kylin.2.5.1.new]#
From: yuzhang <[email protected]>
Sent: Tuesday, April 09, 2019 11:47 AM
To: Lu, Kang-Sen <[email protected]>
Cc:[email protected]
Subject: Re: question about compiling kylin 2.5.1 source error. diamond
operator is not supported in -source 1.6
NOTICE: This email was received from an EXTERNAL sender
Hi, what's your jdk version?
|
|
yuzhang
|
|
Email:[email protected]
|
Signature is customized by Netease Mail Master
I updated my kylin 2.5.1 source sandbox, then I issued
“build/script/package.sh”. I got compilation error.
[INFO]
[INFO] ----------------< org.apache.kylin:kylin-engine-spark >-----------------
[INFO] Building Apache Kylin - Spark Engine 2.5.3-anovadata [10/24]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ kylin-engine-spark
---
[INFO] Deleting /home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/target
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-version) @
kylin-engine-spark ---
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (check-style) @
kylin-engine-spark ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.0:prepare-agent (pre-test) @
kylin-engine-spark ---
[INFO] surefireArgLine set to
-javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.0/org.jacoco.agent-0.8.0-runtime.jar=destfile=/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/../target/jacoco.exec,append=true
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles)
@ kylin-engine-spark ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @
kylin-engine-spark ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- scala-maven-plugin:4.0.1:add-source (scala-compile-first) @
kylin-engine-spark ---
[INFO] Add Source directory:
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/scala
[INFO] Add Test Source directory:
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/test/scala
[INFO]
[INFO] --- scala-maven-plugin:4.0.1:compile (scala-compile-first) @
kylin-engine-spark ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiling 1 Scala source and 15 Java sources to
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/target/classes ...
[WARNING] [Warn] : bootstrap class path not set in conjunction with -source 1.6
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:239:
try-with-resources is not supported in -source 1.6
(use -source 7 or higher to enable try-with-resources)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:251:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingMerge.java:159:
try-with-resources is not supported in -source 1.6
(use -source 7 or higher to enable try-with-resources)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingMerge.java:173:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java:157:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java:267:
try-with-resources is not supported in -source 1.6
(use -source 7 or higher to enable try-with-resources)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkMergingDictionary.java:185:
try-with-resources is not supported in -source 1.6
(use -source 7 or higher to enable try-with-resources)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkMergingDictionary.java:223:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/home/zettics/sandbox/klu/kylin.2.5.1.new/engine-spark/src/main/java/org/apache/kylin/engine/spark/util/IteratorUtils.java:58:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Kylin 2.5.3-anovadata ....................... SUCCESS [ 2.558 s]
[INFO] Apache Kylin - Core Common ......................... SUCCESS [ 5.910 s]
[INFO] Apache Kylin - Core Metadata ....................... SUCCESS [ 5.879 s]
[INFO] Apache Kylin - Core Dictionary ..................... SUCCESS [ 4.300 s]
[INFO] Apache Kylin - Core Cube ........................... SUCCESS [ 5.396 s]
[INFO] Apache Kylin - Core Metrics ........................ SUCCESS [ 1.238 s]
[INFO] Apache Kylin - Core Job ............................ SUCCESS [ 2.801 s]
[INFO] Apache Kylin - Core Storage ........................ SUCCESS [ 2.825 s]
[INFO] Apache Kylin - MapReduce Engine .................... SUCCESS [ 5.395 s]
[INFO] Apache Kylin - Spark Engine ........................ FAILURE [ 7.884 s]
[INFO] Apache Kylin - Hive Source ......................... SKIPPED
[INFO] Apache Kylin - Kafka Source ........................ SKIPPED
[INFO] Apache Kylin - HBase Storage ....................... SKIPPED
[INFO] Apache Kylin - Query ............................... SKIPPED
[INFO] Apache Kylin - Metrics Reporter Hive ............... SKIPPED
[INFO] Apache Kylin - Metrics Reporter Kafka .............. SKIPPED
[INFO] Apache Kylin - REST Server Base .................... SKIPPED
[INFO] Apache Kylin - REST Server ......................... SKIPPED
[INFO] Apache Kylin - JDBC Driver ......................... SKIPPED
[INFO] Apache Kylin - Assembly ............................ SKIPPED
[INFO] Apache Kylin - Tool ................................ SKIPPED
[INFO] Apache Kylin - Tool Assembly ....................... SKIPPED
[INFO] Apache Kylin - Integration Test .................... SKIPPED
[INFO] Apache Kylin - Tomcat Extension 2.5.3-anovadata .... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.573 s
[INFO] Finished at: 2019-04-09T11:38:32-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:4.0.1:compile (scala-compile-first) on
project kylin-engine-spark: Execution scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:4.0.1:compile failed.: CompileFailed ->
[Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :kylin-engine-spark
[root@titanic kylin.2.5.1.new]#
Notice: This e-mail together with any attachments may contain information of
Ribbon Communications Inc. that is confidential and/or proprietary for the sole
use of the intended recipient. Any review, disclosure, reliance or distribution
by others or forwarding without express permission is strictly prohibited. If
you are not the intended recipient, please notify the sender immediately and
then delete all copies, including any attachments.
On 04/09/2019 23:41, Lu, Kang-Sen wrote:
On 04/11/2019 19:57, Lu, Kang-Sen wrote:
