date:20200624

[carbondata] branch master updated: change build status to spark2.4

2020-06-24 Thread chenliang613

This is an automated email from the ASF dual-hosted git repository.

chenliang613 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new 9727e02  change build status to spark2.4
 new 3663760  Update build status to spark2.4 This closes #3806
9727e02 is described below

commit 9727e02353b85844dbbac85e07dfd4c89e138511
Author: chenliang613 
AuthorDate: Thu Jun 25 11:25:27 2020 +0800

change build status to spark2.4
---
 README.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 0293c4c..f706e1c 100644
--- a/README.md
+++ b/README.md
@@ -28,8 +28,8 @@ Visit count: 
[![HitCount](http://hits.dwyl.io/jackylk/apache/carbondata.svg)](ht
 
 
 ## Status
-Spark2.3:
-[![Build 
Status](https://builds.apache.org/buildStatus/icon?job=carbondata-master-spark-2.3)](https://builds.apache.org/view/A-D/view/CarbonData/job/carbondata-master-spark-2.3/lastBuild/testReport)
+Spark2.4:
+[![Build 
Status](https://builds.apache.org/buildStatus/icon?job=carbondata-master-spark-2.4)](https://builds.apache.org/view/A-D/view/CarbonData/job/carbondata-master-spark-2.4/lastBuild/testReport)
 [![Coverage 
Status](https://coveralls.io/repos/github/apache/carbondata/badge.svg?branch=master)](https://coveralls.io/github/apache/carbondata?branch=master)
 https://scan.coverity.com/projects/carbondata;>

Build failed in Jenkins: carbondata-master-spark-2.4 #2325

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[...truncated 11.73 KB...]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-parent ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-parent 
---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-parent ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-parent ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-parent ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-parent:pom:2.1.0-SNAPSHOT...
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-parent >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-parent 
---
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-parent <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-parent ---
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-parent ---
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Common 2.1.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-common ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] 
[INFO] Building Apache

Build failed in Jenkins: carbondata-master-spark-2.4 » Apache CarbonData :: Format #2325

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Format 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.pom
 (4 KB at 127.3 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.jar
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.jar
 (18 KB at 694.4 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-format ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-format 
---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-format ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-thrift-plugin:0.1.11:compile (generate-thrift-java) @ 
carbondata-format ---
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.pom
 (3 KB at 122.9 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.pom
 (2 KB at 67.7 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.jar
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.jar
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.jar
 (8 KB at 96.2 KB/sec)
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
 (23 KB at 293.5 KB/sec)
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.jar
 (473 KB at 4258.7 KB/sec)
[ERROR] thrift failed output: 
[ERROR] thrift failed error: /bin/sh: 1: thrift: not found

Build failed in Jenkins: carbondata-master-spark-2.3 #2324

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[...truncated 10.27 KB...]
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-parent ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-parent ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-parent ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-parent:pom:2.1.0-SNAPSHOT...
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-parent >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-parent 
---
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-parent <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-parent ---
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-parent ---
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Common 2.1.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-common ---
[INFO] Deleting 

[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Format #2324

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Format 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.pom
 (4 KB at 8.7 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.jar
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/apache/thrift/tools/maven-thrift-plugin/0.1.11/maven-thrift-plugin-0.1.11.jar
 (18 KB at 754.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-format ---
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-format 
---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-format ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-thrift-plugin:0.1.11:compile (generate-thrift-java) @ 
carbondata-format ---
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.pom
 (3 KB at 116.8 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.pom
 (2 KB at 83.6 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.jar
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.jar
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.5.8/slf4j-simple-1.5.8.jar
 (8 KB at 194.9 KB/sec)
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
 (23 KB at 532.5 KB/sec)
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/com/google/collections/google-collections/0.8/google-collections-0.8.jar
 (473 KB at 5028.8 KB/sec)
[ERROR] thrift failed output: 
[ERROR] thrift failed error: /bin/sh: 1: thrift: not found

Build failed in Jenkins: carbondata-master-spark-2.3 #2323

2020-06-24 Thread Apache Jenkins Server

See 


Changes:

[liuzhi] [CARBONDATA-3862] Insert stage performance optimazation


--
[...truncated 16.02 KB...]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-common ---
[INFO] Deleting 

[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2323

2020-06-24 Thread Apache Jenkins Server

See 


Changes:

[liuzhi] [CARBONDATA-3862] Insert stage performance optimazation


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[WARNING] The POM for 
org.apache.carbondata:carbondata-format:jar:2.1.0-SNAPSHOT is missing, no 
dependency information available
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar

[carbondata] branch master updated: [CARBONDATA-3862] Insert stage performance optimazation

2020-06-24 Thread liuzhi

This is an automated email from the ASF dual-hosted git repository.

liuzhi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new 755d78c  [CARBONDATA-3862] Insert stage performance optimazation
755d78c is described below

commit 755d78c03e7e08f5b6763f9b206b7ebf3ca2daf2
Author: haomarch 
AuthorDate: Mon Jun 22 05:21:27 2020 +0800

[CARBONDATA-3862] Insert stage performance optimazation

Why is this PR needed?
There are two major performance bottlenecks of 'insert stage'.
1) Get LastModifyTime of stagefiles requires a lot of access to OBS.
2) Parallelism is not supported

What changes were proposed in this PR?
1) Cache the lastmodifytime info when list stage files.
2) support insert stage in parallel. we add a tag 'loading' to the stages 
in process. different insertstage processes can load different data separately 
by choose the stages without 'loading' tag or stages loaded timeout. which 
avoid loading the same data between concurrent insertstage processes. The 
'loading' tag is actually an empty file with '.loading' suffix filename.

Does this PR introduce any user interface change?
NO

Is any new testcase added?
YES

This closes #3799
---
 .../core/constants/CarbonCommonConstants.java  |  10 ++
 .../filesystem/AbstractDFSCarbonFile.java  |   5 +
 .../carbondata/core/util/CarbonProperties.java |  32 +
 .../carbondata/core/util/path/CarbonTablePath.java |   1 +
 docs/configuration-parameters.md   |   1 +
 .../carbon/flink/TestCarbonPartitionWriter.scala   |  28 
 .../management/CarbonInsertFromStageCommand.scala  | 153 +
 7 files changed, 203 insertions(+), 27 deletions(-)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 
b/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
index 4e24618..4a0605e 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
@@ -1521,6 +1521,16 @@ public final class CarbonCommonConstants {
 
   public static final String CARBON_QUERY_STAGE_INPUT_DEFAULT = "false";
 
+  /**
+   * MAX_TIMEOUT_FOR_INSERTSTAGE_JOB
+   */
+  public static final String CARBON_INSERT_STAGE_TIMEOUT = 
"carbon.insert.stage.timeout";
+
+  /**
+   * DFAULT_MAX_TIMEOUT_FOR_INSERTSTAGE_JOB: 8 hour
+   */
+  public static final long CARBON_INSERT_STAGE_TIMEOUT_DEFAULT = 2880;
+
   
//
   // Index parameter start here
   
//
diff --git 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java
 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java
index 4459420..c2d81db 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java
@@ -63,6 +63,7 @@ public abstract class AbstractDFSCarbonFile implements 
CarbonFile {
   protected FileSystem fileSystem;
   protected Configuration hadoopConf;
   protected Path path;
+  protected FileStatus fileStatus;
 
   AbstractDFSCarbonFile(String filePath) {
 this(filePath, FileFactory.getConfiguration());
@@ -89,6 +90,7 @@ public abstract class AbstractDFSCarbonFile implements 
CarbonFile {
 
   AbstractDFSCarbonFile(FileStatus fileStatus) {
 this(fileStatus.getPath());
+this.fileStatus = fileStatus;
   }
 
   @Override
@@ -177,6 +179,9 @@ public abstract class AbstractDFSCarbonFile implements 
CarbonFile {
   @Override
   public long getLastModifiedTime() {
 try {
+  if (this.fileStatus != null) {
+return this.fileStatus.getModificationTime();
+  }
   return fileSystem.getFileStatus(path).getModificationTime();
 } catch (IOException e) {
   throw new CarbonFileException("Unable to get file status: ", e);
diff --git 
a/core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java 
b/core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
index a238877..90409e8 100644
--- a/core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
+++ b/core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
@@ -1904,6 +1904,38 @@ public final class CarbonProperties {
   }
 
   /**
+   * Validate and get the input metrics interval
+   *
+   * @return input metrics interval
+   */
+  public static Long getInsertStageTimeout() {
+String timeout = CarbonProperties.getInstance()
+

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2322

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[WARNING] The POM for 
org.apache.carbondata:carbondata-format:jar:2.1.0-SNAPSHOT is missing, no 
dependency information available
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar

Build failed in Jenkins: carbondata-master-spark-2.3 #2322

2020-06-24 Thread Apache Jenkins Server

See 


Changes:


--
[...truncated 16.08 KB...]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-common ---
[INFO] Deleting 

[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 2 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install

[carbondata] branch master updated: change build status to spark2.4

Build failed in Jenkins: carbondata-master-spark-2.4 #2325

Build failed in Jenkins: carbondata-master-spark-2.4 » Apache CarbonData :: Format #2325

Build failed in Jenkins: carbondata-master-spark-2.3 #2324

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Format #2324

Build failed in Jenkins: carbondata-master-spark-2.3 #2323

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2323

[carbondata] branch master updated: [CARBONDATA-3862] Insert stage performance optimazation

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2322

Build failed in Jenkins: carbondata-master-spark-2.3 #2322

10 matches

Site Navigation

Mail list logo

Footer information