date:20180316

Build failed in Jenkins: carbondata-master-spark-2.1 #2149

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon 
files

--
[...truncated 4.39 MB...]
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.2 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.6.2.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including xml-apis:xml-apis:jar:1.3.04 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded 
jar.
[INFO] Including org.apache.avro:avro:jar:1.8.1 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded 
jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the 
shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.7 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.1.3 in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.8.1 in the shaded 
jar.
[INFO] Including org.tukaani:xz:jar:1.5 in the shaded jar.
[INFO] Including org.slf4j:slf4j-api:jar:1.7.7 in the shaded jar.
[WARNING] commons-logging-1.1.3.jar, jcl-over-slf4j-1.7.16.jar define 6 
overlapping classes: 
[WARNING]   - org.apache.commons.logging.impl.SimpleLog$1
[WARNING]   - org.apache.commons.logging.Log
[WARNING]   - org.apache.commons.logging.impl.SimpleLog
[WARNING]   - org.apache.commons.logging.LogConfigurationException
[WARNING]   - org.apache.commons.logging.impl.NoOpLog
[WARNING]   - org.apache.commons.logging.LogFactory
[WARNING] commons-collections-3.2.2.jar, commons-beanutils-core-1.8.0.jar, 
commons-beanutils-1.7.0.jar define 10 overlapping classes: 
[WARNING]   - org.apache.commons.collections.FastHashMap$EntrySet
[WARNING]   - org.apache.commons.collections.FastHashMap$KeySet
[WARNING]   - org.apache.commons.collections.ArrayStack
[WARNING]   - 
org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
[WARNING]   - org.apache.commons.collections.FastHashMap$Values
[WARNING]   - org.apache.commons.collections.FastHashMap$CollectionView
[WARNING]   - org.apache.commons.collections.FastHashMap$1
[WARNING]   - org.apache.commons.collections.Buffer
[WARNING]   - org.apache.commons.collections.FastHashMap
[WARNING]   - org.apache.commons.collections.BufferUnderflowException
[WARNING] jersey-core-1.9.jar, javax.ws.rs-api-2.0.1.jar define 55 overlapping 
classes: 
[WARNING]   - javax.ws.rs.core.HttpHeaders
[WARNING]   - javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate
[WARNING]   - javax.ws.rs.DefaultValue
[WARNING]   - javax.ws.rs.core.StreamingOutput
[WARNING]   - javax.ws.rs.HEAD
[WARNING]   - javax.ws.rs.core.Request
[WARNING]   - javax.ws.rs.ext.Providers
[WARNING]   - javax.ws.rs.core.NewCookie
[WARNING]   - javax.ws.rs.core.UriBuilderException
[WARNING]   - javax.ws.rs.ext.ContextResolver
[WARNING]   - 45 more...
[WARNING] hadoop-yarn-api-2.2.0.jar, hadoop-yarn-common-2.7.2.jar define 1 
overlapping classes: 
[WARNING]   - org.apache.hadoop.yarn.util.package-info
[WARNING] jersey-server-1.9.jar, jersey-server-2.22.2.jar define 18 overlapping 
classes: 
[WARNING]   - com.sun.research.ws.wadl.Representation
[WARNING]   - com.sun.research.ws.wadl.Method
[WARNING]   - com.sun.research.ws.wadl.Request
[WARNING]   - com.sun.research.ws.wadl.package-info
[WARNING]   - com.sun.research.ws.wadl.ResourceType
[WARNING]   - com.sun.research.ws.wadl.Param
[WARNING]   - com.sun.research.ws.wadl.Link
[WARNING]   - com.sun.research.ws.wadl.Application
[WARNING]   - com.sun.research.ws.wadl.ParamStyle
[WARNING]   - com.sun.research.ws.wadl.Doc
[WARNING]   - 8 more...
[WARNING] commons-beanutils-core-1.8.0.jar, commons-beanutils-1.7.0.jar define 
82 overlapping classes: 
[WARNING]   - org.apache.commons.beanutils.ConvertUtilsBean
[WARNING]   - org.apache.commons.beanutils.converters.SqlTimeConverter
[WARNING]   - org.apache.commons.beanutils.Converter
[WARNING]   - org.apache.commons.beanutils.converters.FloatArrayConverter
[WARNING]   - org.apache.commons.beanutils.NestedNullException
[WARNING]   - org.apache.commons.beanutils.ConvertingWrapDynaBean
[WARNING]   - org.apache.commons.beanutils.converters.LongArrayConverter
[WARNING]   - org.apache.commons.beanutils.converters.SqlDateConverter
[WARNING]   - org.apache.commons.beanutils.converters.BooleanArrayConverter
[WARNING]   - org.apache.commons.beanutils.converters.StringConverter
[WARNING]   - 72 more...
[WARNING] spark-launcher_2.11-2.1.0.jar, spark-core_2.11-2.1.0.jar, 
spark-unsafe_2.11-2.1.0.jar, spark-network-common_2.11-2.1.0.jar, 
spark-sql_2.11-2.1.0.jar, unused-1.0.0.jar, spark-sketch_2.11-2.1.0.jar, 
spark-network-shuffle_2.11-2.1.0.jar, spark-catalyst_2.11-2.1.0.jar, 
spark-tags_2.11-2.1.0.jar define 1 overlapping classes: 
[WARNING]   -

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2149

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon 
files

--
[...truncated 8.54 KB...]
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.xbean:xbean-asm5-shaded:jar:4.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 in the shaded 
jar.
[INFO] Including com.google.inject:guice:jar:3.0 in the shaded jar.
[INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.1.0 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.1.0 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.scala-lang:scala-library:jar:2.11.8 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-common:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.2 in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2 in the shaded 
jar.
[INFO] Including org.glassfish.hk2:osgi-resource-locator:jar:1.0.1 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-server:jar:2.22.2 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.validation:validation-api:jar:1.1.0.Final in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 in the 
shaded jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 in the 
shaded jar.
[INFO] Including io.netty:netty-all:jar:4.0.42.Final in the shaded jar.
[INFO] Including com.clearspring.analytics:stream:jar:2.7.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-jvm:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-json:jar:3.1.2 in the shaded

Build failed in Jenkins: carbondata-master-spark-2.2 #187

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon 
files

--
[...truncated 4.40 MB...]
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.1.2 in the shaded 
jar.
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.1.2 in the shaded jar.
[INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including commons-configuration:commons-configuration:jar:1.6 in the 
shaded jar.
[INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar.
[INFO] Including commons-beanutils:commons-beanutils:jar:1.7.0 in the shaded 
jar.
[INFO] Including commons-beanutils:commons-beanutils-core:jar:1.8.0 in the 
shaded jar.
[INFO] Including org.slf4j:slf4j-log4j12:jar:1.7.10 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.2 in the shaded jar.
[INFO] Including 
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 in the shaded 
jar.
[INFO] Including org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-asn1-api:jar:1.0.0-M20 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-util:jar:1.0.0-M20 in the shaded 
jar.
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded 
jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.2 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.6.2.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including xml-apis:xml-apis:jar:1.3.04 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded 
jar.
[INFO] Including org.apache.avro:avro:jar:1.8.1 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded 
jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the 
shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.7 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.1.3 in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.8.1 in the shaded 
jar.
[INFO] Including org.tukaani:xz:jar:1.5 in the shaded jar.
[INFO] Including org.slf4j:slf4j-api:jar:1.7.7 in the shaded jar.
[WARNING] jersey-core-1.9.jar, javax.ws.rs-api-2.0.1.jar define 55 overlapping 
classes: 
[WARNING]   - javax.ws.rs.core.HttpHeaders
[WARNING]   - javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate
[WARNING]   - javax.ws.rs.DefaultValue
[WARNING]   - javax.ws.rs.core.StreamingOutput
[WARNING]   - javax.ws.rs.HEAD
[WARNING]   - javax.ws.rs.core.Request
[WARNING]   - javax.ws.rs.ext.Providers
[WARNING]   - javax.ws.rs.core.NewCookie
[WARNING]   - javax.ws.rs.core.UriBuilderException
[WARNING]   - javax.ws.rs.ext.ContextResolver
[WARNING]   - 45 more...
[WARNING] commons-logging-1.1.3.jar, jcl-over-slf4j-1.7.16.jar define 6 
overlapping classes: 
[WARNING]   - org.apache.commons.logging.impl.SimpleLog$1
[WARNING]   - org.apache.commons.logging.Log
[WARNING]   - org.apache.commons.logging.impl.SimpleLog
[WARNING]   - org.apache.commons.logging.LogConfigurationException
[WARNING]   - org.apache.commons.logging.impl.NoOpLog
[WARNING]   - org.apache.commons.logging.LogFactory
[WARNING] spark-launcher_2.11-2.2.1.jar, spark-tags_2.11-2.2.1.jar, 
spark-network-common_2.11-2.2.1.jar, spark-catalyst_2.11-2.2.1.jar, 
spark-sketch_2.11-2.2.1.jar, spark-sql_2.11-2.2.1.jar, 
spark-unsafe_2.11-2.2.1.jar, spark-core_2.11-2.2.1.jar, 
spark-network-shuffle_2.11-2.2.1.jar, unused-1.0.0.jar define 1 overlapping 
classes: 
[WARNING]   - org.apache.spark.unused.UnusedStubClass
[WARNING] commons-beanutils-1.7.0.jar, commons-beanutils-core-1.8.0.jar, 
commons-collections-3.2.2.jar define 10 overlapping classes: 
[WARNING]   - org.apache.commons.collections.FastHashMap$EntrySet
[WARNING]   - org.apache.commons.collections.FastHashMap$KeySet
[WARNING]   - org.apache.commons.collections.ArrayStack
[WARNING]   - 
org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
[WARNING]   - org.apache.commons.collections.FastHashMap$Values
[WARNING]   - org.apache.commons.collections.FastHashMap$CollectionView
[WARNING]   -

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #187

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon 
files

--
[...truncated 6.39 KB...]
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-store-sdk ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-store-sdk ---
[INFO] 
[INFO] --- maven-shade-plugin:2.4.3:shade (default) @ carbondata-store-sdk ---
[INFO] Including org.apache.carbondata:carbondata-hadoop:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-processing:jar:1.4.0-SNAPSHOT 
in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-core:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-format:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.thrift:libthrift:jar:0.9.3 in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-common:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.3.1 in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.7 in the shaded jar.
[INFO] Including org.apache.spark:spark-sql_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-sketch_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-core_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 in the shaded 
jar.
[INFO] Including org.apache.avro:avro-ipc:jar:1.7.7 in the shaded jar.
[INFO] Including org.apache.avro:avro-ipc:jar:tests:1.7.7 in the shaded jar.
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.6.5 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.5 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.2.1 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.scala-lang:scala-library:jar:2.11.8 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-xml_2.11:jar:1.0.4 in the shaded 
jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including

Jenkins build is back to normal : carbondata-master-spark-2.2 #186

2018-03-16 Thread Apache Jenkins Server

See

Jenkins build is back to stable : carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #186

2018-03-16 Thread Apache Jenkins Server

See

carbondata git commit: [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon files

2018-03-16 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/master 04ff36764 -> e39b0a14a


[CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon files

This PR adds AvroCarbonWriter in SDK, it can be used to write JSON or Avro data 
to carbon files

This closes #2061


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/e39b0a14
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/e39b0a14
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/e39b0a14

Branch: refs/heads/master
Commit: e39b0a14a196224dbe6fdce2ff53e09b3b76b876
Parents: 04ff367
Author: Jacky Li 
Authored: Wed Mar 14 00:06:37 2018 +0800
Committer: Jacky Li 
Committed: Fri Mar 16 23:02:37 2018 +0800

--
 pom.xml |   6 -
 store/sdk/pom.xml   |  95 +--
 .../carbondata/sdk/file/AvroCarbonWriter.java   | 125 +
 .../sdk/file/CarbonWriterBuilder.java   |  29 +-
 .../sdk/file/AvroCarbonWriterTest.java  | 104 
 .../sdk/file/CSVCarbonWriterSuite.java  | 267 ---
 .../sdk/file/CSVCarbonWriterTest.java   | 267 +++
 7 files changed, 519 insertions(+), 374 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/e39b0a14/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 972be1e..287e052 100644
--- a/pom.xml
+++ b/pom.xml
@@ -579,12 +579,6 @@
   include-all
 
 
-  store-sdk
-  
-store/sdk
-  
-
-
   sdvtest
   
 integration/spark-common-cluster-test

http://git-wip-us.apache.org/repos/asf/carbondata/blob/e39b0a14/store/sdk/pom.xml
--
diff --git a/store/sdk/pom.xml b/store/sdk/pom.xml
index 9f7038a..1d1735e 100644
--- a/store/sdk/pom.xml
+++ b/store/sdk/pom.xml
@@ -25,55 +25,27 @@
   ${project.version}
 
 
-  junit
-  junit
+  tech.allegro.schema.json2avro
+  converter
+  0.2.5
   test
 
 
-  org.scalatest
-  scalatest_${scala.binary.version}
+  junit
+  junit
   test
 
   
 
   
+src/main/java
 
   
-src/resources
-  
-  
 .
   
 
 
   
-org.scala-tools
-maven-scala-plugin
-2.15.2
-
-  
-compile
-
-  compile
-
-compile
-  
-  
-testCompile
-
-  testCompile
-
-test
-  
-  
-process-resources
-
-  compile
-
-  
-
-  
-  
 maven-compiler-plugin
 
   1.7
@@ -82,61 +54,6 @@
   
   
 org.apache.maven.plugins
-maven-surefire-plugin
-2.18
-
-
-  
${project.build.directory}/surefire-reports
-  -Xmx3g -XX:MaxPermSize=512m 
-XX:ReservedCodeCacheSize=512m
-  
-true
-  
-  false
-
-  
-  
-org.scalatest
-scalatest-maven-plugin
-1.0
-
-
-  
${project.build.directory}/surefire-reports
-  .
-  CarbonTestSuite.txt
-   ${argLine} -ea -Xmx3g -XX:MaxPermSize=512m 
-XX:ReservedCodeCacheSize=512m
-  
-  
-  
-  
-  
-true
-  
-
-
-  
-test
-
-  test
-
-  
-
-  
-  
-org.apache.maven.plugins
-maven-deploy-plugin
-
-  true
-
-  
-  
-org.apache.maven.plugins
-maven-install-plugin
-
-  true
-
-  
-  
-org.apache.maven.plugins
 maven-shade-plugin
 
   false

http://git-wip-us.apache.org/repos/asf/carbondata/blob/e39b0a14/store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java
--
diff --git 
a/store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java 
b/store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java
new file mode 100644
index 000..e88164c
--- /dev/null
+++ 
b/store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright

Build failed in Jenkins: carbondata-master-spark-2.1 #2148

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [HOTFIX] Fix CI random failure

--
[...truncated 4.39 MB...]
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded 
jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the 
shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded 
jar.
[INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.2 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.6.2.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including xml-apis:xml-apis:jar:1.3.04 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded 
jar.
[INFO] Including org.scala-lang:scala-library:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-reflect:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-xml_2.11:jar:1.0.2 in the shaded 
jar.
[WARNING] jersey-core-1.9.jar, javax.ws.rs-api-2.0.1.jar define 55 overlapping 
classes: 
[WARNING]   - javax.ws.rs.core.HttpHeaders
[WARNING]   - javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate
[WARNING]   - javax.ws.rs.DefaultValue
[WARNING]   - javax.ws.rs.core.StreamingOutput
[WARNING]   - javax.ws.rs.HEAD
[WARNING]   - javax.ws.rs.core.Request
[WARNING]   - javax.ws.rs.ext.Providers
[WARNING]   - javax.ws.rs.core.NewCookie
[WARNING]   - javax.ws.rs.core.UriBuilderException
[WARNING]   - javax.ws.rs.ext.ContextResolver
[WARNING]   - 45 more...
[WARNING] commons-logging-1.1.3.jar, jcl-over-slf4j-1.7.16.jar define 6 
overlapping classes: 
[WARNING]   - org.apache.commons.logging.impl.SimpleLog$1
[WARNING]   - org.apache.commons.logging.Log
[WARNING]   - org.apache.commons.logging.impl.SimpleLog
[WARNING]   - org.apache.commons.logging.LogConfigurationException
[WARNING]   - org.apache.commons.logging.impl.NoOpLog
[WARNING]   - org.apache.commons.logging.LogFactory
[WARNING] aopalliance-1.0.jar, aopalliance-repackaged-2.4.0-b34.jar define 9 
overlapping classes: 
[WARNING]   - org.aopalliance.intercept.ConstructorInterceptor
[WARNING]   - org.aopalliance.intercept.MethodInvocation
[WARNING]   - org.aopalliance.intercept.MethodInterceptor
[WARNING]   - org.aopalliance.intercept.Invocation
[WARNING]   - org.aopalliance.aop.AspectException
[WARNING]   - org.aopalliance.intercept.Interceptor
[WARNING]   - org.aopalliance.intercept.Joinpoint
[WARNING]   - org.aopalliance.aop.Advice
[WARNING]   - org.aopalliance.intercept.ConstructorInvocation
[WARNING] commons-beanutils-1.7.0.jar, commons-beanutils-core-1.8.0.jar, 
commons-collections-3.2.2.jar define 10 overlapping classes: 
[WARNING]   - org.apache.commons.collections.FastHashMap$EntrySet
[WARNING]   - org.apache.commons.collections.FastHashMap$KeySet
[WARNING]   - org.apache.commons.collections.ArrayStack
[WARNING]   - 
org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
[WARNING]   - org.apache.commons.collections.FastHashMap$Values
[WARNING]   - org.apache.commons.collections.FastHashMap$CollectionView
[WARNING]   - org.apache.commons.collections.FastHashMap$1
[WARNING]   - org.apache.commons.collections.Buffer
[WARNING]   - org.apache.commons.collections.FastHashMap
[WARNING]   - org.apache.commons.collections.BufferUnderflowException
[WARNING] spark-launcher_2.11-2.1.0.jar, spark-sketch_2.11-2.1.0.jar, 
spark-network-shuffle_2.11-2.1.0.jar, spark-unsafe_2.11-2.1.0.jar, 
spark-core_2.11-2.1.0.jar, spark-network-common_2.11-2.1.0.jar, 
spark-sql_2.11-2.1.0.jar, spark-catalyst_2.11-2.1.0.jar, 
spark-tags_2.11-2.1.0.jar, unused-1.0.0.jar define 1 overlapping classes: 
[WARNING]   - org.apache.spark.unused.UnusedStubClass
[WARNING] javax.inject-1.jar, javax.inject-2.4.0-b34.jar define 6 overlapping 
classes: 
[WARNING]   - javax.inject.Inject
[WARNING]   - javax.inject.Singleton
[WARNING]   - javax.inject.Named
[WARNING]   - javax.inject.Scope
[WARNING]   - javax.inject.Provider
[WARNING]   - javax.inject.Qualifier
[WARNING] hadoop-yarn-client-2.2.0.jar, hadoop-yarn-common-2.7.2.jar define 1 
overlapping classes: 
[WARNING]   - org.apache.hadoop.yarn.client.ClientRMProxy
[WARNING] jersey-server-2.22.2.jar, jersey-server-1.9.jar define 18 overlapping 
classes: 
[WARNING]   - com.sun.research.ws.wadl.Representation
[WARNING]   - com.sun.research.ws.wadl.Method
[WARNING]   - com.sun.research.ws.wadl.Request
[WARNING]   -

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2148

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 9.60 KB...]
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.xbean:xbean-asm5-shaded:jar:4.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 in the shaded 
jar.
[INFO] Including com.google.inject:guice:jar:3.0 in the shaded jar.
[INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.1.0 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.1.0 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-common:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.2 in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2 in the shaded 
jar.
[INFO] Including org.glassfish.hk2:osgi-resource-locator:jar:1.0.1 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-server:jar:2.22.2 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.validation:validation-api:jar:1.1.0.Final in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 in the 
shaded jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 in the 
shaded jar.
[INFO] Including io.netty:netty-all:jar:4.0.42.Final in the shaded jar.
[INFO] Including com.clearspring.analytics:stream:jar:2.7.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-jvm:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-json:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-graphite:jar:3.1.2 in the shaded 
jar.
[INFO] Including

carbondata git commit: [HOTFIX] Fix CI random failure

2018-03-16 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/master a386f1f4e -> 04ff36764


[HOTFIX] Fix CI random failure

This closes #2068


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/04ff3676
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/04ff3676
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/04ff3676

Branch: refs/heads/master
Commit: 04ff36764c797264f5396fa3cbf1d6fe883737e0
Parents: a386f1f
Author: Jacky Li 
Authored: Thu Mar 15 19:49:07 2018 +0800
Committer: Jacky Li 
Committed: Fri Mar 16 21:12:25 2018 +0800

--
 .../statusmanager/SegmentStatusManager.java | 66 
 .../carbondata/spark/util/CommonUtil.scala  |  2 +-
 .../preaaggregate/PreAggregateTableHelper.scala | 13 +++-
 3 files changed, 53 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/04ff3676/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
 
b/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
index 820a5a4..f466018 100755
--- 
a/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
@@ -30,7 +30,6 @@ import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 
-import org.apache.carbondata.common.exceptions.TableStatusLockException;
 import org.apache.carbondata.common.logging.LogService;
 import org.apache.carbondata.common.logging.LogServiceFactory;
 import org.apache.carbondata.core.constants.CarbonCommonConstants;
@@ -834,10 +833,30 @@ public class SegmentStatusManager {
 }
   }
 
-  public static void deleteLoadsAndUpdateMetadata(
+  private static class ReturnTuple {
+LoadMetadataDetails[] details;
+boolean isUpdateRequired;
+ReturnTuple(LoadMetadataDetails[] details, boolean isUpdateRequired) {
+  this.details = details;
+  this.isUpdateRequired = isUpdateRequired;
+}
+  }
+
+  private static ReturnTuple isUpdationRequired(
+  boolean isForceDeletion,
   CarbonTable carbonTable,
-  boolean isForceDeletion) throws IOException {
-deleteLoadsAndUpdateMetadata(carbonTable, isForceDeletion, null);
+  AbsoluteTableIdentifier absoluteTableIdentifier) {
+LoadMetadataDetails[] details =
+SegmentStatusManager.readLoadMetadata(carbonTable.getMetadataPath());
+// Delete marked loads
+boolean isUpdationRequired =
+DeleteLoadFolders.deleteLoadFoldersFromFileSystem(
+absoluteTableIdentifier,
+isForceDeletion,
+details,
+carbonTable.getMetadataPath()
+);
+return new ReturnTuple(details, isUpdationRequired);
   }
 
   public static void deleteLoadsAndUpdateMetadata(
@@ -845,31 +864,29 @@ public class SegmentStatusManager {
   boolean isForceDeletion,
   List partitionSpecs) throws IOException {
 if (isLoadDeletionRequired(carbonTable.getMetadataPath())) {
-  LoadMetadataDetails[] details =
-  SegmentStatusManager.readLoadMetadata(carbonTable.getMetadataPath());
   AbsoluteTableIdentifier identifier = 
carbonTable.getAbsoluteTableIdentifier();
-  ICarbonLock carbonTableStatusLock = CarbonLockFactory.getCarbonLockObj(
-  identifier, LockUsage.TABLE_STATUS_LOCK);
-
-  // Delete marked loads
-  boolean isUpdationRequired = 
DeleteLoadFolders.deleteLoadFoldersFromFileSystem(
-  identifier, isForceDeletion, details, carbonTable.getMetadataPath());
-
-  boolean updationCompletionStatus = false;
-
-  if (isUpdationRequired) {
+  ReturnTuple tuple = isUpdationRequired(isForceDeletion, carbonTable, 
identifier);
+  if (tuple.isUpdateRequired) {
+ICarbonLock carbonTableStatusLock =
+CarbonLockFactory.getCarbonLockObj(identifier, 
LockUsage.TABLE_STATUS_LOCK);
+boolean locked = false;
 try {
   // Update load metadate file after cleaning deleted nodes
-  if (carbonTableStatusLock.lockWithRetries()) {
+  locked = carbonTableStatusLock.lockWithRetries();
+  if (locked) {
 LOG.info("Table status lock has been successfully acquired.");
-
+// Again read status and check to verify updation required or not.
+ReturnTuple tuple2 = isUpdationRequired(isForceDeletion, 
carbonTable, identifier);
+if (!tuple2.isUpdateRequired) {
+  return;
+}
 // read latest table status again.

[5/6] carbondata git commit: [CARBONDATA-2224][File Level Reader Support] External File level reader support

2018-03-16 Thread jackylk

http://git-wip-us.apache.org/repos/asf/carbondata/blob/7a124ecd/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
--
diff --git 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
index 0298eea..cf22569 100644
--- 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
+++ 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
@@ -150,7 +150,7 @@ case class CarbonDropTableCommand(
 
   // delete table data only if it is not external table
   if (FileFactory.isFileExist(tablePath, fileType) &&
-  !carbonTable.isExternalTable) {
+  !(carbonTable.isExternalTable || 
carbonTable.isFileLevelExternalTable)) {
 val file = FileFactory.getCarbonFile(tablePath, fileType)
 CarbonUtil.deleteFoldersAndFilesSilent(file)
   }

http://git-wip-us.apache.org/repos/asf/carbondata/blob/7a124ecd/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/CarbonFileFormat.scala
--
diff --git 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/CarbonFileFormat.scala
 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/CarbonFileFormat.scala
deleted file mode 100644
index 2eed988..000
--- 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/CarbonFileFormat.scala
+++ /dev/null
@@ -1,443 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.execution.datasources
-
-import java.io.File
-import java.util
-import java.util.concurrent.ConcurrentHashMap
-import java.util.concurrent.atomic.AtomicLong
-
-import scala.collection.JavaConverters._
-import scala.collection.mutable
-import scala.util.Random
-
-import org.apache.hadoop.fs.{FileStatus, Path}
-import org.apache.hadoop.io.NullWritable
-import org.apache.hadoop.mapreduce.{Job, TaskAttemptContext}
-import org.apache.spark.SparkEnv
-import org.apache.spark.internal.Logging
-import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
-import org.apache.spark.sql.catalyst.{InternalRow, TableIdentifier}
-import org.apache.spark.sql.catalyst.catalog.ExternalCatalogUtils
-import org.apache.spark.sql.internal.SQLConf
-import org.apache.spark.sql.sources.DataSourceRegister
-import org.apache.spark.sql.types._
-
-import org.apache.carbondata.core.constants.{CarbonCommonConstants, 
CarbonLoadOptionConstants}
-import org.apache.carbondata.core.datastore.impl.FileFactory
-import org.apache.carbondata.core.indexstore
-import org.apache.carbondata.core.metadata.SegmentFileStore
-import org.apache.carbondata.core.metadata.datatype.DataTypes
-import org.apache.carbondata.core.metadata.encoder.Encoding
-import org.apache.carbondata.core.statusmanager.{LoadMetadataDetails, 
SegmentStatusManager}
-import org.apache.carbondata.core.util.{CarbonProperties, 
DataTypeConverterImpl, DataTypeUtil}
-import org.apache.carbondata.core.util.path.CarbonTablePath
-import org.apache.carbondata.hadoop.api.{CarbonOutputCommitter, 
CarbonTableOutputFormat}
-import 
org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.CarbonRecordWriter
-import org.apache.carbondata.hadoop.internal.ObjectArrayWritable
-import org.apache.carbondata.hadoop.util.ObjectSerializationUtil
-import org.apache.carbondata.processing.loading.model.{CarbonLoadModel, 
CarbonLoadModelBuilder, LoadOption}
-import org.apache.carbondata.spark.util.{CarbonScalaUtil, Util}
-
-class CarbonFileFormat
-  extends FileFormat
-with DataSourceRegister
-with Logging
-with Serializable {
-
-  override def shortName(): String = "carbondata"
-
-  override def inferSchema(sparkSession: SparkSession,
-  options: Map[String, String],
-  files: Seq[FileStatus]): Option[StructType] = {
-None
-  }
-
-

[1/6] carbondata git commit: [CARBONDATA-2255] Rename the streaming examples

2018-03-16 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/carbonfile 6cb6f8380 -> 7a124ecd8


[CARBONDATA-2255] Rename the streaming examples

optimize streaming examples

This closes #2064


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/d4f9003a
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/d4f9003a
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/d4f9003a

Branch: refs/heads/carbonfile
Commit: d4f9003af3593d30799b573ce729de1918b8c800
Parents: 6cb6f83
Author: QiangCai 
Authored: Wed Mar 14 17:42:39 2018 +0800
Committer: chenliang613 
Committed: Thu Mar 15 15:33:00 2018 +0800

--
 .../CarbonBatchSparkStreamingExample.scala  | 207 --
 .../CarbonStreamSparkStreamingExample.scala | 213 --
 .../CarbonStructuredStreamingExample.scala  | 200 -
 ...CarbonStructuredStreamingWithRowParser.scala | 216 ---
 .../examples/SparkStreamingExample.scala| 213 ++
 .../StreamingUsingBatchLoadExample.scala| 207 ++
 .../StreamingWithRowParserExample.scala | 216 +++
 .../examples/StructuredStreamingExample.scala   | 200 +
 8 files changed, 836 insertions(+), 836 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/d4f9003a/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonBatchSparkStreamingExample.scala
--
diff --git 
a/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonBatchSparkStreamingExample.scala
 
b/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonBatchSparkStreamingExample.scala
deleted file mode 100644
index bcbf190..000
--- 
a/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonBatchSparkStreamingExample.scala
+++ /dev/null
@@ -1,207 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.carbondata.examples
-
-import java.io.{File, PrintWriter}
-import java.net.ServerSocket
-
-import org.apache.spark.rdd.RDD
-import org.apache.spark.sql.{CarbonEnv, SaveMode, SparkSession}
-import org.apache.spark.streaming.{Seconds, StreamingContext, Time}
-
-/**
- * This example introduces how to use CarbonData batch load to integrate
- * with Spark Streaming(it's DStream, not Spark Structured Streaming)
- */
-// scalastyle:off println
-
-case class DStreamData(id: Int, name: String, city: String, salary: Float)
-
-object CarbonBatchSparkStreamingExample {
-
-  def main(args: Array[String]): Unit = {
-
-// setup paths
-val rootPath = new File(this.getClass.getResource("/").getPath
-+ "../../../..").getCanonicalPath
-val checkpointPath =
-  s"$rootPath/examples/spark2/target/spark_streaming_cp_" +
-  System.currentTimeMillis().toString()
-val streamTableName = s"dstream_batch_table"
-
-val spark = 
ExampleUtils.createCarbonSession("CarbonBatchSparkStreamingExample", 4)
-
-val requireCreateTable = true
-
-if (requireCreateTable) {
-  // drop table if exists previously
-  spark.sql(s"DROP TABLE IF EXISTS ${ streamTableName }")
-  // Create target carbon table and populate with initial data
-  // set AUTO_LOAD_MERGE to true to compact segment automatically
-  spark.sql(
-s"""
-   | CREATE TABLE ${ streamTableName }(
-   | id INT,
-   | name STRING,
-   | city STRING,
-   | salary FLOAT
-   | )
-   | STORED BY 'carbondata'
-   | TBLPROPERTIES(
-   | 'sort_columns'='name',
-   | 'dictionary_include'='city',
-   | 'AUTO_LOAD_MERGE'='true',
-   | 'COMPACTION_LEVEL_THRESHOLD'='4,10')
-   | """.stripMargin)
-
-  val carbonTable = CarbonEnv.getCarbonTable(Some("default"), 
streamTableName)(spark)
-  // batch load
-  val path =

[6/6] carbondata git commit: [CARBONDATA-2224][File Level Reader Support] External File level reader support

2018-03-16 Thread jackylk

[CARBONDATA-2224][File Level Reader Support] External File level reader support

File level reader reads any carbondata file placed in any external file path.

This closes #2055


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/7a124ecd
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/7a124ecd
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/7a124ecd

Branch: refs/heads/carbonfile
Commit: 7a124ecd87769c0197ae67a0726c5abb4745d3a8
Parents: a386f1f
Author: sounakr 
Authored: Sat Feb 24 07:55:14 2018 +0530
Committer: Jacky Li 
Committed: Fri Mar 16 17:33:43 2018 +0800

--
 .../core/metadata/schema/table/CarbonTable.java |   6 +
 .../apache/carbondata/core/util/CarbonUtil.java | 209 +-
 .../hadoop/api/CarbonFileInputFormat.java   | 682 +++
 .../carbondata/hadoop/util/SchemaReader.java|  17 +-
 integration/spark-common-test/pom.xml   |   6 +
 ...FileInputFormatWithExternalCarbonTable.scala | 240 +++
 ...tCreateTableUsingSparkCarbonFileFormat.scala | 327 +
 ...tSparkCarbonFileFormatWithSparkSession.scala | 176 +
 .../carbondata/spark/rdd/CarbonScanRDD.scala|  64 +-
 .../VectorizedCarbonRecordReader.java   |  22 +-
 .../management/CarbonLoadDataCommand.scala  |   4 +-
 .../command/table/CarbonDropTableCommand.scala  |   2 +-
 .../datasources/CarbonFileFormat.scala  | 443 
 .../datasources/SparkCarbonFileFormat.scala | 269 
 .../datasources/SparkCarbonTableFormat.scala| 443 
 .../sql/execution/strategy/DDLStrategy.scala|  27 +-
 .../spark/sql/hive/CarbonAnalysisRules.scala|   2 +-
 .../spark/sql/parser/CarbonSparkSqlParser.scala |  47 +-
 .../spark/sql/hive/CarbonSessionState.scala |  17 +-
 .../spark/sql/hive/CarbonSessionState.scala |  16 +-
 apache.spark.sql.sources.DataSourceRegister |   3 +-
 .../sdk/file/CSVCarbonWriterSuite.java  |   6 +-
 22 files changed, 2522 insertions(+), 506 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/7a124ecd/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
 
b/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
index f14672f..278dc96 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
@@ -826,6 +826,12 @@ public class CarbonTable implements Serializable {
 return external != null && external.equalsIgnoreCase("true");
   }
 
+  public boolean isFileLevelExternalTable() {
+String external = 
tableInfo.getFactTable().getTableProperties().get("_filelevelexternal");
+return external != null && external.equalsIgnoreCase("true");
+  }
+
+
   public long size() throws IOException {
 Map dataIndexSize = CarbonUtil.calculateDataIndexSize(this);
 Long dataSize = 
dataIndexSize.get(CarbonCommonConstants.CARBON_TOTAL_DATA_SIZE);

http://git-wip-us.apache.org/repos/asf/carbondata/blob/7a124ecd/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
--
diff --git a/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java 
b/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
index b961b60..ff49edf 100644
--- a/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
+++ b/core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
@@ -41,6 +41,7 @@ import 
org.apache.carbondata.core.datastore.columnar.ColumnGroupModel;
 import org.apache.carbondata.core.datastore.columnar.UnBlockIndexer;
 import 
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException;
 import org.apache.carbondata.core.datastore.filesystem.CarbonFile;
+import org.apache.carbondata.core.datastore.filesystem.CarbonFileFilter;
 import org.apache.carbondata.core.datastore.impl.FileFactory;
 import org.apache.carbondata.core.exception.InvalidConfigurationException;
 import org.apache.carbondata.core.indexstore.BlockletDetailInfo;
@@ -52,18 +53,26 @@ import org.apache.carbondata.core.metadata.SegmentFileStore;
 import org.apache.carbondata.core.metadata.ValueEncoderMeta;
 import org.apache.carbondata.core.metadata.blocklet.DataFileFooter;
 import org.apache.carbondata.core.metadata.blocklet.SegmentInfo;
+import 
org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl;
 import org.apache.carbondata.core.metadata.datatype.DataType;

[3/6] carbondata git commit: [CARBONDATA-2250][DataLoad] Reduce massive object generation in global sort

2018-03-16 Thread jackylk

[CARBONDATA-2250][DataLoad] Reduce massive object generation in global sort

Generate compatator outside the function, otherwise it will be generated for 
every row

This closes #2059


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/31011fc2
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/31011fc2
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/31011fc2

Branch: refs/heads/carbonfile
Commit: 31011fc29f85d949bdabaa3551b7513eb1183ee5
Parents: 5b48e70
Author: xuchuanyin 
Authored: Wed Mar 14 14:45:53 2018 +0800
Committer: Jacky Li 
Committed: Thu Mar 15 17:43:39 2018 +0800

--
 .../spark/load/DataLoadProcessBuilderOnSpark.scala | 13 ++---
 .../processing/sort/sortdata/NewRowComparator.java |  4 +++-
 .../sort/sortdata/NewRowComparatorForNormalDims.java   |  5 -
 3 files changed, 13 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/31011fc2/integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
--
diff --git 
a/integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
 
b/integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
index 1062cd7..dc238fb 100644
--- 
a/integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
+++ 
b/integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
@@ -80,15 +80,14 @@ object DataLoadProcessBuilderOnSpark {
 // 3. Sort
 val configuration = DataLoadProcessBuilder.createConfiguration(model)
 val sortParameters = SortParameters.createSortParameters(configuration)
+val rowComparator: Comparator[Array[AnyRef]] =
+  if (sortParameters.getNoDictionaryCount > 0) {
+new NewRowComparator(sortParameters.getNoDictionaryDimnesionColumn)
+  } else {
+new NewRowComparatorForNormalDims(sortParameters.getDimColCount)
+  }
 object RowOrdering extends Ordering[Array[AnyRef]] {
   def compare(rowA: Array[AnyRef], rowB: Array[AnyRef]): Int = {
-val rowComparator: Comparator[Array[AnyRef]] =
-  if (sortParameters.getNoDictionaryCount > 0) {
-new NewRowComparator(sortParameters.getNoDictionaryDimnesionColumn)
-  } else {
-new NewRowComparatorForNormalDims(sortParameters.getDimColCount)
-  }
-
 rowComparator.compare(rowA, rowB)
   }
 }

http://git-wip-us.apache.org/repos/asf/carbondata/blob/31011fc2/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparator.java
--
diff --git 
a/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparator.java
 
b/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparator.java
index 3f94533..f47ecc7 100644
--- 
a/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparator.java
+++ 
b/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparator.java
@@ -17,11 +17,13 @@
 
 package org.apache.carbondata.processing.sort.sortdata;
 
+import java.io.Serializable;
 import java.util.Comparator;
 
 import org.apache.carbondata.core.util.ByteUtil.UnsafeComparer;
 
-public class NewRowComparator implements Comparator {
+public class NewRowComparator implements Comparator, Serializable {
+  private static final long serialVersionUID = -173987462709436L;
 
   /**
* mapping of dictionary dimensions and no dictionary of sort_column.

http://git-wip-us.apache.org/repos/asf/carbondata/blob/31011fc2/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparatorForNormalDims.java
--
diff --git 
a/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparatorForNormalDims.java
 
b/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparatorForNormalDims.java
index 7538c92..aea83ba 100644
--- 
a/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparatorForNormalDims.java
+++ 
b/processing/src/main/java/org/apache/carbondata/processing/sort/sortdata/NewRowComparatorForNormalDims.java
@@ -16,13 +16,16 @@
  */
 package org.apache.carbondata.processing.sort.sortdata;
 
+import java.io.Serializable;
 import java.util.Comparator;
 
 /**
  * This class is used as comparator for comparing dims

[4/6] carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

2018-03-16 Thread jackylk

[CARBONDATA-2244]fix creating pre-aggregate table bug when there are 
invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main 
table

When there are some invisibility 
INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table, it can 
not create preaggregate table on it.

This closes #2050


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/a386f1f4
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/a386f1f4
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/a386f1f4

Branch: refs/heads/carbonfile
Commit: a386f1f4e220210e031f10a73c26a7f56f57a603
Parents: 31011fc
Author: Zhang Zhichao <441586...@qq.com>
Authored: Fri Mar 9 23:36:08 2018 +0800
Committer: Jacky Li 
Committed: Fri Mar 16 14:43:06 2018 +0800

--
 .../command/preaaggregate/PreAggregateTableHelper.scala  | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/a386f1f4/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
--
diff --git 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
index 1f1e1e6..b64c91e 100644
--- 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
+++ 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
@@ -165,12 +165,12 @@ case class PreAggregateTableHelper(
 // need to fire load for pre-aggregate table. Therefore reading the load 
details for PARENT
 // table.
 SegmentStatusManager.deleteLoadsAndUpdateMetadata(parentTable, false)
-val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetadataPath)
-if (loadAvailable.exists(load => load.getSegmentStatus == 
SegmentStatus.INSERT_IN_PROGRESS ||
-  load.getSegmentStatus == SegmentStatus.INSERT_OVERWRITE_IN_PROGRESS)) {
+if (SegmentStatusManager.isLoadInProgressInTable(parentTable)) {
   throw new UnsupportedOperationException(
 "Cannot create pre-aggregate table when insert is in progress on main 
table")
-} else if (loadAvailable.nonEmpty) {
+}
+val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetadataPath)
+if (loadAvailable.nonEmpty) {
   // Passing segmentToLoad as * because we want to load all the segments 
into the
   // pre-aggregate table even if the user has set some segments on the 
parent table.
   loadCommand.dataFrame = Some(PreAggregateUtil

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2146

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 9.60 KB...]
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.xbean:xbean-asm5-shaded:jar:4.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 in the shaded 
jar.
[INFO] Including com.google.inject:guice:jar:3.0 in the shaded jar.
[INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.1.0 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.1.0 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-common:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.2 in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2 in the shaded 
jar.
[INFO] Including org.glassfish.hk2:osgi-resource-locator:jar:1.0.1 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-server:jar:2.22.2 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.validation:validation-api:jar:1.1.0.Final in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 in the 
shaded jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 in the 
shaded jar.
[INFO] Including io.netty:netty-all:jar:4.0.42.Final in the shaded jar.
[INFO] Including com.clearspring.analytics:stream:jar:2.7.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-jvm:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-json:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-graphite:jar:3.1.2 in the shaded 
jar.
[INFO] Including

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2147

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 9.60 KB...]
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.xbean:xbean-asm5-shaded:jar:4.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.2.0 in the shaded 
jar.
[INFO] Including com.google.inject:guice:jar:3.0 in the shaded jar.
[INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.2.0 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.2.0 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.2.0 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.1.0 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.1.0 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.18.1-GA in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-common:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.2 in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2 in the shaded 
jar.
[INFO] Including org.glassfish.hk2:osgi-resource-locator:jar:1.0.1 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-server:jar:2.22.2 in the 
shaded jar.
[INFO] Including org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.validation:validation-api:jar:1.1.0.Final in the shaded 
jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 in the 
shaded jar.
[INFO] Including 
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 in the 
shaded jar.
[INFO] Including io.netty:netty-all:jar:4.0.42.Final in the shaded jar.
[INFO] Including com.clearspring.analytics:stream:jar:2.7.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-jvm:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-json:jar:3.1.2 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-graphite:jar:3.1.2 in the shaded 
jar.
[INFO] Including

Build failed in Jenkins: carbondata-master-spark-2.2 #184

2018-03-16 Thread Apache Jenkins Server

See 


Changes:

[jacky.likun] [CARBONDATA-2244]fix creating pre-aggregate table bug when there 
are

--
[...truncated 4.38 MB...]
[INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including commons-configuration:commons-configuration:jar:1.6 in the 
shaded jar.
[INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar.
[INFO] Including commons-beanutils:commons-beanutils:jar:1.7.0 in the shaded 
jar.
[INFO] Including commons-beanutils:commons-beanutils-core:jar:1.8.0 in the 
shaded jar.
[INFO] Including org.slf4j:slf4j-api:jar:1.7.10 in the shaded jar.
[INFO] Including org.slf4j:slf4j-log4j12:jar:1.7.10 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded 
jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the 
shaded jar.
[INFO] Including org.apache.avro:avro:jar:1.7.4 in the shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.2 in the shaded jar.
[INFO] Including 
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 in the shaded 
jar.
[INFO] Including org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-asn1-api:jar:1.0.0-M20 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-util:jar:1.0.0-M20 in the shaded 
jar.
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded 
jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the 
shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded 
jar.
[INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.2 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.6.2.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including xml-apis:xml-apis:jar:1.3.04 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded 
jar.
[INFO] Including org.scala-lang:scala-library:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-reflect:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-xml_2.11:jar:1.0.2 in the shaded 
jar.
[WARNING] commons-logging-1.1.3.jar, jcl-over-slf4j-1.7.16.jar define 6 
overlapping classes: 
[WARNING]   - org.apache.commons.logging.impl.SimpleLog$1
[WARNING]   - org.apache.commons.logging.Log
[WARNING]   - org.apache.commons.logging.impl.SimpleLog
[WARNING]   - org.apache.commons.logging.LogConfigurationException
[WARNING]   - org.apache.commons.logging.impl.NoOpLog
[WARNING]   - org.apache.commons.logging.LogFactory
[WARNING] commons-collections-3.2.2.jar, commons-beanutils-core-1.8.0.jar, 
commons-beanutils-1.7.0.jar define 10 overlapping classes: 
[WARNING]   - org.apache.commons.collections.FastHashMap$EntrySet
[WARNING]   - org.apache.commons.collections.FastHashMap$KeySet
[WARNING]   - org.apache.commons.collections.ArrayStack
[WARNING]   - 
org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
[WARNING]   - org.apache.commons.collections.FastHashMap$Values
[WARNING]   - org.apache.commons.collections.FastHashMap$CollectionView
[WARNING]   - org.apache.commons.collections.FastHashMap$1
[WARNING]   - org.apache.commons.collections.Buffer
[WARNING]   - org.apache.commons.collections.FastHashMap
[WARNING]   - org.apache.commons.collections.BufferUnderflowException
[WARNING] jersey-core-1.9.jar, javax.ws.rs-api-2.0.1.jar define 55 overlapping 
classes: 
[WARNING]   - javax.ws.rs.core.HttpHeaders
[WARNING]   - javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate
[WARNING]   - javax.ws.rs.DefaultValue
[WARNING]   - javax.ws.rs.core.StreamingOutput
[WARNING]   - javax.ws.rs.HEAD
[WARNING]   - javax.ws.rs.core.Request
[WARNING]   - javax.ws.rs.ext.Providers
[WARNING]   - javax.ws.rs.core.NewCookie
[WARNING]   - javax.ws.rs.core.UriBuilderException
[WARNING]   - javax.ws.rs.ext.ContextResolver
[WARNING]   - 45 more...
[WARNING] spark-network-common_2.11-2.2.1.jar, spark-launcher_2.11-2.2.1.jar, 
spark-network-shuffle_2.11-2.2.1.jar, spark-sql_2.11-2.2.1.jar, 
spark-catalyst_2.11-2.2.1.jar,

Jenkins build became unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #184

2018-03-16 Thread Apache Jenkins Server

See

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #184

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 7.44 KB...]
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-store-sdk ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-store-sdk ---
[INFO] 
[INFO] --- maven-shade-plugin:2.4.3:shade (default) @ carbondata-store-sdk ---
[INFO] Including org.apache.carbondata:carbondata-hadoop:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-processing:jar:1.4.0-SNAPSHOT 
in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-core:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-format:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.thrift:libthrift:jar:0.9.3 in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-common:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.3.1 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.2.6 in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.7 in the shaded jar.
[INFO] Including org.apache.spark:spark-sql_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-sketch_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-core_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 in the shaded 
jar.
[INFO] Including org.apache.avro:avro-ipc:jar:1.7.7 in the shaded jar.
[INFO] Including org.apache.avro:avro-ipc:jar:tests:1.7.7 in the shaded jar.
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.6.5 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.5 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.2.1 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded

Jenkins build is unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #185

2018-03-16 Thread Apache Jenkins Server

See

Build failed in Jenkins: carbondata-master-spark-2.2 #185

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 4.38 MB...]
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded 
jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the 
shaded jar.
[INFO] Including org.apache.avro:avro:jar:1.7.4 in the shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.2 in the shaded jar.
[INFO] Including 
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15 in the shaded 
jar.
[INFO] Including org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-asn1-api:jar:1.0.0-M20 in the 
shaded jar.
[INFO] Including org.apache.directory.api:api-util:jar:1.0.0-M20 in the shaded 
jar.
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded 
jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the 
shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded 
jar.
[INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.2 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.6.2.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including xml-apis:xml-apis:jar:1.3.04 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded 
jar.
[INFO] Including org.scala-lang:scala-library:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-reflect:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-xml_2.11:jar:1.0.2 in the shaded 
jar.
[WARNING] jersey-core-1.9.jar, javax.ws.rs-api-2.0.1.jar define 55 overlapping 
classes: 
[WARNING]   - javax.ws.rs.core.HttpHeaders
[WARNING]   - javax.ws.rs.ext.RuntimeDelegate$HeaderDelegate
[WARNING]   - javax.ws.rs.DefaultValue
[WARNING]   - javax.ws.rs.core.StreamingOutput
[WARNING]   - javax.ws.rs.HEAD
[WARNING]   - javax.ws.rs.core.Request
[WARNING]   - javax.ws.rs.ext.Providers
[WARNING]   - javax.ws.rs.core.NewCookie
[WARNING]   - javax.ws.rs.core.UriBuilderException
[WARNING]   - javax.ws.rs.ext.ContextResolver
[WARNING]   - 45 more...
[WARNING] commons-logging-1.1.3.jar, jcl-over-slf4j-1.7.16.jar define 6 
overlapping classes: 
[WARNING]   - org.apache.commons.logging.impl.SimpleLog$1
[WARNING]   - org.apache.commons.logging.Log
[WARNING]   - org.apache.commons.logging.impl.SimpleLog
[WARNING]   - org.apache.commons.logging.LogConfigurationException
[WARNING]   - org.apache.commons.logging.impl.NoOpLog
[WARNING]   - org.apache.commons.logging.LogFactory
[WARNING] spark-launcher_2.11-2.2.1.jar, spark-tags_2.11-2.2.1.jar, 
spark-network-common_2.11-2.2.1.jar, spark-catalyst_2.11-2.2.1.jar, 
spark-sketch_2.11-2.2.1.jar, spark-sql_2.11-2.2.1.jar, 
spark-unsafe_2.11-2.2.1.jar, spark-core_2.11-2.2.1.jar, 
spark-network-shuffle_2.11-2.2.1.jar, unused-1.0.0.jar define 1 overlapping 
classes: 
[WARNING]   - org.apache.spark.unused.UnusedStubClass
[WARNING] commons-beanutils-1.7.0.jar, commons-beanutils-core-1.8.0.jar, 
commons-collections-3.2.2.jar define 10 overlapping classes: 
[WARNING]   - org.apache.commons.collections.FastHashMap$EntrySet
[WARNING]   - org.apache.commons.collections.FastHashMap$KeySet
[WARNING]   - org.apache.commons.collections.ArrayStack
[WARNING]   - 
org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator
[WARNING]   - org.apache.commons.collections.FastHashMap$Values
[WARNING]   - org.apache.commons.collections.FastHashMap$CollectionView
[WARNING]   - org.apache.commons.collections.FastHashMap$1
[WARNING]   - org.apache.commons.collections.Buffer
[WARNING]   - org.apache.commons.collections.FastHashMap
[WARNING]   - org.apache.commons.collections.BufferUnderflowException
[WARNING] hadoop-yarn-common-2.7.2.jar, hadoop-yarn-api-2.6.5.jar define 1 
overlapping classes: 
[WARNING]   - org.apache.hadoop.yarn.util.package-info
[WARNING] jersey-server-2.22.2.jar, jersey-server-1.9.jar define 18 overlapping 
classes: 
[WARNING]   - com.sun.research.ws.wadl.Representation
[WARNING]   - com.sun.research.ws.wadl.Method
[WARNING]   - com.sun.research.ws.wadl.Request
[WARNING]   - com.sun.research.ws.wadl.package-info
[WARNING]   - com.sun.research.ws.wadl.ResourceType
[WARNING]   - com.sun.research.ws.wadl.Param

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #185

2018-03-16 Thread Apache Jenkins Server

See 


--
[...truncated 7.44 KB...]
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-store-sdk ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-store-sdk ---
[INFO] 
[INFO] --- maven-shade-plugin:2.4.3:shade (default) @ carbondata-store-sdk ---
[INFO] Including org.apache.carbondata:carbondata-hadoop:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-processing:jar:1.4.0-SNAPSHOT 
in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-core:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-format:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including org.apache.thrift:libthrift:jar:0.9.3 in the shaded jar.
[INFO] Including org.apache.carbondata:carbondata-common:jar:1.4.0-SNAPSHOT in 
the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.3.1 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.2.6 in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.7 in the shaded jar.
[INFO] Including org.apache.spark:spark-sql_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-sketch_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.spark:spark-core_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.avro:avro-mapred:jar:hadoop2:1.7.7 in the shaded 
jar.
[INFO] Including org.apache.avro:avro-ipc:jar:1.7.7 in the shaded jar.
[INFO] Including org.apache.avro:avro-ipc:jar:tests:1.7.7 in the shaded jar.
[INFO] Including com.twitter:chill_2.11:jar:0.8.0 in the shaded jar.
[INFO] Including com.esotericsoftware:kryo-shaded:jar:3.0.3 in the shaded jar.
[INFO] Including com.esotericsoftware:minlog:jar:1.3.0 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:2.1 in the shaded jar.
[INFO] Including com.twitter:chill-java:jar:0.8.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.6.5 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.6.5 in the 
shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.5 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.6.5 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.2 in 
the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.2 in the shaded 
jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.5 
in the shaded jar.
[INFO] Including org.apache.spark:spark-launcher_2.11:jar:2.2.1 in the shaded 
jar.
[INFO] Including org.apache.spark:spark-network-common_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-network-shuffle_2.11:jar:2.2.1 in the 
shaded jar.
[INFO] Including org.apache.spark:spark-unsafe_2.11:jar:2.2.1 in the shaded jar.
[INFO] Including javax.servlet:javax.servlet-api:jar:3.1.0 in the shaded jar.
[INFO] Including org.slf4j:jul-to-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including org.slf4j:jcl-over-slf4j:jar:1.7.16 in the shaded jar.
[INFO] Including com.ning:compress-lzf:jar:1.0.3 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.3.0 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.5.11 in the shaded jar.
[INFO] Including org.json4s:json4s-jackson_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-core_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.json4s:json4s-ast_2.11:jar:3.2.11 in the shaded jar.
[INFO] Including org.scala-lang:scalap:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-compiler:jar:2.11.8 in the shaded jar.
[INFO] Including org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4 
in the shaded jar.
[INFO] Including org.glassfish.jersey.core:jersey-client:jar:2.22.2 in the 
shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.0.1 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-api:jar:2.4.0-b34 in the shaded jar.
[INFO] Including org.glassfish.hk2:hk2-utils:jar:2.4.0-b34 in the shaded jar.
[INFO] Including 
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34 in the shaded 
jar.
[INFO] Including org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34 in the 
shaded jar.
[INFO] Including org.glassfish.hk2:hk2-locator:jar:2.4.0-b34 in the shaded

carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

2018-03-16 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/branch-1.3 0283c938b -> 96e26d7a5


[CARBONDATA-2244]fix creating pre-aggregate table bug when there are 
invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main 
table

When there are some invisibility 
INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table, it can 
not create preaggregate table on it.

This closes #2050


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/96e26d7a
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/96e26d7a
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/96e26d7a

Branch: refs/heads/branch-1.3
Commit: 96e26d7a5da347c3638827f5986aebc7b1eaa237
Parents: 0283c93
Author: Zhang Zhichao <441586...@qq.com>
Authored: Fri Mar 9 23:36:08 2018 +0800
Committer: Jacky Li 
Committed: Fri Mar 16 14:47:22 2018 +0800

--
 .../CreatePreAggregateTableCommand.scala| 23 
 1 file changed, 14 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/96e26d7a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/CreatePreAggregateTableCommand.scala
--
diff --git 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/CreatePreAggregateTableCommand.scala
 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/CreatePreAggregateTableCommand.scala
index 59c43aa..4270d47 100644
--- 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/CreatePreAggregateTableCommand.scala
+++ 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/CreatePreAggregateTableCommand.scala
@@ -179,17 +179,22 @@ case class CreatePreAggregateTableCommand(
 // This will be used to check if the parent table has any segments or not. 
If not then no
 // need to fire load for pre-aggregate table. Therefore reading the load 
details for PARENT
 // table.
-DataLoadingUtil.deleteLoadsAndUpdateMetadata(isForceDeletion = false,
+DataLoadingUtil.deleteLoadsAndUpdateMetadata(
+  isForceDeletion = false,
   parentTable,
-  CarbonFilters.getCurrentPartitions(sparkSession,
-  TableIdentifier(parentTable.getTableName,
-Some(parentTable.getDatabaseName))).map(_.asJava).orNull)
-val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetaDataFilepath)
-if (loadAvailable.exists(load => load.getSegmentStatus == 
SegmentStatus.INSERT_IN_PROGRESS ||
-  load.getSegmentStatus == SegmentStatus.INSERT_OVERWRITE_IN_PROGRESS)) {
+  CarbonFilters.getCurrentPartitions(
+sparkSession,
+TableIdentifier(parentTable.getTableName,
+Some(parentTable.getDatabaseName))
+  ).map(_.asJava).orNull
+)
+
+if (SegmentStatusManager.isLoadInProgressInTable(parentTable)) {
   throw new UnsupportedOperationException(
-"Cannot create pre-aggregate table when insert is in progress on main 
table")
-} else if (loadAvailable.nonEmpty) {
+  "Cannot create pre-aggregate table when insert is in progress on main 
table")
+}
+val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetaDataFilepath)
+if (loadAvailable.nonEmpty) {
   // Passing segmentToLoad as * because we want to load all the segments 
into the
   // pre-aggregate table even if the user has set some segments on the 
parent table.
   loadCommand.dataFrame = Some(PreAggregateUtil

carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

2018-03-16 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/master 31011fc29 -> a386f1f4e


[CARBONDATA-2244]fix creating pre-aggregate table bug when there are 
invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main 
table

When there are some invisibility 
INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table, it can 
not create preaggregate table on it.

This closes #2050


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/a386f1f4
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/a386f1f4
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/a386f1f4

Branch: refs/heads/master
Commit: a386f1f4e220210e031f10a73c26a7f56f57a603
Parents: 31011fc
Author: Zhang Zhichao <441586...@qq.com>
Authored: Fri Mar 9 23:36:08 2018 +0800
Committer: Jacky Li 
Committed: Fri Mar 16 14:43:06 2018 +0800

--
 .../command/preaaggregate/PreAggregateTableHelper.scala  | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/a386f1f4/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
--
diff --git 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
index 1f1e1e6..b64c91e 100644
--- 
a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
+++ 
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
@@ -165,12 +165,12 @@ case class PreAggregateTableHelper(
 // need to fire load for pre-aggregate table. Therefore reading the load 
details for PARENT
 // table.
 SegmentStatusManager.deleteLoadsAndUpdateMetadata(parentTable, false)
-val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetadataPath)
-if (loadAvailable.exists(load => load.getSegmentStatus == 
SegmentStatus.INSERT_IN_PROGRESS ||
-  load.getSegmentStatus == SegmentStatus.INSERT_OVERWRITE_IN_PROGRESS)) {
+if (SegmentStatusManager.isLoadInProgressInTable(parentTable)) {
   throw new UnsupportedOperationException(
 "Cannot create pre-aggregate table when insert is in progress on main 
table")
-} else if (loadAvailable.nonEmpty) {
+}
+val loadAvailable = 
SegmentStatusManager.readLoadMetadata(parentTable.getMetadataPath)
+if (loadAvailable.nonEmpty) {
   // Passing segmentToLoad as * because we want to load all the segments 
into the
   // pre-aggregate table even if the user has set some segments on the 
parent table.
   loadCommand.dataFrame = Some(PreAggregateUtil

Build failed in Jenkins: carbondata-master-spark-2.1 #2149

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2149

Build failed in Jenkins: carbondata-master-spark-2.2 #187

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #187

Jenkins build is back to normal : carbondata-master-spark-2.2 #186

Jenkins build is back to stable : carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #186

carbondata git commit: [CARBONDATA-2253][SDK] Support write JSON/Avro data to carbon files

Build failed in Jenkins: carbondata-master-spark-2.1 #2148

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2148

carbondata git commit: [HOTFIX] Fix CI random failure

[5/6] carbondata git commit: [CARBONDATA-2224][File Level Reader Support] External File level reader support

[1/6] carbondata git commit: [CARBONDATA-2255] Rename the streaming examples

[6/6] carbondata git commit: [CARBONDATA-2224][File Level Reader Support] External File level reader support

[3/6] carbondata git commit: [CARBONDATA-2250][DataLoad] Reduce massive object generation in global sort

[4/6] carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2146

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Store SDK #2147

Build failed in Jenkins: carbondata-master-spark-2.2 #184

Jenkins build became unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #184

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #184

Jenkins build is unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #185

Build failed in Jenkins: carbondata-master-spark-2.2 #185

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Store SDK #185

carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

carbondata git commit: [CARBONDATA-2244]fix creating pre-aggregate table bug when there are invisibility INSERT_IN_PROGRESS/INSERT_OVERWRITE_IN_PROGRESS segments on main table

25 matches

Site Navigation

Mail list logo

Footer information