[15/35] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/4749ca29/core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java -- diff --git a/core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java

[17/35] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/4749ca29/core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java -- diff --git

[18/35] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/4749ca29/core/src/main/java/org/apache/carbondata/core/scan/processor/DataBlockIterator.java -- diff --git

carbondata git commit: [CARBONDATA-1763] Dropped table if exception thrown while creation

2018-02-09 Thread ravipesala
Repository: carbondata Updated Branches: refs/heads/master 957a51fef -> 1be27b085 [CARBONDATA-1763] Dropped table if exception thrown while creation Preaggregate table is not getting dropped when creation fails because Exceptions from undo metadata is not handled If preaggregate table is not

carbondata git commit: [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0

2018-02-09 Thread chenliang613
Repository: carbondata Updated Branches: refs/heads/master 11a795cec -> 957a51fef [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 Upgrade pom version from 1.3.0 to 1.4.0 This closes #1961 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

Jenkins build is back to stable : carbondata-master-spark-2.1 #2046

2018-02-09 Thread Apache Jenkins Server
See

Jenkins build is back to stable : carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #2046

2018-02-09 Thread Apache Jenkins Server
See

carbondata git commit: [CARBONDATA-1763] Dropped table if exception thrown while creation

2018-02-09 Thread ravipesala
Repository: carbondata Updated Branches: refs/heads/branch-1.3 9f73f0e60 -> 1137c285f [CARBONDATA-1763] Dropped table if exception thrown while creation Preaggregate table is not getting dropped when creation fails because Exceptions from undo metadata is not handled If preaggregate table is

[08/36] carbondata git commit: [CARBONDATA-1992] Remove partitionId in CarbonTablePath

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/d3cbb026/processing/src/main/java/org/apache/carbondata/processing/loading/steps/CarbonRowDataWriterProcessorStepImpl.java -- diff --git

[23/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecuterImpl.java -- diff --git

[10/36] carbondata git commit: [CARBONDATA-1827] S3 Carbon Implementation

2018-02-09 Thread jackylk
[CARBONDATA-1827] S3 Carbon Implementation 1.Provide support for s3 in carbondata. 2.Added S3Example to create carbon table on s3. 3.Added S3CSVExample to load carbon table using csv from s3. This closes #1805 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

[03/36] carbondata git commit: [CARBONDATA-2137] Delete query performance improved

2018-02-09 Thread jackylk
[CARBONDATA-2137] Delete query performance improved Following is the configuration used : SPARK_EXECUTOR_MEMORY : 200G SPARK_DRIVER_MEMORY : 20G SPARK_EXECUTOR_CORES : 32 SPARK_EXECUTOR_INSTANCEs : 3 Earlier it was taking 20 minute now it is taking approx 5 minute This closes #1937 Project:

[07/36] carbondata git commit: [CARBONDATA-1968] Add external table support

2018-02-09 Thread jackylk
[CARBONDATA-1968] Add external table support This PR adds support for creating external table with existing carbondata files, using Hive syntax. CREATE EXTERNAL TABLE tableName STORED BY 'carbondata' LOCATION 'path' This closes #1749 Project:

[31/36] carbondata git commit: [CARBONDATA-2018][DataLoad] Optimization in reading/writing for sort temp row

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/1acc4e18/processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/holder/UnsafeSortTempFileChunkHolder.java -- diff --git

[01/36] carbondata git commit: [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 [Forced Update!]

2018-02-09 Thread jackylk
Repository: carbondata Updated Branches: refs/heads/carbonstore-rebase 132fbd4d8 -> 3d055c187 (forced update) [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 Upgrade pom version from 1.3.0 to 1.4.0 This closes #1961 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

[27/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/unsafe/UnsafeFixedLengthDimensionDataChunkStore.java -- diff --git

[29/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
[CARBONDATA-2099] Refactor query scan process to improve readability Unified concepts in scan process flow: 1.QueryModel contains all parameter for scan, it is created by API in CarbonTable. (In future, CarbonTable will be the entry point for various table operations) 2.Use term ColumnChunk to

[06/36] carbondata git commit: [REBASE] Solve conflict after rebasing master

2018-02-09 Thread jackylk
[REBASE] Solve conflict after rebasing master Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/3ee9351a Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/3ee9351a Diff:

[02/36] carbondata git commit: [CARBONDATA-1763] Dropped table if exception thrown while creation

2018-02-09 Thread jackylk
[CARBONDATA-1763] Dropped table if exception thrown while creation Preaggregate table is not getting dropped when creation fails because Exceptions from undo metadata is not handled If preaggregate table is not registered with main table(main table updation fails) then it is not dropped from

[04/36] carbondata git commit: [HOTFIX] Fix documentation errors

2018-02-09 Thread jackylk
[HOTFIX] Fix documentation errors Fix documentation errors This closes #1955 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/420cc35d Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/420cc35d Diff:

[19/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/util/AbstractDataFileFooterConverter.java -- diff --git

[28/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/FixedLengthDimensionDataChunk.java -- diff --git

[35/36] carbondata git commit: [REBASE] Solve conflict after rebasing master

2018-02-09 Thread jackylk
[REBASE] Solve conflict after rebasing master Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/b93f9542 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/b93f9542 Diff:

[05/36] carbondata git commit: [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted

2018-02-09 Thread jackylk
[CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted Problem: Unwanted updatetable status files are being generated for the delete operation where no records are deleted Analysis: when the filter value for delete

[11/36] carbondata git commit: [CARBONDATA-2080] [S3-Implementation] Propagated hadoopConf from driver to executor for s3 implementation in cluster mode.

2018-02-09 Thread jackylk
[CARBONDATA-2080] [S3-Implementation] Propagated hadoopConf from driver to executor for s3 implementation in cluster mode. Problem : hadoopconf was not getting propagated from driver to the executor that's why load was failing to the distributed environment. Solution: Setting the Hadoop conf in

[09/36] carbondata git commit: [CARBONDATA-1992] Remove partitionId in CarbonTablePath

2018-02-09 Thread jackylk
[CARBONDATA-1992] Remove partitionId in CarbonTablePath In CarbonTablePath, there is a deprecated partition id which is always 0, it should be removed to avoid confusion. This closes #1765 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

[24/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/filter/executer/ExcludeColGroupFilterExecuterImpl.java -- diff --git

[33/36] carbondata git commit: [CARBONDATA-1544][Datamap] Datamap FineGrain implementation

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/dc43a46a/datamap/examples/src/minmaxdatamap/main/java/org/apache/carbondata/datamap/examples/MinMaxDataWriter.java -- diff --git

[32/36] carbondata git commit: [CARBONDATA-2018][DataLoad] Optimization in reading/writing for sort temp row

2018-02-09 Thread jackylk
[CARBONDATA-2018][DataLoad] Optimization in reading/writing for sort temp row Pick up the no-sort fields in the row and pack them as bytes array and skip parsing them during merge sort to reduce CPU consumption This closes #1792 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo

[36/36] carbondata git commit: [HotFix][CheckStyle] Fix import related checkstyle

2018-02-09 Thread jackylk
[HotFix][CheckStyle] Fix import related checkstyle This closes #1952 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/7adb295b Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/7adb295b Diff:

[30/36] carbondata git commit: [CARBONDATA-2023][DataLoad] Add size base block allocation in data loading

2018-02-09 Thread jackylk
[CARBONDATA-2023][DataLoad] Add size base block allocation in data loading Carbondata assign blocks to nodes at the beginning of data loading. Previous block allocation strategy is block number based and it will suffer skewed data problem if the size of input files differs a lot. We introduced a

[26/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/metadata/blocklet/SegmentInfo.java -- diff --git

[25/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java -- diff --git

[17/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/processing/src/main/java/org/apache/carbondata/processing/merger/CarbonCompactionExecutor.java -- diff --git

[16/36] carbondata git commit: [CARBONDATA-2025] Unify all path construction through CarbonTablePath static method

2018-02-09 Thread jackylk
[CARBONDATA-2025] Unify all path construction through CarbonTablePath static method Refactory CarbonTablePath: 1.Remove CarbonStorePath and use CarbonTablePath only. 2.Make CarbonTablePath an utility without object creation, it can avoid creating object before using it, thus code is cleaner

[15/36] carbondata git commit: [CARBONDATA-2025] Unify all path construction through CarbonTablePath static method

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/bfa9a2c2/core/src/test/java/org/apache/carbondata/core/cache/dictionary/ForwardDictionaryCacheTest.java -- diff --git

[14/36] carbondata git commit: [CARBONDATA-2025] Unify all path construction through CarbonTablePath static method

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/bfa9a2c2/integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/GlobalDictionaryUtil.scala -- diff --git

[13/36] carbondata git commit: [CARBONDATA-2025] Unify all path construction through CarbonTablePath static method

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/bfa9a2c2/processing/src/main/java/org/apache/carbondata/processing/loading/TableProcessingOperations.java -- diff --git

[34/36] carbondata git commit: [CARBONDATA-1544][Datamap] Datamap FineGrain implementation

2018-02-09 Thread jackylk
[CARBONDATA-1544][Datamap] Datamap FineGrain implementation Implemented interfaces for FG datamap and integrated to filterscanner to use the pruned bitset from FG datamap. FG Query flow as follows. 1.The user can add FG datamap to any table and implement there interfaces. 2. Any filter query

[18/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java -- diff --git a/core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java

[12/36] carbondata git commit: [CARBONDATA-1480]Min Max Index Example for DataMap

2018-02-09 Thread jackylk
[CARBONDATA-1480]Min Max Index Example for DataMap Datamap Example. Implementation of Min Max Index through Datamap. And Using the Index while prunning. This closes #1359 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

[22/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFiterExecuterImpl.java -- diff --git

[20/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java -- diff --git

[21/36] carbondata git commit: [CARBONDATA-2099] Refactor query scan process to improve readability

2018-02-09 Thread jackylk
http://git-wip-us.apache.org/repos/asf/carbondata/blob/7b79ff18/core/src/main/java/org/apache/carbondata/core/scan/processor/DataBlockIterator.java -- diff --git

Jenkins build became unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #89

2018-02-09 Thread Apache Jenkins Server
See

Jenkins build became unstable: carbondata-master-spark-2.2 #89

2018-02-09 Thread Apache Jenkins Server
See

carbondata git commit: Revert "[CARBONDATA-2023][DataLoad] Add size base block allocation in data loading"

2018-02-09 Thread jackylk
Repository: carbondata Updated Branches: refs/heads/carbonstore 6dd8b038f -> e5c32ac96 Revert "[CARBONDATA-2023][DataLoad] Add size base block allocation in data loading" This reverts commit 6dd8b038fc898dbf48ad30adfc870c19eb38e3d0. Project:

carbondata git commit: [CARBONDATA-2137] Delete query performance improved

2018-02-09 Thread ravipesala
Repository: carbondata Updated Branches: refs/heads/branch-1.3 1137c285f -> 6e435de5e [CARBONDATA-2137] Delete query performance improved Following is the configuration used : SPARK_EXECUTOR_MEMORY : 200G SPARK_DRIVER_MEMORY : 20G SPARK_EXECUTOR_CORES : 32 SPARK_EXECUTOR_INSTANCEs : 3

Jenkins build is back to normal : carbondata-master-spark-2.2 » Apache CarbonData :: Hadoop #85

2018-02-09 Thread Apache Jenkins Server
See

carbondata git commit: [CARBONDATA-2137] Delete query performance improved

2018-02-09 Thread ravipesala
Repository: carbondata Updated Branches: refs/heads/master 1be27b085 -> 7c05f5f18 [CARBONDATA-2137] Delete query performance improved Following is the configuration used : SPARK_EXECUTOR_MEMORY : 200G SPARK_DRIVER_MEMORY : 20G SPARK_EXECUTOR_CORES : 32 SPARK_EXECUTOR_INSTANCEs : 3 Earlier

Jenkins build is back to normal : carbondata-master-spark-2.1 #2049

2018-02-09 Thread Apache Jenkins Server
See

Jenkins build is back to normal : carbondata-master-spark-2.1 » Apache CarbonData :: Hadoop #2049

2018-02-09 Thread Apache Jenkins Server
See

Jenkins build is back to normal : carbondata-master-spark-2.2 #85

2018-02-09 Thread Apache Jenkins Server
See

carbondata git commit: [HOTFIX] Fix documentation errors

2018-02-09 Thread chenliang613
Repository: carbondata Updated Branches: refs/heads/master 7c05f5f18 -> 420cc35d5 [HOTFIX] Fix documentation errors Fix documentation errors This closes #1955 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit:

Build failed in Jenkins: carbondata-master-spark-2.2 #83

2018-02-09 Thread Apache Jenkins Server
See Changes: [chenliang613] [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 -- [...truncated 62.07 KB...] [INFO] Downloading:

Build failed in Jenkins: carbondata-master-spark-2.1 #2048

2018-02-09 Thread Apache Jenkins Server
See -- [...truncated 60.67 KB...] [INFO] Analyzed bundle 'carbondata-core' with 0 classes [INFO] All coverage checks have been met. [INFO] [INFO] ---

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Hadoop #84

2018-02-09 Thread Apache Jenkins Server
See -- [INFO] [INFO]

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Hadoop #2048

2018-02-09 Thread Apache Jenkins Server
See -- [INFO] [INFO]

Jenkins build is back to stable : carbondata-master-spark-2.2 #82

2018-02-09 Thread Apache Jenkins Server
See

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Hadoop #83

2018-02-09 Thread Apache Jenkins Server
See Changes: [chenliang613] [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 -- [INFO]

Jenkins build is back to stable : carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #82

2018-02-09 Thread Apache Jenkins Server
See

Build failed in Jenkins: carbondata-master-spark-2.1 #2047

2018-02-09 Thread Apache Jenkins Server
See Changes: [chenliang613] [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 -- [...truncated 61.50 KB...] [INFO] Building Apache CarbonData :: Processing

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Hadoop #2047

2018-02-09 Thread Apache Jenkins Server
See Changes: [chenliang613] [HOTFIX] Upgrade pom version from 1.3.0 to 1.4.0 -- [INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 #84

2018-02-09 Thread Apache Jenkins Server
See -- [...truncated 60.31 KB...] [INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-core --- [INFO] Loading execution data file

carbondata git commit: [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted

2018-02-09 Thread kunalkapoor
Repository: carbondata Updated Branches: refs/heads/branch-1.3 6e435de5e -> bc3f82510 [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted Problem: Unwanted updatetable status files are being generated for the delete

carbondata git commit: [CARBONDATA-2023][DataLoad] Add size base block allocation in data loading [Forced Update!]

2018-02-09 Thread jackylk
Repository: carbondata Updated Branches: refs/heads/carbonstore-rebase 5c55dfe19 -> 132fbd4d8 (forced update) [CARBONDATA-2023][DataLoad] Add size base block allocation in data loading Carbondata assign blocks to nodes at the beginning of data loading. Previous block allocation strategy is

carbondata git commit: [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted

2018-02-09 Thread kunalkapoor
Repository: carbondata Updated Branches: refs/heads/master 420cc35d5 -> da549c2b7 [CARBONDATA-2150] Unwanted updatetable status files are being generated for the delete operation where no records are deleted Problem: Unwanted updatetable status files are being generated for the delete