This is an automated email from the ASF dual-hosted git repository.
xxyu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kylin.git.
from ead437a KYLIN-4524 system-cube.sh script can't work
new 4a7c568 KYLIN-4343 Build Global Dict by MR/Hive, configuration
new 85b7832 KYLIN-4345 Build Global Dict by MR/Hive, Parallel Part Build
Step implementation
new 428de35 KYLIN-4346 Build Global Dict by MR/Hive, Parallel Total Build
Step implementation
new ad954b7 KYLIN-4344 Build Global Dict by MR/Hive, Extract Fact Table
Distinct Columns Step
new 3e50581 KYLIN-4366 Build Global Dict by MR/Hive, Merge to dict table
Step implementation
new 5addc9d KYLIN-4367 Build Global Dict by MR/Hive, Replace intermediate
table Step implementation
new 4ebf403 KYLIN-4347 Spark engine BatchCubingJobBuilder implementation
new 17ae024 KYLIN-4348 Fix distributed concurrency lock bug
new 276f79b KYLIN-4368 Global domain dict for MR/Hive global dict
new a2489aa KYLIN-4342 Build Global Dict by MR/Hive New Version, fix
some potential bugs, such as null pointer exceptions
new 5731f43 KYLIN-4342 Improve code smell
The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/kylin/common/KylinConfigBase.java | 117 +++++++++-
.../kylin/job/constant/ExecutableConstants.java | 10 +-
.../lock/zookeeper/ZookeeperDistributedLock.java | 2 +-
.../bitmap/BitmapIntersectValueAggFunc.java | 10 +-
.../kylin/measure/bitmap/BitmapMeasureType.java | 6 +-
.../apache/kylin/metadata/model/FunctionDesc.java | 4 +
.../kylin/engine/mr/BatchCubingJobBuilder2.java | 37 +++-
.../java/org/apache/kylin/engine/mr/IInput.java | 25 ++-
.../apache/kylin/engine/mr/JobBuilderSupport.java | 76 ++++++-
.../kylin/engine/mr/common/AbstractHadoopJob.java | 4 +
.../kylin/engine/mr/common/BaseCuboidBuilder.java | 8 +-
.../kylin/engine/mr/common/BatchConstants.java | 10 +-
.../mr/steps/BuildGlobalHiveDictPartBuildJob.java | 153 +++++++++++++
.../steps/BuildGlobalHiveDictPartBuildMapper.java | 110 ++++++++++
.../steps/BuildGlobalHiveDictPartBuildReducer.java | 83 +++++++
...ava => BuildGlobalHiveDictPartPartitioner.java} | 44 ++--
.../mr/steps/BuildGlobalHiveDictTotalBuildJob.java | 130 +++++++++++
.../steps/BuildGlobalHiveDictTotalBuildMapper.java | 238 ++++++++++++++++++++
.../engine/spark/SparkBatchCubingJobBuilder2.java | 35 ++-
.../localmeta/cube_desc/ci_inner_join_cube.json | 3 +-
kubernetes/README.md | 12 +-
.../org/apache/kylin/rest/service/JobService.java | 26 ++-
.../kylin/source/hive/CreateMrHiveDictStep.java | 243 ++++++++++++++-------
.../apache/kylin/source/hive/HiveInputBase.java | 242 +++++++++++++-------
.../apache/kylin/source/hive/MRHiveDictUtil.java | 157 ++++++++++---
.../apache/kylin/source/kafka/KafkaInputBase.java | 5 +
26 files changed, 1509 insertions(+), 281 deletions(-)
create mode 100644
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/BuildGlobalHiveDictPartBuildJob.java
create mode 100644
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/BuildGlobalHiveDictPartBuildMapper.java
create mode 100644
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/BuildGlobalHiveDictPartBuildReducer.java
copy
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/{FactDistinctColumnPartitioner.java
=> BuildGlobalHiveDictPartPartitioner.java} (55%)
create mode 100644
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/BuildGlobalHiveDictTotalBuildJob.java
create mode 100644
engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/BuildGlobalHiveDictTotalBuildMapper.java
