This is an automated email from the ASF dual-hosted git repository. xxyu pushed a change to branch kylin-on-parquet-v2 in repository https://gitbox.apache.org/repos/asf/kylin.git.
from d43aca4 KYLIN-4721 The default source source type should be CSV not Hive with the local debug mode new 7a336fa KYLIN-4660 Remove duplicated/misleading code or comment new 56cf4e6 KYLIN-4698 Delete segment storage path after merging segment, deleting segment and droping cube new cfdcb9f KYLIN-4699 Delete job_tmp path after build/merge successfully new ff338de KYLIN-2971 Fix the wrong "Realization Names" and missing "Cuboid Ids" in logQuery when hit cache new 621868c KYLIN-4660 remove deprecated config 'spark.yarn.executor.memoryOverhead' new 2d998f1 KYLIN-4695 Automatically start sparder (for query) application when start kylin instance new ea267e2 KYLIN-4704 Add 'PERCENTILE' measure for sample metadata new 2d6d8b5 KYLIN-4695 Rewrite application id to file when restart sparder application The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .gitignore | 3 +- .../java/org/apache/kylin/common/KylinConfig.java | 24 +++++----- .../org/apache/kylin/common/KylinConfigBase.java | 1 + .../src/main/resources/kylin-defaults.properties | 2 +- .../java/org/apache/kylin/cube/CubeSegment.java | 2 +- .../kylin/job/execution/AbstractExecutable.java | 9 ++-- .../org/apache/kylin/job/execution/Idempotent.java | 2 +- .../apache/kylin/job/execution/JobTypeEnum.java | 25 ----------- .../apache/kylin/metadata/MetadataConstants.java | 1 - .../metadata/realization/RealizationType.java | 2 +- .../template/cube_desc/kylin_sales_cube.json | 23 +++++++++- examples/test_case_data/sandbox/kylin.properties | 4 +- .../engine/spark/metadata/cube/PathManager.java | 52 +++++++++++++++++++++- .../kylin/engine/spark/job/JobStepFactory.java | 13 ------ .../apache/kylin/engine/spark/job/JobStepType.java | 4 +- .../kylin/engine/spark/job/NSparkCubingStep.java | 12 +++++ .../kylin/engine/spark/job/NSparkCubingUtil.java | 4 +- .../kylin/engine/spark/job/NSparkMergingJob.java | 3 +- .../NSparkUpdateMetaAndCleanupAfterMergeStep.java | 28 ++++++++---- .../merger/AfterMergeOrRefreshResourceMerger.java | 4 +- .../engine/spark/builder/CreateFlatTable.scala | 2 +- .../kylin/engine/spark/job/CubeBuildJob.java | 7 +-- .../engine/spark/LocalWithSparkSessionTest.java | 35 ++++++++++++++- .../kylin/engine/spark/job/JobStepFactoryTest.java | 11 +---- .../org/apache/spark/sql/SparderContext.scala | 15 ++++++- .../org/apache/kylin/query/routing/Candidate.java | 1 - .../kylin/rest/init/InitialSparderContext.java | 30 ++++++------- .../org/apache/kylin/rest/service/CubeService.java | 23 ++++------ .../apache/kylin/rest/service/QueryService.java | 25 ++++++++++- server/src/main/resources/applicationContext.xml | 1 + 30 files changed, 235 insertions(+), 133 deletions(-) delete mode 100644 core-job/src/main/java/org/apache/kylin/job/execution/JobTypeEnum.java copy storage-hbase/src/main/java/org/apache/kylin/storage/hbase/cube/v2/CubeStorageQuery.java => server-base/src/main/java/org/apache/kylin/rest/init/InitialSparderContext.java (59%)