[jira] [Created] (KYLIN-2846) Add a config of hbase namespace for cube storage
Liu Shaohui created KYLIN-2846: -- Summary: Add a config of hbase namespace for cube storage Key: KYLIN-2846 URL: https://issues.apache.org/jira/browse/KYLIN-2846 Project: Kylin Issue Type: New Feature Components: Storage - HBase Affects Versions: v2.1.0 Reporter: Liu Shaohui Assignee: liyang Priority: Minor Fix For: Future In multi-tenancy HBase cluster, namespace is important for quota management and permission control. So we add a global configuration of hbase namespace for cube storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-3357) Sum of small int measure may be nagetive after KYLIN-2982
Liu Shaohui created KYLIN-3357: -- Summary: Sum of small int measure may be nagetive after KYLIN-2982 Key: KYLIN-3357 URL: https://issues.apache.org/jira/browse/KYLIN-3357 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v2.3.0 Reporter: Liu Shaohui After KYLIN-2982, the sum of small int measure may be nagetive. Same problem is reported in kylin user mail with title "negative result in kylin 2.3.0" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3156) Failed to delete meta path in SparkCubingByLayer
Liu Shaohui created KYLIN-3156:
--
Summary: Failed to delete meta path in SparkCubingByLayer
Key: KYLIN-3156
URL: https://issues.apache.org/jira/browse/KYLIN-3156
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
After KYLIN-2945, the meta url in SparkCubingByLayer will be a string of
StorageURL not string with path@hdfs format. This will make the deleteHDFSMeta
method failed in SparkCubingByLayer.
{quote}
2018-01-08,11:51:50,903 ERROR org.apache.spark.deploy.yarn.ApplicationMaster:
User class threw exception: java.lang.RuntimeException: error execute
org.apache.kylin.engine.spark.SparkCubingByLayer
java.lang.RuntimeException: error execute
org.apache.kylin.engine.spark.SparkCubingByLayer
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653)
Caused by: java.lang.IllegalArgumentException: Cannot create FileSystem from
URI: kylin_tst:kylin_metadata
at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:98)
at
org.apache.kylin.common.util.HadoopUtil.getFileSystem(HadoopUtil.java:78)
at
org.apache.kylin.engine.spark.SparkCubingByLayer.deleteHDFSMeta(SparkCubingByLayer.java:484)
at
org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:207)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 6 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at
index 5: kylin_tst:kylin_metadata
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parse(URI.java:3048)
at java.net.URI.(URI.java:588)
at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:96)
... 10 more
{quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-3495) Wrong datatypo when using on max function on a empty column
Liu Shaohui created KYLIN-3495:
--
Summary: Wrong datatypo when using on max function on a empty
column
Key: KYLIN-3495
URL: https://issues.apache.org/jira/browse/KYLIN-3495
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
SQL:
{code:java}
select
count(*),sum(PER_BYTES_TIME_COST)/count(PER_BYTES_TIME_COST),max(PER_BYTES_TIME_COST),min(PER_BYTES_TIME_COST)
from KYLIN_ONEBOX.HIVE_METRICS_JOB_DEV where KDAY_DATE >= '2018-07-01' and
KDAY_DATE <= '2018-07-31' and PROJECT ='LEARN_KYLIN'{code}
Exception:
{code:java}
NoSuchMethodException: SqlFunctions.greater(java.math.BigDecimal, double)
while resolving method 'greater[class java.math.BigDecimal, double]' in class
class org.apache.calcite.runtime.SqlFunctions
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at
org.apache.kylin.rest.service.QueryService.execute(QueryService.java:940)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:670)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:188)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:505)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:464)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:390)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3780) Add built instance in Job info
Liu Shaohui created KYLIN-3780: -- Summary: Add built instance in Job info Key: KYLIN-3780 URL: https://issues.apache.org/jira/browse/KYLIN-3780 Project: Kylin Issue Type: New Feature Reporter: Liu Shaohui Assignee: Liu Shaohui In DistributedScheduler, it's hard to known which machine the kylin job is running on. But this info is helpful to debug the failed jobs. So we add the job built instance info in job info and kylin web ui. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3726) KylinSession should load spark properties from spark-defaults.conf
Liu Shaohui created KYLIN-3726:
--
Summary: KylinSession should load spark properties from
spark-defaults.conf
Key: KYLIN-3726
URL: https://issues.apache.org/jira/browse/KYLIN-3726
Project: Kylin
Issue Type: Sub-task
Components: Storage - Parquet
Reporter: Liu Shaohui
When testing parquet storage, the spark session job failed to be submit for no
JAVA_HOME in executor env. This config is set in the spark default property
file: spark-defaults.conf.
{code}
2018-12-18,15:13:15,466 ERROR org.apache.spark.deploy.yarn.YarnAllocator:
Failed to launch executor 6 on container
container_e823_1541646991414_1025309_01_07
java.util.NoSuchElementException: key not found: JAVA_HOME at
scala.collection.MapLike$class.default(MapLike.scala:228) at
scala.collection.AbstractMap.default(Map.scala:59) at
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:286)
at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:275)
at scala.Option.foreach(Option.scala:257) at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:275)
at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:274)
at scala.Option.foreach(Option.scala:257) at
org.apache.spark.deploy.yarn.ExecutorRunnable.prepareEnvironment(ExecutorRunnable.scala:274)
at
org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:92)
at
org.apache.spark.deploy.yarn.ExecutorRunnable.run(ExecutorRunnable.scala:69) at
org.apache.spark.deploy.yarn.YarnAllocator$$anonfun$runAllocatedContainers$1$$anon$1.run(YarnAllocator.scala:556)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3900) Discard all expired ERROR or STOPPED jobs to cleanup kylin metadata
Liu Shaohui created KYLIN-3900: -- Summary: Discard all expired ERROR or STOPPED jobs to cleanup kylin metadata Key: KYLIN-3900 URL: https://issues.apache.org/jira/browse/KYLIN-3900 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Currently metadata cleanup job only delete expired discarded and succeed jobs, ERROR or STOPPED jobs are left which may cause too many meta in hbase in a long term. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3909) kylin job failed for MappeableRunContainer is not registered
Liu Shaohui created KYLIN-3909: -- Summary: kylin job failed for MappeableRunContainer is not registered Key: KYLIN-3909 URL: https://issues.apache.org/jira/browse/KYLIN-3909 Project: Kylin Issue Type: Bug Reporter: Liu Shaohui |Job aborted due to stage failure: Task 2 in stage 1.0 failed 4 times, most recent failure: Lost task 2.3 in stage 1.0 (TID 2621, zjy-hadoop-prc-st2587.bj, executor 53): com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MappeableRunContainer| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3917) Add max segment merge span to cleanup intermediate data of cube building
Liu Shaohui created KYLIN-3917: -- Summary: Add max segment merge span to cleanup intermediate data of cube building Key: KYLIN-3917 URL: https://issues.apache.org/jira/browse/KYLIN-3917 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Currently the intermediate data of building cube can not be deleted for it maybe used for later cubing merging. But it result in double space used in HDFS. In actual scenario, we only need month-level segment span in maximum. So if a span of segment is larger than a month, we think it don't need be merged and the intermediate data can be deleted. So we can add a config kylin.cube.max-segment-merge.span, default is -1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3918) Add project name in cube and job pages
Liu Shaohui created KYLIN-3918: -- Summary: Add project name in cube and job pages Key: KYLIN-3918 URL: https://issues.apache.org/jira/browse/KYLIN-3918 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui In a production cluster, there will be many projects and each project has many cubes. It's useful to show project name in cube and job pages. So the admin can be quick to known which project the abnormal cube or failed job belongs to and get contact with the users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3901) Use multi threads to speed up the storage cleanup job
Liu Shaohui created KYLIN-3901: -- Summary: Use multi threads to speed up the storage cleanup job Key: KYLIN-3901 URL: https://issues.apache.org/jira/browse/KYLIN-3901 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3911) Check if HBase table is enabled before diabling table in DeployCoprocessorCLI
Liu Shaohui created KYLIN-3911:
--
Summary: Check if HBase table is enabled before diabling table in
DeployCoprocessorCLI
Key: KYLIN-3911
URL: https://issues.apache.org/jira/browse/KYLIN-3911
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
HBase tables may be disabled for operation issues and the previous interrupted
DeployCoprocessorCLI, which cause the new DeployCoprocessorCLI failed.
{code:java}
2018-06-08 10:40:23,489 ERROR [pool-5-thread-6] util.DeployCoprocessorCLI:383 :
Error processing kylin_bigdata_prod:KYLIN_A9520J93GU
org.apache.hadoop.hbase.TableNotEnabledException:
org.apache.hadoop.hbase.TableNotEnabledException:
kylin_bigdata_prod:KYLIN_A9520J93GU
at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.prepare(DisableTableHandler.java:102)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:2609)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:2619)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:44586)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2061)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:125)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:83)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
@c3-hadoop-prc-ct36.bj/10.136.14.13:33500
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:220)
ingCaller.java:86)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3239)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsync(HBaseAdmin.java:919)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:948)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:294)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI$ResetCoprocessorWorker.run(DeployCoprocessorCLI.java:375)
```{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3912) Support cube level mapreduuce queue config for BeelineHiveClient
Liu Shaohui created KYLIN-3912: -- Summary: Support cube level mapreduuce queue config for BeelineHiveClient Key: KYLIN-3912 URL: https://issues.apache.org/jira/browse/KYLIN-3912 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui To support multi tenants, we set different mapreduce queue config for different projects and cubes, but BeelineHiveClient don't use those configs. So the getHiveTableRows api always run on same queue in kylin_hive_conf or jdbc url, which cause computing resource competition. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3913) Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata
Liu Shaohui created KYLIN-3913: -- Summary: Remove getAllOutputs api in ExecutableManager to avoid OOM for large metadata Key: KYLIN-3913 URL: https://issues.apache.org/jira/browse/KYLIN-3913 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui In a big cluster, there will be many job info left in the metadata. The kylin server will be OOM when search the jobs with a long time range. The reason is that ExecutableManager will load all job output info into memory when search a job. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3880) DataType is incompatible in Kylin HBase coprocessor
Liu Shaohui created KYLIN-3880:
--
Summary: DataType is incompatible in Kylin HBase coprocessor
Key: KYLIN-3880
URL: https://issues.apache.org/jira/browse/KYLIN-3880
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
During upgrade kylin from 2.4.1 to 2.5.2, the query will failed for the
incompatible class in Kylin HBase coprocessor
{code:java}
2019-03-12,17:48:11,530 INFO
[FifoRWQ.default.readRpcServer.handler=197,queue=13,port=24600]
org.apache.hadoop.hdfs.DFSClient: Access token was invalid when connecting to
/10.152.33.45:22402 :
org.apache.hadoop.hdfs.security.token.block.InvalidBlockTokenException: Got
access token error for OP_READ_BLOCK, self=/10.152.33.44:55387,
remote=/10.152.33.45:22402, for file
/hbase/zjyprc-xiaomi/data/miui_sec/data/4b88a72f5bd37daca00efb842e676ca8/C/6593503eb213431998db117cf3dab3a6,
for pool BP-792581576-10.152.48.22-1510572454905 block 1899006034_825272806
2019-03-12,17:48:12,135 INFO
[FifoRWQ.default.readRpcServer.handler=231,queue=15,port=24600]
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService:
start query dc0fadcf-3689-5508-9a45-559aaebfd4e0 in thread
FifoRWQ.default.readRpcServer.handler=231,queue=15,port=24600
2019-03-12,17:48:12,135 ERROR
[FifoRWQ.default.readRpcServer.handler=231,queue=15,port=24600]
org.apache.hadoop.ipc.RpcServer: Unexpected throwable object
java.lang.RuntimeException: java.io.InvalidClassException:
org.apache.kylin.metadata.datatype.DataType; local class incompatible: stream
classdesc serialVersionUID = -8891652700267537109, local class serialVersionUID
= -406124487097947
at
org.apache.kylin.cube.gridtable.TrimmedCubeCodeSystem.readDimensionEncoding(TrimmedCubeCodeSystem.java:87)
at
org.apache.kylin.cube.gridtable.TrimmedCubeCodeSystem$1.deserialize(TrimmedCubeCodeSystem.java:122)
at
org.apache.kylin.cube.gridtable.TrimmedCubeCodeSystem$1.deserialize(TrimmedCubeCodeSystem.java:91)
at org.apache.kylin.gridtable.GTInfo$1.deserialize(GTInfo.java:346)
at org.apache.kylin.gridtable.GTInfo$1.deserialize(GTInfo.java:307)
at
org.apache.kylin.gridtable.GTScanRequest$2.deserialize(GTScanRequest.java:466)
at
org.apache.kylin.gridtable.GTScanRequest$2.deserialize(GTScanRequest.java:412)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:259)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6625)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:4336)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:4318)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34964)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2059)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:126)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:152)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:128)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.InvalidClassException:
org.apache.kylin.metadata.datatype.DataType; local class incompatible: stream
classdesc serialVersionUID = -8891652700267537109, local class serialVersionUID
= -406124487097947
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1885)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at
org.apache.kylin.dimension.AbstractDateDimEnc.readExternal(AbstractDateDimEnc.java:137)
at
java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2118)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at
[jira] [Created] (KYLIN-3882) kylin master build failed for pom issues
Liu Shaohui created KYLIN-3882:
--
Summary: kylin master build failed for pom issues
Key: KYLIN-3882
URL: https://issues.apache.org/jira/browse/KYLIN-3882
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
As title.
1, Kyligence repo id : nexus conflicts with local maven settings.xml
{code:java}
[ERROR] Failed to execute goal on project kylin-core-metadata: Could not
resolve dependencies for project
org.apache.kylin:kylin-core-metadata:jar:3.0.0-SNAPSHOT: Failure to find
org.apache.calcite:calcite-core:jar:1.16.0-kylin-r2 in
http://nexus.x./nexus/content/groups/public was cached in the local
repository, resolution will not be reattempted until the update interval of
nexus has elapsed or updates are forced -> [Help 1]
{code}
2, maven.compiler.source/target is not set
{code:java}
[INFO] Compiling 2 Scala sources and 18 Java sources to
/ssd/liushaohui/workspace/computing/kylin/engine-spark/target/classes ...
[WARNING] [Warn] : bootstrap class path not set in conjunction with -source 1.6
[ERROR] [Error]
/ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkBatchCubingJobBuilder2.java:148:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator)
[ERROR] [Error]
/ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:239:
try-with-resources is not supported in -source 1.6
(use -source 7 or higher to enable try-with-resources)
[ERROR] [Error]
/ssd/liushaohui/workspace/computing/kylin/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkCubingByLayer.java:251:
diamond operator is not supported in -source 1.6
(use -source 7 or higher to enable diamond operator){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
Liu Shaohui created KYLIN-3884:
--
Summary: loading hfile to HBase failed for temporary dir in
output path
Key: KYLIN-3884
URL: https://issues.apache.org/jira/browse/KYLIN-3884
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
{code:java}
2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
LoadIncrementalHFiles
2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197 :
Skipping non-directory
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352 :
-
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
error execute HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08,
name=Load HFile to HBase Table, state=RUNNING}
java.io.FileNotFoundException: Path is not a file:
/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
Caused by:
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
not a file:
/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3886) Missing argument for options for yarn command
Liu Shaohui created KYLIN-3886: -- Summary: Missing argument for options for yarn command Key: KYLIN-3886 URL: https://issues.apache.org/jira/browse/KYLIN-3886 Project: Kylin Issue Type: Bug Reporter: Liu Shaohui 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing argument for options 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage: application 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -appStates Works with -list to filter applications 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on input comma-separated list of 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : application states. The valid application 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state can be one of the following: 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : NING,FINISHED,FAILED,KILLED 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -appTypes Works with -list to filter applications 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on input comma-separated list of 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : application types. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help Displays help for all commands. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill Kills the application. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list List applications. Supports optional use 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of -appTypes to filter applications based -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3885) Build dimension dictionary job costs too long when using Spark fact distinct
Liu Shaohui created KYLIN-3885:
--
Summary: Build dimension dictionary job costs too long when using
Spark fact distinct
Key: KYLIN-3885
URL: https://issues.apache.org/jira/browse/KYLIN-3885
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
Build dimension dictionary job costs less than 20 minutes when using mapreduce
fact distinct,but but it costs more than 3 hours when using spark fact distinct.
{code:java}
"Scheduler 542945608 Job 05c62aca-853f-396e-9653-f20c9ebd8ebc-329" #329 prio=5
os_prio=0 tid=0x7f312109c800 nid=0x2dc0b in Object.wait()
[0x7f30d8d24000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.hadoop.ipc.Client.call(Client.java:1482)
- locked <0x0005c3110fc0> (a org.apache.hadoop.ipc.Client$Call)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy33.delete(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:573)
at sun.reflect.GeneratedMethodAccessor193.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:249)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:107)
at com.sun.proxy.$Proxy34.delete(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2057)
at
org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:682)
at
org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:675)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:696)
at
org.apache.hadoop.fs.FilterFileSystem.delete(FilterFileSystem.java:232)
at
org.apache.hadoop.fs.viewfs.ChRootedFileSystem.delete(ChRootedFileSystem.java:198)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.delete(ViewFileSystem.java:334)
at
org.apache.hadoop.hdfs.FederatedDFSFileSystem.delete(FederatedDFSFileSystem.java:232)
at
org.apache.kylin.dict.global.GlobalDictHDFSStore.deleteSlice(GlobalDictHDFSStore.java:211)
at
org.apache.kylin.dict.global.AppendTrieDictionaryBuilder.flushCurrentNode(AppendTrieDictionaryBuilder.java:137)
at
org.apache.kylin.dict.global.AppendTrieDictionaryBuilder.addValue(AppendTrieDictionaryBuilder.java:97)
at
org.apache.kylin.dict.GlobalDictionaryBuilder.addValue(GlobalDictionaryBuilder.java:85)
at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:82)
at
org.apache.kylin.dict.DictionaryManager.buildDictFromReadableTable(DictionaryManager.java:303)
at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:290)
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildDictionary(CubeManager.java:1043)
at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:1012)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:72)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
Liu Shaohui created KYLIN-3887:
--
Summary: Query with decimal sum measure of double complied failed
after KYLIN-3703
Key: KYLIN-3887
URL: https://issues.apache.org/jira/browse/KYLIN-3887
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
After KYLIN-3703, Query with decimal sum measure of double complied failed.
{code:java}
Caused by: org.codehaus.commons.compiler.CompileException:
Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
"java.lang.Double"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3893) Cube build failed for wrong row key column description
Liu Shaohui created KYLIN-3893:
--
Summary: Cube build failed for wrong row key column description
Key: KYLIN-3893
URL: https://issues.apache.org/jira/browse/KYLIN-3893
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
User created wrong RowKeyColDesc, eg,
RowKeyColDesc\{column=MYSQL_FEEDBACK_USER_AUDIT.DATE,
encoding=integer:undefined}
which cause the cube build forever.
{code:java}
org.apache.kylin.engine.mr.exception.HadoopShellException:
java.lang.NumberFormatException: For input string: "undefined" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580) at
java.lang.Integer.parseInt(Integer.java:615) at
org.apache.kylin.dimension.IntegerDimEnc$Factory.createDimensionEncoding(IntegerDimEnc.java:65)
at
org.apache.kylin.dimension.DimensionEncodingFactory.create(DimensionEncodingFactory.java:65)
at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:74) at
org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:206)
at
org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
at
org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:102)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)result code:2 at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3809) Support Zookeeper based rest server discovery
Liu Shaohui created KYLIN-3809: -- Summary: Support Zookeeper based rest server discovery Key: KYLIN-3809 URL: https://issues.apache.org/jira/browse/KYLIN-3809 Project: Kylin Issue Type: New Feature Reporter: Liu Shaohui Currently to broadcast config or meta changes, all kylin servers must be set in kylin.properties. It's not convenient when adding or removing kylin server especially in k8s env. So we can register the endpoint to zk and make the rest server discovery automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3962) Support streaming cubing using Spark Streaming
Liu Shaohui created KYLIN-3962: -- Summary: Support streaming cubing using Spark Streaming Key: KYLIN-3962 URL: https://issues.apache.org/jira/browse/KYLIN-3962 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui KYLIN-3654 introduced the Real-time Streaming, but in my opinion, the arch is a little too complicated to handle. As streaming frameworks like spark streaming, flink are widely used in many companies.Can we use the streaming framework to support real time cubing in Kylin. This is just a proposal. More discussion and suggestions are welcomed~ More details of this proposal will be added later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4029) Overwriting conflict when create a new data model
Liu Shaohui created KYLIN-4029:
--
Summary: Overwriting conflict when create a new data model
Key: KYLIN-4029
URL: https://issues.apache.org/jira/browse/KYLIN-4029
Project: Kylin
Issue Type: Improvement
Reporter: Liu Shaohui
A creating model request failed for Overwriting conflicts when saving project
metadata.
It left a stale state in meta and user can not delete it or creating a one with
same name.
{code:java}
2019-05-31 16:35:11,668 ERROR [http-bio-7070-exec-57]
controller.BasicController:63 :
org.apache.kylin.common.persistence.WriteConflictException: Overwriting
conflict /project/BigBI_Hive.json, expect old TS 1559291698212, but it is
1559291711327
at
org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:326)
at
org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:327)
at
org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:309)
at
org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:288)
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192)
at
org.apache.kylin.metadata.project.ProjectManager.save(ProjectManager.java:373)
at
org.apache.kylin.metadata.project.ProjectManager.addModelToProject(ProjectManager.java:251)
at
org.apache.kylin.metadata.model.DataModelManager.createDataModelDesc(DataModelManager.java:256)
at
org.apache.kylin.rest.service.ModelService.createModelDesc(ModelService.java:148)
at
org.apache.kylin.rest.controller.ModelController.saveModelDesc(ModelController.java:128){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4021) Async Broadcast of project schema may cause creating cube failed
Liu Shaohui created KYLIN-4021:
--
Summary: Async Broadcast of project schema may cause creating cube
failed
Key: KYLIN-4021
URL: https://issues.apache.org/jira/browse/KYLIN-4021
Project: Kylin
Issue Type: Bug
Components: Metadata
Reporter: Liu Shaohui
In our prod cluster, we found some creating cube requests failed for the model
not found.
The problem is that users will create the cube right after creating the model
success. But the the two requests may be routed to two different servers.
When the other server receive creating cube request, the project schema may be
not updated for the async Broadcast and the server can not found the model
related to the cube.
the log at query server 1
{code:java}
kylin.log.11:2019-05-27 10:26:44,143 INFO [http-bio-7070-exec-962]
model.DataModelManager:248 : Saving Model model_k1_bb_83_uyyy3636 to
Project BigBI_Hive with bigbi_kylin as owner
kylin.log.11:2019-05-27 10:26:44,144 INFO [http-bio-7070-exec-962]
model.DataModelManager:185 : Model model_k1_bb_83_uyyy3636 is missing or
unloaded yet
kylin.log.11:2019-05-27 10:26:44,145 INFO [http-bio-7070-exec-962]
persistence.ResourceStore:309 : Update resource:
/model_desc/model_k1_bb_83_uyyy3636.json with content:{code}
and the log at query server 2
{code:java}
2019-05-27 10:26:44,296 WARN [http-bio-7070-exec-132] cube.CubeDescManager:195
: Broken cube desc CubeDesc [name=cube_b_bb_83_uyyy3636]
java.lang.NullPointerException: DateModelDesc(model_k1_bb_83_uyyy3636) not
found
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:235)
at org.apache.kylin.cube.model.CubeDesc.init(CubeDesc.java:664)
at
org.apache.kylin.cube.CubeDescManager.createCubeDesc(CubeDescManager.java:193)
at
org.apache.kylin.rest.service.CubeService.createCubeAndDesc(CubeService.java:216)
at
org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at
org.apache.kylin.rest.service.CubeService$$EnhancerBySpringCGLIB$$20946622.createCubeAndDesc()
at
org.apache.kylin.rest.controller.CubeController.saveCubeDesc(CubeController.java:735)
at sun.reflect.GeneratedMethodAccessor341.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4025) Add detail exception in kylin http response
Liu Shaohui created KYLIN-4025: -- Summary: Add detail exception in kylin http response Key: KYLIN-4025 URL: https://issues.apache.org/jira/browse/KYLIN-4025 Project: Kylin Issue Type: New Feature Components: REST Service Affects Versions: v2.5.2 Reporter: Liu Shaohui Assignee: Liu Shaohui Currently there is no detailed in http response when the requests go wrong, because InternalErrorException in controller wrap the execption and remove the exception stack. It's better add the detail exception in kylin http response -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4026) Avoid too many file append operation in HiveProducer of hive metrics reporter
Liu Shaohui created KYLIN-4026: -- Summary: Avoid too many file append operation in HiveProducer of hive metrics reporter Key: KYLIN-4026 URL: https://issues.apache.org/jira/browse/KYLIN-4026 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Currently for each write in HiveProducer, there will be a hdfs append operation, which is heavy for HDFS. A improvement is to keep a FSDataOutputStream in HiveProducer and write data to it continuously -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4048) Too long spark cube building time for too many eviction and loading for dict slices
Liu Shaohui created KYLIN-4048:
--
Summary: Too long spark cube building time for too many eviction
and loading for dict slices
Key: KYLIN-4048
URL: https://issues.apache.org/jira/browse/KYLIN-4048
Project: Kylin
Issue Type: Improvement
Affects Versions: v2.5.2
Reporter: Liu Shaohui
In our cluster, a cube building costs too long. In the log for spark, we found
there are too many eviction and loading for dict slices in AppendTrieDictionary.
{code:java}
$ grep "read slice from" spark.log | wc -l
119721
$ grep "Evict slice with key" spark.log| wc -l
119634
{code}
The reason is that the memory of spark executor(4G) is not enough to hold all
the slices of dict(3.3G in hdfs) in memory, which cause the bad performance of
cube building.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4005) Saving Cube of a aggregation Groups(40 Dimensions, Max Dimension Combination:5) may cause kylin server OOM
Liu Shaohui created KYLIN-4005:
--
Summary: Saving Cube of a aggregation Groups(40 Dimensions, Max
Dimension Combination:5) may cause kylin server OOM
Key: KYLIN-4005
URL: https://issues.apache.org/jira/browse/KYLIN-4005
Project: Kylin
Issue Type: Bug
Components: REST Service
Affects Versions: v2.5.2
Reporter: Liu Shaohui
Fix For: Future
A user try to save a cube with a aggregation Groups(40 Dimensions, Max
Dimension Combination:5) caused the kylin server OOM. The reason is that the
DefaultCuboidScheduler will cost a lot memory when calculating all cube ids.
The stack is following
{code}
http-bio-7070-exec-35
at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
at java.util.HashMap.resize()[Ljava/util/HashMap$Node; (HashMap.java:704)
at
java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object;
(HashMap.java:663)
at
java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
(HashMap.java:612)
at java.util.HashSet.add(Ljava/lang/Object;)Z (HashSet.java:220)
at java.util.AbstractCollection.addAll(Ljava/util/Collection;)Z
(AbstractCollection.java:344)
at
org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.getOnTreeParentsByLayer(Ljava/util/Collection;)Ljava/util/Set;
(DefaultCuboidScheduler.java:240)
at
org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.buildTreeBottomUp()Lorg/apache/kylin/common/util/Pair;
(DefaultCuboidScheduler.java:183)
at
org.apache.kylin.cube.cuboid.DefaultCuboidScheduler.(Lorg/apache/kylin/cube/model/CubeDesc;)V
(DefaultCuboidScheduler.java:58)
at
sun.reflect.GeneratedConstructorAccessor140.newInstance([Ljava/lang/Object;)Ljava/lang/Object;
(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance([Ljava/lang/Object;)Ljava/lang/Object;
(DelegatingConstructorAccessorImpl.java:45)
at
java.lang.reflect.Constructor.newInstance([Ljava/lang/Object;)Ljava/lang/Object;
(Constructor.java:423)
at
org.apache.kylin.cube.cuboid.CuboidScheduler.getInstance(Lorg/apache/kylin/cube/model/CubeDesc;)Lorg/apache/kylin/cube/cuboid/CuboidScheduler;
(CuboidScheduler.java:41)
at
org.apache.kylin.cube.model.CubeDesc.getInitialCuboidScheduler()Lorg/apache/kylin/cube/cuboid/CuboidScheduler;
(CubeDesc.java:750)
at
org.apache.kylin.cube.cuboid.CuboidCLI.simulateCuboidGeneration(Lorg/apache/kylin/cube/model/CubeDesc;Z)I
(CuboidCLI.java:47)
at
org.apache.kylin.rest.service.CubeService.updateCubeAndDesc(Lorg/apache/kylin/cube/CubeInstance;Lorg/apache/kylin/cube/model/CubeDesc;Ljava/lang/String;Z)Lorg/apache/kylin/cube/model/CubeDesc;
(CubeService.java:287)
at
org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
(Unknown Source)
at
org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object;
(CglibAopProxy.java:669)
at
org.apache.kylin.rest.service.CubeService$$EnhancerBySpringCGLIB$$34de75c4.updateCubeAndDesc(Lorg/apache/kylin/cube/CubeInstance;Lorg/apache/kylin/cube/model/CubeDesc;Ljava/lang/String;Z)Lorg/apache/kylin/cube/model/CubeDesc;
(Unknown Source)
at
org.apache.kylin.rest.controller.CubeController.updateCubeDesc(Lorg/apache/kylin/rest/request/CubeRequest;)Lorg/apache/kylin/rest/request/CubeReq
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3994) StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager
Liu Shaohui created KYLIN-3994: -- Summary: StorageCleanupJob may delete cube id data of new built segment because of cube cache in CubeManager Key: KYLIN-3994 URL: https://issues.apache.org/jira/browse/KYLIN-3994 Project: Kylin Issue Type: Bug Affects Versions: v2.5.2 Reporter: Liu Shaohui In our production cluster, we found that the cube id data of a new-built segment is deleted by the StorageCleanupJob. After checking the code of cleanUnusedHdfsFiles in StorageCleanupJob, we found that there is a bug here: CubeManager read all cube meta in initiation and cache it for later listAllCubes operations, the metadata will be out of data after listing the hdfs working dir. So the working directory of a finished job may be deleted unexpectedly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3997) Add a health check job of Kylin
Liu Shaohui created KYLIN-3997:
--
Summary: Add a health check job of Kylin
Key: KYLIN-3997
URL: https://issues.apache.org/jira/browse/KYLIN-3997
Project: Kylin
Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Kylin has many inner meta data and outer dependencies. There may be no
inconsistent for bugs or failures. It's better to have a a health check job to
find these inconsistent issues in advance。
The inconsistent issues we found in our clusters are followings
* {color:#808080}the cubeid data not exist for cube merging
{color}
* {color:#808080}hbase table not exist or online for a segment{color}
* {color:#808080}there are holes in cube segments(The build of some days
failed, but user not found it){color}
* {color:#808080}Too many segment(hbase tables){color}
* {color:#808080}metadata of stale segment left in cube{color}
* {color:#808080}Some cubes have not be updated/built for a long time{color}
* {color:#808080}Some important parameters are no set in cube desc{color}
* {color:#808080}...{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4092) Support setting seperate jvm params for kylin backgroud tools
Liu Shaohui created KYLIN-4092: -- Summary: Support setting seperate jvm params for kylin backgroud tools Key: KYLIN-4092 URL: https://issues.apache.org/jira/browse/KYLIN-4092 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Usually, the memory set in setenv.sh for query server is larger then 8G, which is not suitable for kylin background tools (meta cleaup, storage cleanup, health check) So It's better to have a seperate env for kylin tools -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4093) Slow query pages should be open to all users of the project
Liu Shaohui created KYLIN-4093: -- Summary: Slow query pages should be open to all users of the project Key: KYLIN-4093 URL: https://issues.apache.org/jira/browse/KYLIN-4093 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Currently the show query page only can been seen for kylin admins. It's very useful for modlers and analysts for this project. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4094) Add script to create system tables and cubes automatically
Liu Shaohui created KYLIN-4094: -- Summary: Add script to create system tables and cubes automatically Key: KYLIN-4094 URL: https://issues.apache.org/jira/browse/KYLIN-4094 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui See: [http://kylin.apache.org/docs/tutorial/setup_systemcube.html] It's a little complex to setup the the system cubes. We can add a scripts to make it easier. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4095) Add RESOURCE_PATH_PREFIX option in ResourceTool
Liu Shaohui created KYLIN-4095: -- Summary: Add RESOURCE_PATH_PREFIX option in ResourceTool Key: KYLIN-4095 URL: https://issues.apache.org/jira/browse/KYLIN-4095 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui ResourceTool is very useful to fix the metadata with overlap segments. But downloading and uploading entire metadata is too heavy. It's better to have a RESOURCE_PATH_PREFIX option for downloading and uploading cmds. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4097) Throw exception when too many dict slice eviction in AppendTrieDictionary
Liu Shaohui created KYLIN-4097: -- Summary: Throw exception when too many dict slice eviction in AppendTrieDictionary Key: KYLIN-4097 URL: https://issues.apache.org/jira/browse/KYLIN-4097 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui When the global dict is too large than the spark executor memory, there will be too many dict slice evictions and loads in AppendTrieDictionary, and the build job will be very slow. It's better to throw an exception in advance in this case. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4098) Add cube auto merge api
Liu Shaohui created KYLIN-4098: -- Summary: Add cube auto merge api Key: KYLIN-4098 URL: https://issues.apache.org/jira/browse/KYLIN-4098 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui Currently the auto merging of cube is triggered by the event of new segment is ready automatically. When the cluster restart, there may be too many merging job. It's better to have a rest api to trigger the merging and make it more controllable. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4099) Using no blocking unpersist in spark cubing job
Liu Shaohui created KYLIN-4099:
--
Summary: Using no blocking unpersist in spark cubing job
Key: KYLIN-4099
URL: https://issues.apache.org/jira/browse/KYLIN-4099
Project: Kylin
Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
By default, the unpersist operation of RDD in spark is blocking which may cost
a lot time and
some times it may failed for some spark executors lost.
We can set blocking false to improve it.
{code:java}
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:208)
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190)
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
scala.concurrent.Await$.result(package.scala:190)
org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:81)
org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:127)
org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1709)
org.apache.spark.rdd.RDD.unpersist(RDD.scala:216)
org.apache.spark.api.java.JavaPairRDD.unpersist(JavaPairRDD.scala:73)
org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:204)
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653){code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (KYLIN-4100) Add overall job number statistics in monitor page
Liu Shaohui created KYLIN-4100: -- Summary: Add overall job number statistics in monitor page Key: KYLIN-4100 URL: https://issues.apache.org/jira/browse/KYLIN-4100 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui Currently it's hard to get pending and running job number in mointor page, we can only continue to click more until the end. It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4101) set hive and spark job name when building cube
Liu Shaohui created KYLIN-4101:
--
Summary: set hive and spark job name when building cube
Key: KYLIN-4101
URL: https://issues.apache.org/jira/browse/KYLIN-4101
Project: Kylin
Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Currently the job name of spark is
{color:#22}org.apache.kylin.common.util.SparkEntry{color}, which is the
main class name of spark . The mapreduce job name of hive sql is substring of
the query, which is difficult to read.
It's better to set a more readable name for the hive and spark jobs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
Liu Shaohui created KYLIN-4103: -- Summary: Make the user string in granting operation of project is case insensitive Key: KYLIN-4103 URL: https://issues.apache.org/jira/browse/KYLIN-4103 Project: Kylin Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui Currently the user name of login operation is case insensitive. User can login in kylin with lower case string or upper case string. But it is not granting operation. If we use lower case string of user name in project granting operation, there will be no exception but the user can not set the project. The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4109) CubeHFileMapperTest failed after commit: f4d2405f6aa978bbc3153c9ca9fa339b9d7e6c30
Liu Shaohui created KYLIN-4109:
--
Summary: CubeHFileMapperTest failed after commit:
f4d2405f6aa978bbc3153c9ca9fa339b9d7e6c30
Key: KYLIN-4109
URL: https://issues.apache.org/jira/browse/KYLIN-4109
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
CubeHFileMapperTest failed in 2.5.x-hadoop3.1 for following changes
{code:java}
- assertEquals("cf1", new String(p1.getSecond().getFamily(),
StandardCharsets.UTF_8));
- assertEquals("usd_amt", new String(p1.getSecond().getQualifier(),
StandardCharsets.UTF_8));
- assertEquals("35.43", new String(p1.getSecond().getValue(),
StandardCharsets.UTF_8));
+ assertEquals("cf1", new String(copy(p1.getSecond(;
+ assertEquals("usd_amt", new String(copy(p1.getSecond(;
+ assertEquals("35.43", new String(copy(p1.getSecond(;
assertEquals(key, p2.getFirst());
- assertEquals("cf1", new String(p2.getSecond().getFamily(),
StandardCharsets.UTF_8));
- assertEquals("item_count", new String(p2.getSecond().getQualifier(),
StandardCharsets.UTF_8));
- assertEquals("2", new String(p2.getSecond().getValue(),
StandardCharsets.UTF_8));
+ assertEquals("cf1", new String(copy(p2.getSecond(;
+ assertEquals("item_count", new String(copy(p2.getSecond(;
+ assertEquals("2", new String(copy(p2.getSecond(;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (KYLIN-4111) drop table failed with no valid privileges after KYLIN-3857
Liu Shaohui created KYLIN-4111:
--
Summary: drop table failed with no valid privileges after
KYLIN-3857
Key: KYLIN-4111
URL: https://issues.apache.org/jira/browse/KYLIN-4111
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
After KYLIN-3857, there will be quote ` around database and table.
The drop table sql will be:
{code:java}
DROP TABLE IF EXISTS
`kylin_onebox.kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`;{code}
Hive (1.2)with sentry will throw exception:
{code:java}
Error: Error while compiling statement: FAILED: HiveAccessControlException No
valid privileges
Required privileges for this query:
Server=server1->Db=`kylin_onebox->Table=kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`->action=drop;
Query log:
http://zjy-hadoop-prc-ct14.bj:18201/log?qid=898c7878-a961-443d-b120-cca0e2667d15_f486bd16-4bbd-4014-a0a7-c2ebfdbe6668
(state=42000,code=4)
{code}
The reason is that hive identify the databse be `kylin_onebox and table be:
kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`
May be we can fix it in hive and sentry. Just create a jira to show this
problem.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (KYLIN-4112) Add hdfs keberos token delegation in Spark to support HBase and MR use different HDFS clusters
Liu Shaohui created KYLIN-4112: -- Summary: Add hdfs keberos token delegation in Spark to support HBase and MR use different HDFS clusters Key: KYLIN-4112 URL: https://issues.apache.org/jira/browse/KYLIN-4112 Project: Kylin Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui Currently the SparkExecutable only delegate the token for yarn hdfs cluster, not for the hdfs cluster used by the HBase cluster. The spark job of Convert Cuboid Data to HFile will failed for kerberos issue. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
