Re:Re: Kylin will not delete old hbase table when refresh the segment
+1, the behavior can be the same as merge segments, old htables can be deleted directly, not sure it make sense or not to keep old htables to have the capability to roll back the old data. At 2018-12-17 17:49:50, "ShaoFeng Shi" wrote: >It can be improved to do the cleanup automatically. I heard some community >user has implemented that, a patch is welcomed! > >Best regards, > >Shaofeng Shi 史少锋 >Apache Kylin PMC >Work email: shaofeng@kyligence.io >Kyligence Inc: https://kyligence.io/ > >Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html >Join Kylin user mail group: user-subscr...@kylin.apache.org >Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > > > >Chao Long 于2018年12月17日周一 下午5:40写道： > >> Hi, >> You can use storage cleanup tool. >> http://kylin.apache.org/docs/howto/howto_cleanup_storage.html >> >> >> -- >> Best Regards, >> Chao Long >> >> >> -- Original -- >> From: "mailpig"; >> Date: Mon, Dec 17, 2018 05:16 PM >> To: "dev"; >> >> Subject: Kylin will not delete old hbase table when refresh the segment >> >> >> >> Hi, my kylin version is 2.1.0. It has run one year. Howerver, there has too >> many tables in hbase. >> I found that when I refresh the segment, kylin will not delete the old >> hbase >> table. Is there has a tool to delete the old hbase table? >> >> -- >> Sent from: http://apache-kylin.74782.x6.nabble.com/
[jira] [Created] (KYLIN-3729) CLUSTER BY CAST(field AS STRING) will accelerate base cuboid build with UHC global dict
Fangyuan Deng created KYLIN-3729: Summary: CLUSTER BY CAST(field AS STRING) will accelerate base cuboid build with UHC global dict Key: KYLIN-3729 URL: https://issues.apache.org/jira/browse/KYLIN-3729 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v2.5.2 Reporter: Fangyuan Deng Assignee: Fangyuan Deng -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3728) unexpected behavior when do fix holes for steaming cube
wangxianbin created KYLIN-3728: -- Summary: unexpected behavior when do fix holes for steaming cube Key: KYLIN-3728 URL: https://issues.apache.org/jira/browse/KYLIN-3728 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v2.5.1 Reporter: wangxianbin Attachments: fix hole finished.png, fix_holes_kylin.log, in process of fix holes.png after we finished fix holes, exist holes did not been filled up, and sometime more exist segment will become hole, check fix hole log in attachment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3727) cdh6 kylin2.5.2 不能构建空的cube
石宇 created KYLIN-3727: - Summary: cdh6 kylin2.5.2 不能构建空的cube Key: KYLIN-3727 URL: https://issues.apache.org/jira/browse/KYLIN-3727 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v2.5.2 Reporter: 石宇 Fix For: v2.5.2 Attachments: image-2018-12-19-08-38-19-791.png 注:kylin 已经开启,是否容忍数据源为空，默认值为 TRUE kylin.job.allow-empty-segment=true 出错位置:#12 Step Name: Load HFile to HBase Table !image-2018-12-19-08-38-19-791.png! 异常信息:HadoopShellException org.apache.kylin.engine.mr.exception.HadoopShellException: result code:-1 at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3726) KylinSession should load spark properties from spark-defaults.conf
Liu Shaohui created KYLIN-3726:
--
Summary: KylinSession should load spark properties from
spark-defaults.conf
Key: KYLIN-3726
URL: https://issues.apache.org/jira/browse/KYLIN-3726
Project: Kylin
Issue Type: Sub-task
Components: Storage - Parquet
Reporter: Liu Shaohui
When testing parquet storage, the spark session job failed to be submit for no
JAVA_HOME in executor env. This config is set in the spark default property
file: spark-defaults.conf.
{code}
2018-12-18,15:13:15,466 ERROR org.apache.spark.deploy.yarn.YarnAllocator:
Failed to launch executor 6 on container
container_e823_1541646991414_1025309_01_07
java.util.NoSuchElementException: key not found: JAVA_HOME at
scala.collection.MapLike$class.default(MapLike.scala:228) at
scala.collection.AbstractMap.default(Map.scala:59) at
scala.collection.mutable.HashMap.apply(HashMap.scala:65) at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:286)
at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:275)
at scala.Option.foreach(Option.scala:257) at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:275)
at
org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:274)
at scala.Option.foreach(Option.scala:257) at
org.apache.spark.deploy.yarn.ExecutorRunnable.prepareEnvironment(ExecutorRunnable.scala:274)
at
org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:92)
at
org.apache.spark.deploy.yarn.ExecutorRunnable.run(ExecutorRunnable.scala:69) at
org.apache.spark.deploy.yarn.YarnAllocator$$anonfun$runAllocatedContainers$1$$anon$1.run(YarnAllocator.scala:556)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
