Agree. A new patch is needed in KYLIN-2846. It's not a bug still, but an unfinished improvement I think. Let's continue to improve this issue.
2017-10-18 21:05 GMT+08:00 ShaoFeng Shi <[email protected]>: > Hi Yaowu, > > Could you please update this patch in KYLIN-2846? Storage cleanup should > also support customized namespace. Thanks. > > 2017-10-18 20:37 GMT+08:00 曾耀武 <[email protected]>: > >> >> Yeah I have checked it was a bug , because I used hbase namespace >> “kylin” with the patch "https://issues.apache.org/jir >> a/secure/attachment/12885320/KYLIN-2846-001.patch” >> >> And I modified the code as below and it works well: >> >> >> MOMO@MOMOdeMacBook-Pro-5:~/gitworkspace/kylin2.0/kylin/dist/apache-kylin-2.1.0-bin/tool$ >> git diff ../../../server-base/src/main/java/org/apache/kylin/rest/job >> /StorageCleanJobHbaseUtil.java >> diff --git >> a/server-base/src/main/java/org/apache/kylin/rest/job/StorageCleanJobHbaseUtil.java >> b/server-base/src/main/java/org/apache/kylin/rest/job/Storag >> eCleanJobHbaseUtil.java >> index 3728ea1..937d02d 100644 >> --- a/server-base/src/main/java/org/apache/kylin/rest/job/Storag >> eCleanJobHbaseUtil.java >> +++ b/server-base/src/main/java/org/apache/kylin/rest/job/Storag >> eCleanJobHbaseUtil.java >> @@ -46,15 +46,28 @@ public class StorageCleanJobHbaseUtil { >> >> public static void cleanUnusedHBaseTables(boolean delete, int >> deleteTimeout) throws IOException { >> Configuration conf = HBaseConfiguration.create(); >> - CubeManager cubeMgr = CubeManager.getInstance(KylinC >> onfig.getInstanceFromEnv()); >> + KylinConfig config = KylinConfig.getInstanceFromEnv(); >> + CubeManager cubeMgr = CubeManager.getInstance(config); >> // get all kylin hbase tables >> try (HBaseAdmin hbaseAdmin = new HBaseAdmin(conf)) { >> - String tableNamePrefix = IRealizationConstants.SharedHb >> aseStorageLocationPrefix; >> + String namespace = config.getHBaseStorageNameSpace(); >> + StringBuffer sb = new StringBuffer(); >> + String tableNamePrefix = null; >> + if(namespace.equals("default")){ >> + tableNamePrefix = IRealizationConstants.SharedHb >> aseStorageLocationPrefix; >> + }else{ >> + sb.append(config.getHBaseStor >> ageNameSpace()).append(":"); >> + sb.append(IRealizationConstan >> ts.SharedHbaseStorageLocationPrefix); >> + tableNamePrefix = sb.toString(); >> + } >> + >> >> >> >> >> >> 发件人: ShaoFeng Shi <[email protected]> >> 答复: user <[email protected]> >> 日期: 2017年10月17日 星期二 下午4:16 >> 至: user <[email protected]> >> 主题: [外部邮件] Re: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 >> >> Hi yaowu, >> >> The StorageCleanupJob will delete the hbase tables that has no reference; >> If you found it didn't work, please check the output log of that command. >> It should say something. >> >> 在 2017年10月17日 下午3:39，yuyong.zhai <[email protected]>写道： >> >>> >>> sh metastore.sh clean --delete true >>> >>> sh kylin.sh org.apache.kylin.storage.hbase.util.StorageCleanupJob >>> --delete true >>> 原始邮件 >>> *发件人:* 曾耀武<[email protected]> >>> *收件人:* user<[email protected]> >>> *发送时间:* 2017年10月17日(周二) 15:23 >>> *主题:* Re: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 >>> >>> 这个我看了,执行之后发现只是删除了 hive 中间表 kylin_intermediate_* 任务失败遗留的中间表 >>> 和 hdfs://nameservice1:8020/kylin/kylin-kylin_metadata/ 目录下没用的临时目录， >>> hbase 里的 segment 表仍然存在的 >>> >>> 发件人: "yuyong.zhai" <[email protected]> >>> 答复: user <[email protected]> >>> 日期: 2017年10月17日 星期二 下午3:04 >>> 至: user <[email protected]> >>> 主题: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 >>> >>> cleanup storage >>> >>> http://kylin.apache.org/docs15/howto/howto_cleanup_storage.html >>> >>> 原始邮件 >>> *发件人:* 曾耀武<[email protected]> >>> *收件人:* user<[email protected]> >>> *发送时间:* 2017年10月17日(周二) 14:02 >>> *主题:* kylin drop cube segment Htable 仍然存在 >>> >>> >>> >>> 嗨，all ，在清理没用的cube （kylin2.1.0) 时，发现cube 被purge和drop 之后，对应的segment hbase >>> 表仍然在 >>> base中存在，只是在kylinmeta 表的里的元信息被清除， 这是我的特例还是为了数据安全特意让他存在， >>> 只能人工删除相关的表呢？ >>> >>> >>> Best regards >>> >>> >> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > >
