Hi Yaowu, Could you please update this patch in KYLIN-2846? Storage cleanup should also support customized namespace. Thanks.
2017-10-18 20:37 GMT+08:00 曾耀武 <zeng.ya...@immomo.com>: > > Yeah I have checked it was a bug , because I used hbase namespace “kylin” > with the patch "https://issues.apache.org/jira/secure/attachment/ > 12885320/KYLIN-2846-001.patch” > > And I modified the code as below and it works well: > > > MOMO@MOMOdeMacBook-Pro-5:~/gitworkspace/kylin2.0/kylin/ > dist/apache-kylin-2.1.0-bin/tool$ git diff ../../../server-base/src/main/ > java/org/apache/kylin/rest/job/StorageCleanJobHbaseUtil.java > diff --git > a/server-base/src/main/java/org/apache/kylin/rest/job/StorageCleanJobHbaseUtil.java > b/server-base/src/main/java/org/apache/kylin/rest/job/ > StorageCleanJobHbaseUtil.java > index 3728ea1..937d02d 100644 > --- a/server-base/src/main/java/org/apache/kylin/rest/job/ > StorageCleanJobHbaseUtil.java > +++ b/server-base/src/main/java/org/apache/kylin/rest/job/ > StorageCleanJobHbaseUtil.java > @@ -46,15 +46,28 @@ public class StorageCleanJobHbaseUtil { > > public static void cleanUnusedHBaseTables(boolean delete, int > deleteTimeout) throws IOException { > Configuration conf = HBaseConfiguration.create(); > - CubeManager cubeMgr = CubeManager.getInstance(KylinConfig. > getInstanceFromEnv()); > + KylinConfig config = KylinConfig.getInstanceFromEnv(); > + CubeManager cubeMgr = CubeManager.getInstance(config); > // get all kylin hbase tables > try (HBaseAdmin hbaseAdmin = new HBaseAdmin(conf)) { > - String tableNamePrefix = IRealizationConstants. > SharedHbaseStorageLocationPrefix; > + String namespace = config.getHBaseStorageNameSpace(); > + StringBuffer sb = new StringBuffer(); > + String tableNamePrefix = null; > + if(namespace.equals("default")){ > + tableNamePrefix = IRealizationConstants. > SharedHbaseStorageLocationPrefix; > + }else{ > + sb.append(config.getHBaseStorageNameSpace()).append(":"); > + sb.append(IRealizationConstants. > SharedHbaseStorageLocationPrefix); > + tableNamePrefix = sb.toString(); > + } > + > > > > > > 发件人: ShaoFeng Shi <shaofeng...@apache.org> > 答复: user <user@kylin.apache.org> > 日期: 2017年10月17日 星期二 下午4:16 > 至: user <user@kylin.apache.org> > 主题: [外部邮件] Re: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 > > Hi yaowu, > > The StorageCleanupJob will delete the hbase tables that has no reference; > If you found it didn't work, please check the output log of that command. > It should say something. > > 在 2017年10月17日 下午3:39，yuyong.zhai <yuyong.z...@ele.me>写道： > >> >> sh metastore.sh clean --delete true >> >> sh kylin.sh org.apache.kylin.storage.hbase.util.StorageCleanupJob >> --delete true >> 原始邮件 >> *发件人:* 曾耀武<zeng.ya...@immomo.com> >> *收件人:* user<user@kylin.apache.org> >> *发送时间:* 2017年10月17日(周二) 15:23 >> *主题:* Re: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 >> >> 这个我看了,执行之后发现只是删除了 hive 中间表 kylin_intermediate_* 任务失败遗留的中间表 >> 和 hdfs://nameservice1:8020/kylin/kylin-kylin_metadata/ 目录下没用的临时目录， >> hbase 里的 segment 表仍然存在的 >> >> 发件人: "yuyong.zhai" <yuyong.z...@ele.me> >> 答复: user <user@kylin.apache.org> >> 日期: 2017年10月17日 星期二 下午3:04 >> 至: user <user@kylin.apache.org> >> 主题: [外部邮件] 回复：kylin drop cube segment Htable 仍然存在 >> >> cleanup storage >> >> http://kylin.apache.org/docs15/howto/howto_cleanup_storage.html >> >> 原始邮件 >> *发件人:* 曾耀武<zeng.ya...@immomo.com> >> *收件人:* user<user@kylin.apache.org> >> *发送时间:* 2017年10月17日(周二) 14:02 >> *主题:* kylin drop cube segment Htable 仍然存在 >> >> >> >> 嗨，all ，在清理没用的cube （kylin2.1.0) 时，发现cube 被purge和drop 之后，对应的segment hbase >> 表仍然在 >> base中存在，只是在kylinmeta 表的里的元信息被清除， 这是我的特例还是为了数据安全特意让他存在， >> 只能人工删除相关的表呢？ >> >> >> Best regards >> >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Best regards, Shaofeng Shi 史少锋
