please make it happen! On Thu, May 14, 2015 at 10:12 PM, ShaoFeng Shi <[email protected]> wrote:
> excellent, I like it; thanks George! > > 2015-05-14 21:23 GMT+08:00 nichunen <[email protected]>: > > > Hi, > > > > We found that after calling the deleteCube() rest api, the cube was > > deleted, but the data generated during the cube building process were > still > > left, listed as: > > 1. Folders begin with "kylin_job_meta" and "meta_tmp" in local machine's > > tomcat/tmp; > > 2. Tables begin with "KYLIN_" in hbase; > > 3. Folders begin with "kylin-" in hdfs; > > 4. Tables begin with "kylin_intermediate" in hive. > > Then we found we can run com.kylinolap.job.hadoop.cube.StorageCleanupJob > > to clean all the hbase and hdfs data, but seems not available for hive > > tables now. > > We add some code in StorageCleanupJob.java, made some improvement for the > > storage cleaning job, features listed as follows: > > 1. Finish the function "cleanUnusedIntermediateHiveTable(Configuration > > conf)", remove all the unused hive tables begin with > "kylin_intermediate"; > > 2. Add function "cleanUnusedHBaseTables(Configuration conf, String[] > uuids)", > > to remove hbase tables with job uuids; > > 3. Add function "cleanUnusedHdfsFiles(Configuration conf, String[] uuids) > > ", to remove hdfs data with job uuids; > > 4. Add function "cleanUnusedIntermediateHiveTable(Configuration conf, > > String[] uuids)", to remove hive tables with job uuids. > > 5. Add command line option "--file" and "--uuids", to specify the uuids > > written in a file or/and with a string. > > > > The code change is based on kylin0.6, and tested work well. I noticed > that > > the latest version of StorageCleanupJob.java filtered out the hbase > > tables related with invertedindex from the drop list, no other change, so > > I think our code can still work well on the latest kylin. > > > > May I make a pull request for our code of features? > > > > > > ------------------------------ > > > > Best Regards, > > > > > > > > George/倪春恩 > > > > Software Engineer/软件工程师 > > > > Mobile:+86-13501723787| Fax:+8610-56842040 > > > > 北京明略软件系统有限公司（MiningLamp.COM <http://www.semidata.com/>） > > > > 北京市昌平区东小口镇中东路398号中煤建设集团大厦1号楼4层 > > > > F4,1#,Zhongmei Construction Group Plaza,398# Zhongdong Road,Changping > > District,Beijing,102218 > > > > > > > ---------------------------------------------------------------------------------------------------------------------------- > > > > [image: cid:[email protected]] > > > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
