hit-lacus commented on pull request #1732: URL: https://github.com/apache/kylin/pull/1732#issuecomment-900466160
## Dry run and check ```sh [root@cdh-master apache-kylin-4.0.0-SNAPSHOT-bin]# bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --cleanupJobTmp true --cleanupThreshold 0 Retrieving hive dependency... Retrieving hadoop conf dir... Retrieving Spark dependency... Skip spark which not owned by kylin. SPARK_HOME is /root/xiaoxiang.yu/spark-2.4.6-bin-hadoop2.7 and KYLIN_HOME is /root/xiaoxiang.yu/release_manager/release-candidate/storage_cleanup_tool/apache-kylin-4.0.0-SNAPSHOT-bin . Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/root/xiaoxiang.yu/release_manager/release-candidate/storage_cleanup_tool/apache-kylin-4.0.0-SNAPSHOT-bin/tool/kylin-tool-4.0.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2021-08-18 00:54:41,334 INFO [main] common.KylinConfig:117 : Loading kylin-defaults.properties from file:/root/xiaoxiang.yu/release_manager/release-candidate/storage_cleanup_tool/apache-kylin-4.0.0-SNAPSHOT-bin/tool/kylin-tool-4.0.0-SNAPSHOT.jar!/kylin-defaults.properties 2021-08-18 00:54:41,350 INFO [main] common.KylinConfig:369 : Use KYLIN_HOME=/root/xiaoxiang.yu/release_manager/release-candidate/storage_cleanup_tool/apache-kylin-4.0.0-SNAPSHOT-bin 2021-08-18 00:54:41,404 INFO [main] common.KylinConfigBase:263 : Kylin Config was updated with kylin.metadata.url.identifier : xiaoxiangyu_kylin400_cdh5_spark2 2021-08-18 00:54:41,411 INFO [main] common.KylinConfigBase:263 : Kylin Config was updated with kylin.log.spark-executor-properties-file : /root/xiaoxiang.yu/release_manager/release-candidate/storage_cleanup_tool/apache-kylin-4.0.0-SNAPSHOT-bin/conf/spark-executor-log4j.properties 2021-08-18 00:54:41,411 INFO [main] common.KylinConfig:160 : Initialized a new KylinConfig from getInstanceFromEnv : 1690254271 2021-08-18 00:54:42,361 WARN [main] util.NativeCodeLoader:62 : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2021-08-18 00:54:42,397 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.job.execution.ExecutableManager 2021-08-18 00:54:42,407 INFO [main] execution.ExecutableManager:92 : Using metadata url: xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 2021-08-18 00:54:42,411 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.job.dao.ExecutableDao 2021-08-18 00:54:42,413 INFO [main] dao.ExecutableDao:83 : Using metadata url: xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 2021-08-18 00:54:42,415 INFO [main] persistence.ResourceStore:90 : Using metadata url xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 for resource store 2021-08-18 00:54:42,421 INFO [main] persistence.JDBCConnectionManager:92 : Connecting to Jdbc with url:jdbc:mysql://cdh-master:3306/kylin_test by user kylin Wed Aug 18 00:54:42 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. Wed Aug 18 00:54:43 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2021-08-18 00:54:43,052 INFO [main] persistence.JDBCConnectionManager:65 : Connected to MySQL 5.7.26 2021-08-18 00:54:43,069 INFO [main] persistence.JDBCResourceStore:115 : Table [xiaoxiangyu_kylin400_cdh5_spark2] already exists 2021-08-18 00:54:43,072 INFO [main] persistence.JDBCResourceStore:115 : Table [xiaoxiangyu_kylin400_cdh5_spark2_log] already exists 2021-08-18 00:54:43,326 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster Running org.apache.kylin.rest.job.StorageCleanupJob --cleanupJobTmp true --cleanupThreshold 0 2021-08-18 00:54:43,337 INFO [main] job.StorageCleanupJob:143 : options: ' cleanupJobTmp=true cleanupThreshold=0' 2021-08-18 00:54:43,337 INFO [main] job.StorageCleanupJob:160 : =================================================================== delete : false; cleanupTableSnapshot : true; cleanupGlobalDict : true; cleanupJobTmp : true; cleanBeforeDate : Wed Aug 18 00:54:43 CST 2021. 2021-08-18 00:54:43,340 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.metadata.project.ProjectManager 2021-08-18 00:54:43,342 INFO [main] project.ProjectManager:81 : Initializing ProjectManager with metadata url xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 2021-08-18 00:54:43,362 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.cube.CubeManager 2021-08-18 00:54:43,368 INFO [main] cube.CubeManager:122 : Initializing CubeManager with config xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 2021-08-18 00:54:43,415 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.cube.CubeDescManager 2021-08-18 00:54:43,415 INFO [main] cube.CubeDescManager:91 : Initializing CubeDescManager with config xiaoxiangyu_kylin400_cdh5_spark2@jdbc,url=jdbc:mysql://cdh-master:3306/kylin_test,username=kylin,password=kylin,maxActive=10,maxIdle=10 2021-08-18 00:54:43,467 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.metadata.model.DataModelManager 2021-08-18 00:54:43,472 INFO [main] common.KylinConfig:493 : Creating new manager instance of class org.apache.kylin.metadata.TableMetadataManager 2021-08-18 00:54:43,608 INFO [main] model.RowKeyColDesc:99 : Implicitly convert encoding to time 2021-08-18 00:54:43,608 INFO [main] model.RowKeyColDesc:99 : Implicitly convert encoding to time 2021-08-18 00:54:43,608 INFO [main] model.RowKeyColDesc:95 : Implicitly convert encoding to date 2021-08-18 00:54:43,653 INFO [main] job.StorageCleanupJob:174 : Start to clean up unreferenced projects and cubes ... 2021-08-18 00:54:43,765 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.NATION/61d57f0f-9083-41ce-bde6-cf2ced01b33a 2021-08-18 00:54:43,766 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.PART/f60af472-dff7-43d2-8563-be5037a86ad1 2021-08-18 00:54:43,766 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.PARTSUPP/c62abf37-fa33-47d3-a592-5836a964986b 2021-08-18 00:54:43,766 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.REGION/7c6941a2-4f1e-4973-a258-986dcedc9bbe 2021-08-18 00:54:43,766 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.SUPPLIER/874736a4-c0be-4d29-8004-1e861c0e4f2a 2021-08-18 00:54:43,766 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/table_snapshot/TPCH_FLAT_ORC_10.V_ORDERS/d35768aa-8f0b-497d-b87d-b2c13369d7d9 2021-08-18 00:54:43,784 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/185ee890-c1bb-4765-b52e-2aacadfee7a3 2021-08-18 00:54:43,784 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/1bbcdbab-6a02-4555-bb9d-4f569c8841d9 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/25d8e3b9-2614-45da-aa13-d7416f04f0ad 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/32420bc6-2ae7-495b-a377-413969203e79 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/5929ccb4-9b4b-422f-9449-922daece4152 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/abafd9d1-4d82-4ab9-bab0-3c35a418f36c 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/ad8359c7-2365-4b3c-b744-a8ecbf351383 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/af3f3cb2-a1e4-4672-82b3-7ce2896f2125 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/b3b6c19b-03c6-444c-9bb1-c49d204eceb4 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/ca753b48-037b-4d11-b42f-4d646054800a 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/cbbf9df1-840a-4158-b79c-1c9a3b91ce50 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/ce093bfd-d543-4c9a-85ce-8bfbd9a6f855 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_ACCOUNT/fd51cc46-5e1f-4acd-95bd-d32a7d4f2bbc 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CAL_DT/36dbfe24-bb40-443b-bc14-6b185b626171 2021-08-18 00:54:43,785 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CAL_DT/6e8328e7-a53d-4b6e-a326-f7641205e26f 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CAL_DT/c353efec-f059-4e6f-8487-2006b02650e0 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CAL_DT/f73de6e9-b638-4bbf-a08e-2050ff7ce962 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CATEGORY_GROUPINGS/12da5414-f852-4010-9b23-03d66b629822 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CATEGORY_GROUPINGS/f2e91f69-dc68-4c57-a807-6512b3694135 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CATEGORY_GROUPINGS/fb22d867-3dca-4a61-993f-173963fa9fd8 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_CATEGORY_GROUPINGS/ffe78093-b3e4-4c89-8ae4-e0c8a496d475 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/0705ddcd-932d-4eef-a3fd-5b0c194fd4b7 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/20cabfd8-40a3-4813-a22b-066b50041ade 2021-08-18 00:54:43,786 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/2fa69a7c-0942-482b-b39f-7526f46a49d3 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/3ae3faa0-451d-4241-8380-594a42727f22 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/619bb512-c866-4b4e-a0ce-ace081c52d6c 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/742b5df1-f222-4c26-b6e2-bab966513d08 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/80e7a79b-3131-4023-9047-1e56a2679362 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/831fb9d3-a8fc-4596-8435-ad3f6cdd3362 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/8be2bb7c-1d45-4f3d-a9e0-8882bee13830 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/9117e741-94f0-4339-b88d-1a65450b23eb 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/cd4d92d3-aef4-4ce3-af5b-c136f246570c 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/d410aaec-949e-40d6-8c29-6cb07f1a16e9 2021-08-18 00:54:43,787 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path TABLE_SNAPSHOT, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/table_snapshot/DEFAULT.KYLIN_COUNTRY/f2306b38-eed3-4e42-96c9-75a0875a0b85 2021-08-18 00:54:43,790 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path CUBE_DIR, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/DELETE_CUBE 2021-08-18 00:54:43,790 INFO [main] job.StorageCleanupJob:200 : Start to clean up no unreferenced segments ... 2021-08-18 00:54:43,793 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/ACOUNT_C2 2021-08-18 00:54:43,799 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path SEGMENT_DIR, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/ERROR_CUBE/20120101000000_20130405000000_TGC 2021-08-18 00:54:43,801 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/kylin_sales_cube_01 2021-08-18 00:54:43,802 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/kylin_sales_cube_02 2021-08-18 00:54:43,803 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/kylin_sales_cube_03 2021-08-18 00:54:43,805 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/kylin_streaming_cube 2021-08-18 00:54:43,807 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path SEGMENT_DIR, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/SSB_20_PRJ/parquet/LICube_01/19920101000000_19940101000000_KMO 2021-08-18 00:54:43,807 WARN [main] job.StorageCleanupJob:222 : Cube path doesn't exist! The path is hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/PAUSE_CUBE 2021-08-18 00:54:43,809 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path SEGMENT_DIR, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/parquet/PURGE_CUBE/20120101000000_20120603000000_1GM 2021-08-18 00:54:43,809 INFO [main] job.StorageCleanupJob:227 : Start to clean up stale job_tmp ... 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/42399513-2b0e-41a0-9601-cbe2d51071fd 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/42399513-2b0e-41a0-9601-cbe2d51071fd-00 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/42399513-2b0e-41a0-9601-cbe2d51071fd-00_jobId 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/42399513-2b0e-41a0-9601-cbe2d51071fd-01 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/42399513-2b0e-41a0-9601-cbe2d51071fd-01_jobId 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/51df7a36-502b-44b7-b4f8-e4c6aaabfa09 2021-08-18 00:54:43,811 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/51df7a36-502b-44b7-b4f8-e4c6aaabfa09-00 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/51df7a36-502b-44b7-b4f8-e4c6aaabfa09-00_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/51df7a36-502b-44b7-b4f8-e4c6aaabfa09-01 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/51df7a36-502b-44b7-b4f8-e4c6aaabfa09-01_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/61639653-7925-40d8-8b5b-81c442dbf5a4 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/61639653-7925-40d8-8b5b-81c442dbf5a4-00 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/61639653-7925-40d8-8b5b-81c442dbf5a4-00_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/61639653-7925-40d8-8b5b-81c442dbf5a4-01 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/61639653-7925-40d8-8b5b-81c442dbf5a4-01_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/62b0112e-87a0-4f39-8db7-667d06f8e208 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/62b0112e-87a0-4f39-8db7-667d06f8e208-00 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/62b0112e-87a0-4f39-8db7-667d06f8e208-00_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/62b0112e-87a0-4f39-8db7-667d06f8e208-01 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/62b0112e-87a0-4f39-8db7-667d06f8e208-01_jobId 2021-08-18 00:54:43,812 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/d7d6ce2f-4868-4166-8acc-444ddc06a824 2021-08-18 00:54:43,813 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/d7d6ce2f-4868-4166-8acc-444ddc06a824-00 2021-08-18 00:54:43,813 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/d7d6ce2f-4868-4166-8acc-444ddc06a824-00_jobId 2021-08-18 00:54:43,813 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/d7d6ce2f-4868-4166-8acc-444ddc06a824-01 2021-08-18 00:54:43,813 INFO [main] job.StorageCleanupJob:337 : Dry run, pending delete unreferenced path JOB_TMP, hdfs://cdh-master:8020/LacusDir/xiaoxiangyu/xiaoxiangyu_kylin400_cdh5_spark2/learn_kylin/job_tmp/d7d6ce2f-4868-4166-8acc-444ddc06a824-01_jobId ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
