A known issue, is fixed on master branch. Thanks for reporting! On Wed, Jun 28, 2017 at 10:16 AM, zhongyuhai <[email protected]> wrote:
> hello > > > when I run command as follow > > > kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true > > > and I got the exception msg as follow : > > > 2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:145]: options: ' > delete=true' > 2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:146]: delete option > value: 'true' > 2017-06-28 10:08:17,432 INFO [main StorageCleanupJob:147]: force option > value: 'null' > 2017-06-28 10:08:17,759 WARN [main HiveConf:3683]: HiveConf of name > hive.multigroupby.singlemar does not exist > 2017-06-28 10:08:17,872 INFO [main HiveMetaStore:564]: 0: Opening raw > store > with implementation class:org.apache.hadoop.hive.metastore.ObjectStore > 2017-06-28 10:08:17,956 INFO [main ObjectStore:325]: ObjectStore, > initialize called > 2017-06-28 10:08:18,208 INFO [main Persistence:77]: Property > datanucleus.fixedDatastore unknown - will be ignored > 2017-06-28 10:08:18,209 INFO [main Persistence:77]: Property > hive.metastore.integral.jdo.pushdown unknown - will be ignored > 2017-06-28 10:08:18,209 INFO [main Persistence:77]: Property > datanucleus.cache.level2 unknown - will be ignored > 2017-06-28 10:08:19,617 WARN [main HiveConf:3683]: HiveConf of name > hive.multigroupby.singlemar does not exist > 2017-06-28 10:08:19,619 INFO [main ObjectStore:429]: Setting MetaStore > object pin classes with > hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo, > Partition,Database,Type,FieldSchema,Order" > 2017-06-28 10:08:21,953 INFO [main MetaStoreDirectSql:145]: Using direct > SQL, underlying DB is MYSQL > 2017-06-28 10:08:21,957 INFO [main ObjectStore:308]: Initialized > ObjectStore > 2017-06-28 10:08:22,455 INFO [main HiveMetaStore:651]: Added admin role in > metastore > 2017-06-28 10:08:22,459 INFO [main HiveMetaStore:660]: Added public role > in > metastore > 2017-06-28 10:08:22,532 INFO [main HiveMetaStore:700]: No user is added in > admin role, since config is empty > 2017-06-28 10:08:22,808 INFO [main HiveMetaStore:734]: 0: get_all_tables: > db=default > 2017-06-28 10:08:22,811 INFO [main audit:295]: ugi=root > ip=unknown-ip-addr cmd=get_all_tables: db=default > 2017-06-28 10:08:22,947 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 0363e422-1be2-4652-89e5-24a727ce98ba, ignore it > 2017-06-28 10:08:22,954 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 05990f76-b2e7-463c-bd5f-9b6906db4a63, ignore it > 2017-06-28 10:08:22,958 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 198969b3-dd27-4bd6-b5eb-476d963cc59c, ignore it > 2017-06-28 10:08:22,964 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 1f97122e-1e38-4007-a8a2-705b8af1521a, ignore it > 2017-06-28 10:08:22,969 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 39f59489-aaea-43a6-be2e-436447d67dae, ignore it > 2017-06-28 10:08:22,974 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 42de30ea-82a6-455c-9cbc-2c6df04812cd, ignore it > 2017-06-28 10:08:22,978 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 6849dbc5-6a48-41e3-b258-b248c77e0a52, ignore it > 2017-06-28 10:08:22,983 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 6c9aa465-ae82-402f-ba7b-cbbac2adca3f, ignore it > 2017-06-28 10:08:22,989 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 82994592-ff8b-4863-aa42-dc0bab66d47d, ignore it > 2017-06-28 10:08:22,994 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 84cc3f86-29fc-438a-be45-5ec46792c568, ignore it > 2017-06-28 10:08:22,998 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 90219e38-3b2a-4ec8-8d3a-7c86930d5caa, ignore it > 2017-06-28 10:08:23,003 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID 9dc10d6b-d1b4-4470-adf2-dedbe1f0f90e, ignore it > 2017-06-28 10:08:23,008 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID b4d99f1d-bc13-45eb-8433-84de238279a1, ignore it > 2017-06-28 10:08:23,013 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID c0f24c74-44f9-4171-b56e-8056d19e3ae1, ignore it > 2017-06-28 10:08:23,017 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID ca7a8143-e676-4c9f-8ffe-fdadcc84724a, ignore it > 2017-06-28 10:08:23,024 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID dd8f4ea5-5c3c-4165-88e1-f06a796ca095, ignore it > 2017-06-28 10:08:23,029 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID e57934e1-1398-457f-9070-6bc5010f461a, ignore it > 2017-06-28 10:08:23,032 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID e9a845c5-5aba-4c83-a589-99984b1b07ee, ignore it > 2017-06-28 10:08:23,037 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID eadcc797-b9ef-439c-891c-9ac912cc7ce6, ignore it > 2017-06-28 10:08:23,043 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID ef9e5b30-d8fa-498d-ad38-48494a15fb22, ignore it > 2017-06-28 10:08:23,048 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID f1c5089a-b6e1-4118-b58e-48164833de91, ignore it > 2017-06-28 10:08:23,052 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID f469d883-d066-4f48-ad4a-098dede4a74b, ignore it > 2017-06-28 10:08:23,056 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID f4ddb713-be4e-4c30-a24a-645d87847f01, ignore it > 2017-06-28 10:08:23,060 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID f6579632-3b05-4546-a826-96e761c4e678, ignore it > 2017-06-28 10:08:23,065 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID f9ae3fd4-9f74-4c59-9b47-d604388e1970, ignore it > 2017-06-28 10:08:23,069 WARN [main StorageCleanupJob:285]: Failed to find > segment ID from job ID fbe1e38d-c4f0-47ad-a93a-7489f5638cdf, ignore it > 2017-06-28 10:08:23,069 INFO [main StorageCleanupJob:289]: Working jobIDs: > [0363e422-1be2-4652-89e5-24a727ce98ba, 05990f76-b2e7-463c-bd5f- > 9b6906db4a63, > 198969b3-dd27-4bd6-b5eb-476d963cc59c, 1f97122e-1e38-4007-a8a2- > 705b8af1521a, > 39f59489-aaea-43a6-be2e-436447d67dae, 42de30ea-82a6-455c-9cbc- > 2c6df04812cd, > 6849dbc5-6a48-41e3-b258-b248c77e0a52, 6c9aa465-ae82-402f-ba7b- > cbbac2adca3f, > 82994592-ff8b-4863-aa42-dc0bab66d47d, 84cc3f86-29fc-438a-be45- > 5ec46792c568, > 90219e38-3b2a-4ec8-8d3a-7c86930d5caa, 9dc10d6b-d1b4-4470-adf2- > dedbe1f0f90e, > b4d99f1d-bc13-45eb-8433-84de238279a1, c0f24c74-44f9-4171-b56e- > 8056d19e3ae1, > ca7a8143-e676-4c9f-8ffe-fdadcc84724a, dd8f4ea5-5c3c-4165-88e1- > f06a796ca095, > e57934e1-1398-457f-9070-6bc5010f461a, e9a845c5-5aba-4c83-a589- > 99984b1b07ee, > eadcc797-b9ef-439c-891c-9ac912cc7ce6, ef9e5b30-d8fa-498d-ad38- > 48494a15fb22, > f1c5089a-b6e1-4118-b58e-48164833de91, f469d883-d066-4f48-ad4a- > 098dede4a74b, > f4ddb713-be4e-4c30-a24a-645d87847f01, f6579632-3b05-4546-a826- > 96e761c4e678, > f9ae3fd4-9f74-4c59-9b47-d604388e1970, fbe1e38d-c4f0-47ad-a93a- > 7489f5638cdf] > 2017-06-28 10:08:23,096 INFO [main StorageCleanupJob:292]: Checking table > kylin_intermediate_biam_busi_anl_day_kind_p_29fd1441_f7c4_ > 4625_af15_58a39cbdd08a > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.tool.StorageCleanupJob > at > org.apache.kylin.common.util.AbstractApplication.execute( > AbstractApplication.java:42) > at > org.apache.kylin.tool.StorageCleanupJob.main(StorageCleanupJob.java:394) > Caused by: java.lang.NullPointerException > at > org.apache.kylin.common.util.ClassUtil.forRenamedClass(ClassUtil.java:84) > at org.apache.kylin.common.util.ClassUtil.forName(ClassUtil. > java:76) > at > org.apache.kylin.job.execution.ExecutableManager. > parseTo(ExecutableManager.java:439) > at > org.apache.kylin.job.execution.ExecutableManager. > getJob(ExecutableManager.java:129) > at > org.apache.kylin.tool.StorageCleanupJob.getSegmentIdFromJobId( > StorageCleanupJob.java:375) > at > org.apache.kylin.tool.StorageCleanupJob.isTableInUse( > StorageCleanupJob.java:382) > at > org.apache.kylin.tool.StorageCleanupJob.cleanUnusedIntermediateHiveTab > le(StorageCleanupJob.java:311) > at > org.apache.kylin.tool.StorageCleanupJob.execute( > StorageCleanupJob.java:153) > at > org.apache.kylin.common.util.AbstractApplication.execute( > AbstractApplication.java:37) > ... 1 more > 2017-06-28 10:08:23,110 INFO [Thread-0 > ConnectionManager$HConnectionImplementation:2139]: Closing master > protocol: > MasterService > 2017-06-28 10:08:23,142 INFO [Thread-0 > ConnectionManager$HConnectionImplementation:1710]: Closing zookeeper > sessionid=0x685caabe66b20097 > 2017-06-28 10:08:23,146 INFO [Thread-0 ZooKeeper:684]: Session: > 0x685caabe66b20097 closed > 2017-06-28 10:08:23,146 INFO [main-EventThread ClientCnxn:512]: > EventThread > shut down > > > > My software version as follow: > > > hadoop-2.6.0-cdh5.4.0 > hbase-1.2.3-bin > apache-hive-2.1.0-bin > kylin.version:2.0.0 > commit:375fd807c281d8c5deff0620747c806be2019782; > os.name:Linux > os.arch:amd64 > os.version:3.10.0-229.el7.x86_64 > java.version:1.8.0_102 > java.vendor:Oracle Corporation > > > Thank you ! > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/errors-when-run-StorageCleanupJob-tp8321.html > Sent from the Apache Kylin mailing list archive at Nabble.com. >
