Hi
My test hadoop storage is almost full, so I use StorageCleanupJob to clear
the unused data, but got the following error, please help analyze it and give
me some suggestions, My kylin has upgraded from 1.6 to 2.0 now. Thanks.
2017-07-03 16:33:36,111 DEBUG [main MetadataManager:384]: Loaded 36 SourceTable
EXD(s)
2017-07-03 16:33:36,111 DEBUG [main MetadataManager:529]: Reloading DataModel
from folder kylin_metadata(key='/model_desc')@kylin_metadata@hbase
2017-07-03 16:33:36,184 ERROR [main MetadataManager:538]: Error to load
DataModel at /model_desc/test_model.json
java.lang.IllegalStateException: Error to load /model_desc/test_model.json
at
org.apache.kylin.metadata.MetadataManager.reloadDataModelDescAt(MetadataManager.java:558)
at
org.apache.kylin.metadata.MetadataManager.reloadAllDataModel(MetadataManager.java:536)
at
org.apache.kylin.metadata.MetadataManager.init(MetadataManager.java:284)
at
org.apache.kylin.metadata.MetadataManager.<init>(MetadataManager.java:113)
at
org.apache.kylin.metadata.MetadataManager.getInstance(MetadataManager.java:83)
at org.apache.kylin.job.dao.ExecutableDao.<init>(ExecutableDao.java:76)
at
org.apache.kylin.job.dao.ExecutableDao.getInstance(ExecutableDao.java:57)
at
org.apache.kylin.job.execution.ExecutableManager.<init>(ExecutableManager.java:55)
at
org.apache.kylin.job.execution.ExecutableManager.getInstance(ExecutableManager.java:64)
at
org.apache.kylin.storage.hbase.util.StorageCleanupJob.<clinit>(StorageCleanupJob.java:78)
Caused by: java.lang.IllegalArgumentException: Column not found by
KYLIN_SALES.USER_ID
at
org.apache.kylin.metadata.model.DataModelDesc.findColumn(DataModelDesc.java:240)
at
org.apache.kylin.metadata.model.ModelDimensionDesc.init(ModelDimensionDesc.java:66)
at
org.apache.kylin.metadata.model.DataModelDesc.initDimensionsAndMetrics(DataModelDesc.java:381)
at
org.apache.kylin.metadata.model.DataModelDesc.init(DataModelDesc.java:302)
at
org.apache.kylin.metadata.MetadataManager.reloadDataModelDescAt(MetadataManager.java:554)
... 9 more
2017-07-03 16:33:36,194 DEBUG [main MetadataManager:543]: Loaded 8 DataModel(s)
……
2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking table OK
2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking table Time
taken: 1.566 seconds
2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking table OK
2017-07-03 16:33:43,335 INFO [main StorageCleanupJob:283]: Checking table
kylin_intermediate_cube_dim_order_one_6b1de133_a10b_4bb6_86a7_f249e6d0c418
Exception in thread "main" java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.util.StorageCleanupJob
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
org.apache.kylin.storage.hbase.util.StorageCleanupJob.main(StorageCleanupJob.java:362)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.common.util.ClassUtil.forRenamedClass(ClassUtil.java:84)
at org.apache.kylin.common.util.ClassUtil.forName(ClassUtil.java:76)
at
org.apache.kylin.job.execution.ExecutableManager.parseTo(ExecutableManager.java:439)
at
org.apache.kylin.job.execution.ExecutableManager.getJob(ExecutableManager.java:129)
at
org.apache.kylin.storage.hbase.util.StorageCleanupJob.isTableInUse(StorageCleanupJob.java:347)
at
org.apache.kylin.storage.hbase.util.StorageCleanupJob.cleanUnusedIntermediateHiveTable(StorageCleanupJob.java:302)
at
org.apache.kylin.storage.hbase.util.StorageCleanupJob.execute(StorageCleanupJob.java:153)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 1 more
2017-07-03 16:33:43,342 INFO [Thread-0
ConnectionManager$HConnectionImplementation:2259]: Closing master protocol:
MasterService
2017-07-03 16:33:43,368 INFO [Thread-0
ConnectionManager$HConnectionImplementation:1830]: Closing zookeeper
sessionid=0x25cbe7e555d9dc5
2017-07-03 16:33:43,371 INFO [Thread-0 ZooKeeper:684]: Session:
0x25cbe7e555d9dc5 closed
2017-07-03 16:33:43,371 INFO [main-EventThread ClientCnxn:512]: EventThread
shut down
Best regards.
