I can not load the files to this JIRA, so hier they are.
Jie
Am 22.06.2016 um 08:59 schrieb Jie Tao (JIRA):
Jie Tao created KYLIN-1813:
------------------------------
Summary: intermediate table in Hive not cleaned up
Key: KYLIN-1813
URL: https://issues.apache.org/jira/browse/KYLIN-1813
Project: Kylin
Issue Type: Bug
Components: General
Affects Versions: v1.5.2
Environment: linux
Reporter: Jie Tao
Priority: Minor
after the cube building with ERROR I discarded the cube, but the cube is not marked as
discarded like other jobs that I discarded during the cube building wa srunning (see
job.png: the job is not black colour marked but has only one choice
"diagnose"). Hence, as I manually clean up intermediate data, I got info like:
Remove intermediate hive table with job id 493fd20b-3074-403e-9963-fe4fb7ff7c65
with job status ERROR
2016-06-17 09:37:12,648 INFO [main StorageCleanupJob:262]: Remove
intermediate hive table with job id 8a377e30-e3ba-4fe2-be12-e7d412afec5ewith
job status ERROR
In this case the intermediate tables were not removed from hive (see hive.png).
The json is also attached json.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
{
"uuid": "4974bc14-b792-4fd6-910d-598e6a16bb6d",
"version": "1.5.2",
"name": "logout_full_cube",
"description": "",
"dimensions": [
{
"name": "RAWDATA.USER_LOGOUT_FULL.GAME",
"table": "RAWDATA.USER_LOGOUT_FULL",
"column": "GAME",
"derived": null
},
{
"name": "RAWDATA.USER_LOGOUT_FULL.LANG",
"table": "RAWDATA.USER_LOGOUT_FULL",
"column": "LANG",
"derived": null
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1",
"next_parameter": null
},
"returntype": "bigint"
},
"dependent_measure_ref": null
},
{
"name": "TOTAL_PLAYTIME",
"function": {
"expression": "SUM",
"parameter": {
"type": "column",
"value": "PLAYTIME",
"next_parameter": null
},
"returntype": "bigint"
},
"dependent_measure_ref": null
},
{
"name": "TOP_PLAYER",
"function": {
"expression": "TOP_N",
"parameter": {
"type": "column",
"value": "PLAYTIME",
"next_parameter": {
"type": "column",
"value": "USER_ID",
"next_parameter": null
}
},
"returntype": "topn(100)"
},
"dependent_measure_ref": null
}
],
"rowkey": {
"rowkey_columns": [
{
"column": "GAME",
"encoding": "dict",
"isShardBy": false
},
{
"column": "LANG",
"encoding": "dict",
"isShardBy": false
}
]
},
"signature": "jM6BX7iZE3oHEN+aw0tXaw==",
"last_modified": 1465892097992,
"model_name": "logout_full",
"null_string": null,
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_",
"TOTAL_PLAYTIME",
"TOP_PLAYER"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"GAME",
"LANG"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [],
"joint_dims": []
}
}
],
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 3153600000000,
"auto_merge_time_ranges": [
604800000,
2419200000
],
"retention_range": 0,
"engine_type": 2,
"storage_type": 2,
"override_kylin_properties": {}
}
