Shaofeng SHI created KYLIN-3142:
-----------------------------------
Summary: Too many temp partition files left in Hadoop /tmp folder
Key: KYLIN-3142
URL: https://issues.apache.org/jira/browse/KYLIN-3142
Project: Kylin
Issue Type: Bug
Components: Job Engine
Reporter: Shaofeng SHI
Assignee: Shaofeng SHI
I have a cluster running several streaming cubes; After months running, there
are many files left in Hadoop /tmp folder, and it has impacted on the DFS's
performance. After cleanup it, the cluster is much stable than before.
hadoop fs -ls /tmp/
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/hdp/2.5.4.0-121/hadoop/lib/alluxio-core-client-runtime-1.6.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/hdp/2.5.4.0-121/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/hdp/2.5.4.0-121/hadoop/alluxio-core-client-runtime-1.6.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Found 46 items
-rw-r--r-- 1 root supergroup 153 2017-12-29 11:33
/tmp/partitions_02e317ac-03de-4cec-8906-6b18bc974858
-rw-r--r-- 1 root supergroup 153 2017-12-29 07:03
/tmp/partitions_03108f0b-7873-4610-9346-e1c12388b229
-rw-r--r-- 1 root supergroup 153 2017-12-29 03:00
/tmp/partitions_032d22ed-2ed3-401f-8601-b4b9d6f7800b
-rw-r--r-- 1 root supergroup 153 2017-12-29 02:41
/tmp/partitions_0869a983-4b20-40b7-a1f9-4ca9413ecd8d
-rw-r--r-- 1 root supergroup 153 2017-12-29 04:03
/tmp/partitions_10c47970-d4c2-4af1-819b-93717591ff5d
-rw-r--r-- 1 root supergroup 153 2017-12-29 06:03
/tmp/partitions_172e6684-f825-4101-8860-06aaad920c59
-rw-r--r-- 1 root supergroup 153 2017-12-29 03:12
/tmp/partitions_249a224d-101d-4eff-b223-5453865e34b8
-rw-r--r-- 1 root supergroup 153 2017-12-29 03:01
/tmp/partitions_24b74354-3b5d-44ad-984d-d9d7849988f9
-rw-r--r-- 1 root supergroup 1341 2017-12-29 03:08
/tmp/partitions_27c0089b-ed0f-4a37-af4a-adc686203ac3
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
