[ https://issues.apache.org/jira/browse/KYLIN-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472053#comment-16472053 ]
ASF subversion and git services commented on KYLIN-3368: -------------------------------------------------------- Commit 18b34e52b35abe82c9df1a3d88dfff4f2e278edf in kylin's branch refs/heads/KYLIN-3368 from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=18b34e5 ] KYLIN-3368 Move Spark cubing metadata dump to job folder > "/kylin/kylin_metadata/metadata/" has many garbage for spark cubing > ------------------------------------------------------------------- > > Key: KYLIN-3368 > URL: https://issues.apache.org/jira/browse/KYLIN-3368 > Project: Kylin > Issue Type: Bug > Components: Spark Engine > Affects Versions: v2.2.0, v2.3.0 > Reporter: Shaofeng SHI > Priority: Major > > If using Spark as cube engine, Kylin will dump metadata to HDFS in > "/kylin/kylin_metadata/metadata/"; As time goes on, many files were left > there. > > They should be deleted when the job is finished. > > hadoop fs -ls /kylin/kylin_metadata/metadata > Found 583 items > drwxr-xr-x - root hdfs 0 2018-02-11 16:20 > /kylin/kylin_metadata/metadata/007d8b6c-5db1-478f-a373-7fd57231494b > drwxr-xr-x - root hdfs 0 2018-02-21 08:24 > /kylin/kylin_metadata/metadata/00f93335-0d85-4c19-953c-5e4660f4d58e > drwxr-xr-x - root hdfs 0 2018-03-21 10:17 > /kylin/kylin_metadata/metadata/011d3c7b-e707-42ca-b355-a0037820159b > drwxr-xr-x - root hdfs 0 2018-04-07 00:41 > /kylin/kylin_metadata/metadata/0263789f-969a-485f-aa6c-df4394a45df8 > drwxr-xr-x - root hdfs 0 2018-02-23 00:24 > /kylin/kylin_metadata/metadata/02977825-92cd-475d-8a3f-0f75255a9ba9 > drwxr-xr-x - root hdfs 0 2018-02-15 08:24 > /kylin/kylin_metadata/metadata/02e058fd-ac6a-451f-99b4-f0b03ad6e3a5 > drwxr-xr-x - root hdfs 0 2018-03-15 10:21 > /kylin/kylin_metadata/metadata/0337c6c9-aebb-4ae0-8485-9905283fc818 > drwxr-xr-x - root hdfs 0 2018-02-23 12:22 > /kylin/kylin_metadata/metadata/03d7f34f-10dc-4329-9dde-f646cc040ba9 > drwxr-xr-x - root hdfs 0 2018-04-10 12:20 > /kylin/kylin_metadata/metadata/03fa7654-7b14-4bda-aed8-64ac9d08081a > drwxr-xr-x - root hdfs 0 2018-03-30 00:23 > /kylin/kylin_metadata/metadata/0406e5bb-112f-4c54-bdf8-7b1e34652522 > drwxr-xr-x - root hdfs 0 2018-03-24 00:33 > /kylin/kylin_metadata/metadata/0426907d-541f-4f23-9db7-9a965f5b6739 > drwxr-xr-x - root hdfs 0 2018-02-07 15:03 > /kylin/kylin_metadata/metadata/04771302-2090-4793-956d-bf5ba9360d9b > drwxr-xr-x - root hdfs 0 2018-03-30 00:20 > /kylin/kylin_metadata/metadata/0482b928-6b09-48d1-909f-68c66f71a827 > drwxr-xr-x - root hdfs 0 2018-03-04 16:36 > /kylin/kylin_metadata/metadata/04943bbd-8425-4304-b9c8-cc13a55e8ac1 > drwxr-xr-x - root hdfs 0 2018-03-05 08:35 > /kylin/kylin_metadata/metadata/04ee2c3d-bd07-4e7a-90db-3d87d088ff7d > drwxr-xr-x - root hdfs 0 2018-03-02 16:24 > /kylin/kylin_metadata/metadata/05808a9d-2e55-4dc3-8c28-5b662068dda8 > drwxr-xr-x - root hdfs 0 2018-03-04 08:35 > /kylin/kylin_metadata/metadata/05c63d7c-4ea5-48fc-897d-6db9d78ffaf7 > drwxr-xr-x - root hdfs 0 2018-03-01 01:02 > /kylin/kylin_metadata/metadata/05ea3568-e85f-452a-b44d-4657145a724a -- This message was sent by Atlassian JIRA (v7.6.3#76005)