it might have something to do with the bug described in https://issues.apache.org/jira/browse/KYLIN-1828, I'll cross check
On Tue, Jun 21, 2016 at 10:36 PM, ShaoFeng Shi <[email protected]> wrote: > Hi Jie, would you mind to report a JIRA with this problem you found? If you > can attache the json of this job and a couple of screen shot, that would be > great for analysis. Thank you! > > 2016-06-21 16:49 GMT+08:00 Jie Tao <[email protected]>: > > > actually the jobs have been disgarded. Maybe it is a bug that the status > > of the job is still "ERROR". I have a look of the jobs at the Kylin Web > UI > > and found that the job was not marked with black colors like other > > discarded jobs althouth the "action" button only has one choice > > "diagonose". My Kylin is 1.5.2.1. > > > > Cheers, > > > > Jie > > > > > > Am 17.06.2016 um 11:05 schrieb ShaoFeng Shi: > > > >> by default the web UI only shows the jobs in LAST ONE WEEK, pls have a > >> check. > >> > >> 2016-06-17 16:58 GMT+08:00 Jie Tao <[email protected]>: > >> > >> actually I discarded all jobs and I do not see any ERROR job in the > >>> Monitor view of Kylin UI. > >>> > >>> Where can I see these error jobs? > >>> > >>> Jie > >>> > >>> > >>> Am 17.06.2016 um 10:31 schrieb ShaoFeng Shi: > >>> > >>> Hi Jie, > >>>> > >>>> If a job is "ERROR", the intermediate hive table of it will not be > >>>> dropped, > >>>> as "ERROR" is not a final state; User can resume an "Error" job at any > >>>> time, so Kylin skipped to cleanup for that. > >>>> > >>>> If you discard these error jobs, and re-run the cleanup, the > >>>> intermediate > >>>> hive table will be dropped. > >>>> > >>>> The message here is not clear, will change the wording... > >>>> > >>>> 2016-06-17 15:48 GMT+08:00 Jie Tao <[email protected]>: > >>>> > >>>> You are correct, the intermediate tables are left by fail-building. I > do > >>>> > >>>>> clean up storage based on the linked guide. Intermediate data in HDFS > >>>>> and > >>>>> Hbase are deleted, but the intermediate tables in Hive not. The > command > >>>>> shows the tables but do not drop them. I donot have a lookup table > but > >>>>> my > >>>>> fact table is a view. > >>>>> > >>>>> As I run the cleanup command, > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20151001000000 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160529010500 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160608000000 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160608010500 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160609010500 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160615000000 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160626000000 > >>>>> kylin_intermediate_logout_full_cube_19700101000000_20160626042000 > >>>>> kylin_intermediate_test_cube_19700101000000_20151201010500 > >>>>> kylin_intermediate_test_cube_19700101000000_20151231234000 > >>>>> kylin_intermediate_test_cube_19700101000000_20160302063000 > >>>>> kylin_intermediate_test_cube_19700101000000_20160626000000 > >>>>> kylin_intermediate_test_cube_19700101000000_20160626042000 > >>>>> kylin_intermediate_test_cube_19700101000000_20160704082000 > >>>>> Time taken: 0.189 seconds, Fetched: 14 row(s) > >>>>> 2016-06-17 09:37:12,645 INFO [main StorageCleanupJob:262]: Remove > >>>>> intermediate hive table with job id > >>>>> 493fd20b-3074-403e-9963-fe4fb7ff7c65 > >>>>> with job status ERROR > >>>>> 2016-06-17 09:37:12,648 INFO [main StorageCleanupJob:262]: Remove > >>>>> intermediate hive table with job id > >>>>> 8a377e30-e3ba-4fe2-be12-e7d412afec5e > >>>>> with job status ERROR > >>>>> > >>>>> Best regards, > >>>>> > >>>>> Jie > >>>>> > >>>>> > >>>>> Am 17.06.2016 um 09:16 schrieb ShaoFeng Shi: > >>>>> > >>>>> BTW, are you using a view as lookup table? > >>>>> > >>>>>> 2016-06-17 15:15 GMT+08:00 ShaoFeng Shi <[email protected]>: > >>>>>> > >>>>>> This is common; If you have a job failed in between, and you discard > >>>>>> that > >>>>>> > >>>>>> job, the "Garbage collection" step will not be executed, so the > >>>>>>> garbages > >>>>>>> will be left there. > >>>>>>> > >>>>>>> This is why we still recommend user to run offline cleanup every > some > >>>>>>> period; It is not perfert, but be good for most scenarios: > >>>>>>> https://kylin.apache.org/docs15/howto/howto_cleanup_storage.html > >>>>>>> > >>>>>>> 2016-06-17 15:00 GMT+08:00 Li Yang <[email protected]>: > >>>>>>> > >>>>>>> Woo... something new to me. Anybody knows? > >>>>>>> > >>>>>>> On Tue, Jun 14, 2016 at 6:57 PM, Jie Tao <[email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Kylin actually drops useless intermediate tables after cube > >>>>>>>> building, > >>>>>>>> but > >>>>>>>> > >>>>>>>> I still see one "kylin_intermediate_cubename_searchdata" table for > >>>>>>>> > >>>>>>>>> each > >>>>>>>>> cube building in Hive. Are these tables still usefull for Kylin? > I > >>>>>>>>> use > >>>>>>>>> Kylin 1.5.2.1. > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> > >>>>>>>>> Jie > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> > >>>>>>>> Best regards, > >>>>>>> > >>>>>>> Shaofeng Shi > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >> > > > > > -- > Best regards, > > Shaofeng Shi > -- Regards, *Bin Mahone | 马洪宾*
