Yup, the delete api works. Thanks! 发件人: 星辰#&勿语 [mailto:[email protected]] 发送时间: 2017年1月25日 11:04 收件人: 市场中心-ZHANGDA32698 主题: Re:答复: 答复: more than 1 append dict for globalDict
you can delete segment by rest api in kylin web UI ------------------ 原始邮件 ------------------ 发件人: "市场中心-ZHANGDA32698" <[email protected]>; 发送时间: 2017年1月25日(星期三) 10:54 收件人: "[email protected]" <[email protected]>;"dev" <[email protected]>;"Yerui Sun" <[email protected]>; 主题: 答复: 答复: more than 1 append dict for globalDict Hi ShaoFeng, I see, thanks for the solution. Actually I notice that the extra dictionary appeared right after the last segment were successfully built, which means all segments in the cube.json are pointing to the older dictionary except the last one. So similar to your suggestion, I wonder if I can remove the last segment together with the extra the dictionary, then do a rebuild on the last segment. Is it ok to do so? Another thing, how do I remove an existing segment? Found this https://issues.apache.org/jira/browse/KYLIN-1540 but seem doesn’t work in v1.6.0 yet. Can I just edit the cube.json file and remove the want-to-remove segment part? Thanks ! 发件人: ShaoFeng Shi [mailto:[email protected]]<mailto:[mailto:[email protected]]> 发送时间: 2017年1月24日 20:27 收件人: user; dev; Yerui Sun 主题: Re: 答复: more than 1 append dict for globalDict Yerui is on vacation; If you're urgent, you can try this way (also confirmed with yerui); 1) backup your metadata to local folder; 2) copy to backup the two global dictionaries in some other folder; 3) among the two dictionaries, pick the bigger one (it should includes all values in the smaller one), and then in the cube json, replace all the references to smaller dict with the bigger one; You can edit the Cube json from Kylin's web gui, in the "Admin" -> "Edit JSON"; 4) remove the smaller one dict from kylin metadata, use kylin's command, like : "./bin/metastore.sh remove /dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict" 2017-01-24 20:14 GMT+08:00 ShaoFeng Shi <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>>: @Yerui, do you have suggestion on this? 2017-01-24 18:19 GMT+08:00 市场中心-ZHANGDA32698 <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>>: Hi ShaoFeng, Thanks for your suggestion. However since ours is a daily build, I guess there’s no concurrent issue. The problem stops us from building subsequent segments. I try to remove one of the Dicts from hbase and try a rebuild, but seems that they are referenced at somewhere else and I got ‘java.lang.IllegalStateException: No resource found at -- /dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict’ problem at step 3 ‘Extract Fact Table Distinct Columns’. Of course I can add the key back and let it continue, but without surprise it will stop at setp4 ‘Build Dimension Dictionary’ with same error I encountered before ‘GlobalDict xxxx should have 0 or 1 append dict but 2’ How can I resolve this and continue my cube building? Any suggestion is appreciated, thanks! 发件人: ShaoFeng Shi [mailto:[email protected]<mailto:[email protected]>]<mailto:[mailto:[email protected]%3cmailto:[email protected]%3e]> 发送时间: 2017年1月24日 17:01 收件人: user 主题: Re: more than 1 append dict for globalDict Hi zhangda, Do you have multiple segments (which has the count distinct measure) building concurrently? In 1.6.0 and before there is a concurrent bug I think; Yerui Sun fixed it in https://issues.apache.org/jira/browse/KYLIN-2192 So, please check whether it is this problem first; If yes, you need add some control (lock) to avoid concurrent build for this cube. 2017-01-24 16:33 GMT+08:00 市场中心-ZHANGDA32698 <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>>: Hi there, I have a cube computing for some UV statistics. Since it requires global count distinct operation, in the advanced dictionaries setting I put UV key column’s builder class as ‘GlobalDictionaryBuilder’. Everyday build went on without any problem until yesterday there was a ‘GlobalDict xxxx should have 0 or 1 append dict but 2"’ exception. I checked the ‘kylin_metadata’ table in the hbase , and saw there were indeed 2 dict '/dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict' /dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/LOGINKEY/cedce514-657d-4032-a591-7f01d984df65.dict' Fyi, APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1 is the hive table name, LOGINKEY is the count distinct key I’m not sure what had gone wrong, and I suppose global dict on 1 key should be unique. Has anyone ever encountered similar error before, can share some ideas? Thanks! -- Best regards, Shaofeng Shi 史少锋 -- Best regards, Shaofeng Shi 史少锋 -- Best regards, Shaofeng Shi 史少锋
