Many thanks for the advice. Do you plan to release version 2.4.1 in September?
From: ShaoFeng Shi [mailto:[email protected]] Sent: viernes, 24 de agosto de 2018 5:01 To: user <[email protected]> Subject: Re: Segment overlap issues when refreshing a cube segment Kylin 2.4.1 will be kicked off soon, which includes several bugfixes in 2.4.0. You can directly upgrade to 2.4.1. 2018-08-23 19:07 GMT+08:00 <[email protected] <mailto:[email protected]> >: Shaofeng, thank you very much for the help. I will share if version 2.4 solves our segment overlap issues. Roberto Tardío Olmos Head of Big Data Analytics Avenida de Brasil, 17 <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g> , Planta 16.28020 Madrid Fijo: 91.788.34.10 <http://bigdata.stratebi.com/> http://bigdata.stratebi.com/ <http://www.stratebi.com/> http://www.stratebi.com From: ShaoFeng Shi [mailto:[email protected] <mailto:[email protected]> ] Sent: miércoles, 22 de agosto de 2018 15:21 To: user <[email protected] <mailto:[email protected]> > Subject: Re: Segment overlap issues when refreshing a cube segment Yes, KYLIN-3311 introduces a lock mechanism when updating metadata, that can avoid concurrent updates on the same entity, so to avoid two duplicated segments be inserted. 2018-08-22 19:52 GMT+08:00 <[email protected] <mailto:[email protected]> >: Many thanks ShaoFeng! Yes, I could see the new button because we also have a test installation of Kylin 2.4 and we are thinking about to update kylin of the production environment from 2.2 to 2.4. As for the API, I was aware of its existence. However, in the case I'm telling you we can't delete it because the segment is in the middle of other segments and in NEW state, duplicated. That's why I was talking about the JIRA advocacy https://issues.apache.org/jira/browse/KYLIN-3311 . Do you think that the error causing the segment overlap is related to this JIRA? Kind Regards, Roberto From: ShaoFeng Shi [mailto:[email protected] <mailto:[email protected]> ] Sent: martes, 21 de agosto de 2018 14:54 To: user <[email protected] <mailto:[email protected]> > Subject: Re: Segment overlap issues when refreshing a cube segment Hi Roberto, in the latest Kylin (I'm running), on the web GUI, there is a "Delete segment" button, which allows you to delete a segment easily. You can try to use the Rest API to delete a segment, this API should have existed for a long time: https://kylin.apache.org/docs/howto/howto_use_restapi.html#delete-segment I 2018-08-21 6:15 GMT+08:00 <[email protected] <mailto:[email protected]> >: Hi, I am using Kylin 2.2, where I have several OLAP cubes in which I am constantly adding and refreshing segments. I am faced with a problem that has generated two segments for the same period of time and therefore an error of the "segments overlap" type. Kylin is also cluster mounted, with one machine for cube building and one for querying. Here is the log of the query machine that seems to reflect the error (cube segment building process was successfully completed): 2018-08-16 09:19:35,833 INFO [http-bio-7070-exec-93] cube.CubeManager:799 : Reloaded cube trafico_cubo_v8 being CUBE[name=trafico_cubo_v8] having 20 segments 2018-08-16 09:19:35,833 DEBUG [http-bio-7070-exec-93] cachesync.Broadcaster:256 : BroadcastingUPDATE, project_data, My_Cube_Trafico 2018-08-16 09:19:35,834 INFO [http-bio-7070-exec-93] service.CacheService:143 : removeOLAPDataSource is called for project My_Cube_Trafico 2018-08-16 09:19:35,834 INFO [http-bio-7070-exec-93] service.CacheService:125 : cleaning cache for project My_Cube_Trafico (currently remove all entries) 2018-08-16 09:19:35,834 DEBUG [http-bio-7070-exec-93] cachesync.Broadcaster:290 : Done broadcastingUPDATE, project_data, My_Cube_Trafico 2018-08-16 09:19:35,834 DEBUG [http-bio-7070-exec-93] cachesync.Broadcaster:290 : Done broadcastingUPDATE, cube, trafico_cubo_v8 2018-08-16 09:19:36,817 INFO [http-bio-7070-exec-104] cube.CubeManager:358 : Updating cube instance 'plataforma_cubo_mes_v2' 2018-08-16 09:19:36,817 ERROR [http-bio-7070-exec-104] controller.CubeController:337 : Segments overlap: plataforma_cubo_mes_v2[20180801000000_20180901000000] and plataforma_cubo_mes_v2[20180801000000_20180901000000] 2018-08-16 09:19:36,818 ERROR [http-bio-7070-exec-104] controller.BasicController:57 : 2018-08-16 09:19:37,264 DEBUG [http-bio-7070-exec-93] cachesync.Broadcaster:256 : BroadcastingUPDATE, cube, trafico_cubo_v8 2018-08-16 09:19:37,265 INFO [http-bio-7070-exec-93] cube.CubeManager:799 : Reloaded cube trafico_cubo_v8 being CUBE[name=trafico_cubo_v8] having 20 segments Could this error be related to the JIRA entry https://issues.apache.org/jira/browse/KYLIN-3311? In that case, I see that it has been fixed in version 2.4 of Kylin. Also, to fix the error, we had to stop Kylin, remove the duplicate segment from the metadata, restart Kylin and then refresh the segment. As far as I can read on the website https://issues.apache.org/jira/browse/KYLIN-2849 , from version 2.3 onwards you can delete a segment in any position from the UI or the Kylin API after disabling the cube. Is that correct? This would help us mitigate the problem more quickly. Thanks in advance! Roberto Tardío Olmos Head of Big Data Analytics Avenida de Brasil, 17 <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g> , Planta 16.28020 Madrid Fijo: 91.788.34.10 http://bigdata.stratebi.com/ http://www.stratebi.com <http://www.stratebi.com/> -- Best regards, Shaofeng Shi 史少锋 -- Best regards, Shaofeng Shi 史少锋 -- Best regards, Shaofeng Shi 史少锋
