many thanks, the issue may not that be difficult but very very urgent since it affects the rollback feature
2015-03-17 9:50 GMT+08:00 Shi, Shaofeng <[email protected]>: > Okay, I was able to reproduce this problem; will look into it now; > > On 3/17/15, 9:45 AM, "dong wang" <[email protected]> wrote: > > >Shaofeng, I think mine is a little different from what you mentioned: > >1, the partition date column can be recognized correctly > >2, the segment is built successfully already > >3, I just refresh the LATEST successful segment due to some base data > >changes > > > >2015-03-16 22:03 GMT+08:00 Shi, Shaofeng <[email protected]>: > > > >> Hi Dong, which version of the Kylin build that you deployed? > >> > >> In last week we found a bug which will cause Kylin fail to allocate a > >>new > >> a segment; Kylin will report error saying that the startDate is not > >> matched with expectation; Here is the JIRA: > >> > >> https://issues.apache.org/jira/browse/KYLIN-628 > >> > >> If you¹re using Kylin¹s binary package, please update to the latest one > >>on > >> https://kylin.incubator.apache.org/download/ > >> If you¹re running from source code, please pull the latest code on > >>staging > >> branch; > >> > >> On 3/16/15, 7:40 PM, "dong wang" <[email protected]> wrote: > >> > >> >since the following codes pass invalid startdate and enddate, which > >>leads > >> >to the issue > >> > > >> > public CubeSegment appendSegments(CubeInstance cube, long endDate) > >> >throws > >> >IOException { > >> > checkNoBuildingSegment(cube); > >> > > >> > CubeSegment newSegment; > >> > if > >> >(cube.getDescriptor().getModel().getPartitionDesc().isPartitioned()) { > >> > long startDate = calculateStartDateForAppendSegment(cube); > >> > newSegment = newSegment(cube, startDate, endDate); > >> > } else { > >> > newSegment = newSegment(cube, 0, Long.MAX_VALUE); > >> > } > >> > > >> > validateNewSegments(cube, newSegment); > >> > cube.getSegments().add(newSegment); > >> > Collections.sort(cube.getSegments()); > >> > updateCube(cube); > >> > > >> > return newSegment; > >> > } > >> > > >> >2015-03-16 19:38 GMT+08:00 dong wang <[email protected]>: > >> > > >> >> The related codes, since I'm not that clear about the logic it should > >> >>be: > >> >> > >> >> private CubeSegment newSegment(CubeInstance cubeInstance, long > >> >> startDate, long endDate) { > >> >> if (startDate >= endDate) > >> >> throw new IllegalArgumentException("New segment range > >> >>invalid, > >> >> start date must be earlier than end date, " + startDate + " < " + > >> >>endDate); > >> >> > >> >> CubeSegment segment = new CubeSegment(); > >> >> String incrementalSegName = > >> >>CubeSegment.getSegmentName(startDate, > >> >> endDate); > >> >> segment.setUuid(UUID.randomUUID().toString()); > >> >> segment.setName(incrementalSegName); > >> >> Date creatTime = new Date(); > >> >> segment.setCreateTimeUTC(creatTime.getTime()); > >> >> segment.setDateRangeStart(startDate); > >> >> segment.setDateRangeEnd(endDate); > >> >> segment.setStatus(SegmentStatusEnum.NEW); > >> >> > >>segment.setStorageLocationIdentifier(generateStorageLocation()); > >> >> > >> >> segment.setCubeInstance(cubeInstance); > >> >> > >> >> segment.validate(); > >> >> return segment; > >> >> } > >> >> > >> >> 2015-03-16 19:29 GMT+08:00 dong wang <[email protected]>: > >> >> > >> >>> more notes. this segment to be refreshed is built sccessful first, > >> >>>then, > >> >>> I plan to refresh it due to some incorrect data. > >> >>> > >> >>> 2015-03-16 19:25 GMT+08:00 dong wang <[email protected]>: > >> >>> > >> >>>> watched with firebug, the parameters pass to the backend seem to be > >> >>>> CORRECT as below: > >> >>>> > >> >>>> {buildType: "BUILD", startTime: 1426377600000, endTime: > >>1426464000000} > >> >>>> > >> >>>> 2015-03-16 19:15 GMT+08:00 dong wang <[email protected]>: > >> >>>> > >> >>>>> segment detail page shows: > >> >>>>> > >> >>>>> Start Date (Include) 2015-03-15 00:00:00 > >> >>>>> End Date (Exclude) 2015-03-16 00:00:00 > >> >>>>> > >> >>>>> 2015-03-16 19:11 GMT+08:00 dong wang <[email protected]>: > >> >>>>> > >> >>>>>> The latest segment is the newest one which contains today's data, > >> >>>>>>due > >> >>>>>> to some incorrect data, I want to refresh the segment, however, > >>when > >> >>>>>> cube->actions->refresh->select the latest segment, then the > >> >>>>>>following error > >> >>>>>> occurs: > >> >>>>>> > >> >>>>>> New segment range invalid, start date must be earlier than end > >>date, > >> >>>>>> 1426464000000 < 1426464000000 > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>> > >> >>> > >> >> > >> > >> > >
