It should be a bug; I took a quick look but didn't find the root cause (very busy in the year-end). Could you please investigate it and then contribute a patch? Thanks!
2017-12-22 16:41 GMT+08:00 ketan dikshit <[email protected]>: > Hi Guys, > Any update on this, as I see this is a bug or ? > > We are passing in null as TSRange and the check > Preconditions.checkArgument(tsRange != null); is always failing. > > Help would be appreciated. > > Thanks, > Ketan > > > On 21-Dec-2017, at 3:06 PM, ketan dikshit <[email protected]> > wrote: > > > > Hi Guys, > > I am using Kylin 2.2 and getting some issues with Kylin auto merge. > > The error log in kylin.log is here: > > > > 2017-12-21 08:31:18,811 INFO [Thread-50937] service.CubeService:552 : > checking keepCubeRetention > > 2017-12-21 08:31:18,811 DEBUG [Thread-50937] model.Segments:201 : Cube > publishers_v4 has 1 building segments > > 2017-12-21 08:31:18,811 ERROR [Thread-50937] service.CacheService:87 : > Error in updateOnNewSegmentReady() > > java.lang.IllegalArgumentException > > at com.google.common.base.Preconditions.checkArgument( > Preconditions.java:76) > > at org.apache.kylin.cube.CubeManager.mergeSegments( > CubeManager.java:544) > > at org.apache.kylin.rest.service.CubeService.mergeCubeSegment( > CubeService.java:601) > > at org.apache.kylin.rest.service.CubeService. > updateOnNewSegmentReady(CubeService.java:545) > > at org.apache.kylin.rest.service.CubeService$$ > FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>) > > at org.springframework.cglib.proxy.MethodProxy.invoke( > MethodProxy.java:204) > > at org.springframework.aop.framework.CglibAopProxy$ > DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669) > > at org.apache.kylin.rest.service.CubeService$$ > EnhancerBySpringCGLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > > at org.apache.kylin.rest.service.CacheService$1$1.run( > CacheService.java:85) > > > > > > Auto Merge Thresholds > > 8 (Hours) > > 1 (Days) > > 7 (Days) > > 15 (Days) > > > > > > As I see inside the code, I see, > > private void mergeCubeSegment(String cubeName) { > > CubeInstance cube = getCubeManager().getCube(cubeName); > > if (!cube.needAutoMerge()) { > > return; > > } > > > > synchronized (CubeService.class) { > > try { > > cube = getCubeManager().getCube(cubeName); > > SegmentRange offsets = cube.autoMergeCubeSegments(); > > if (offsets != null) { > > CubeSegment newSeg = getCubeManager().mergeSegments(cube, > null, offsets, true); > > logger.debug("Will submit merge job on " + newSeg); > > DefaultChainedExecutable job = > > EngineFactory.createBatchMergeJob(newSeg, > "SYSTEM"); > > getExecutableManager().addJob(job); > > } else { > > logger.debug("Not ready for merge on cube " + cubeName); > > } > > } catch (IOException e) { > > logger.error("Failed to auto merge cube " + cubeName, e); > > } > > } > > } > > > > ie; getCubeManager().mergeSegments(cube, null, offsets, true); is > passing the TSRange as ‘null’; > > to CubeManager:public CubeSegment mergeSegments(CubeInstance cube, > TSRange tsRange, SegmentRange segRange, boolean force) method; > > which is defined as: > > if (cube.getSegments().isEmpty()) > > throw new IllegalArgumentException("Cube " + cube + " has no > segments"); > > > > checkInputRanges(tsRange, segRange); > > checkBuildingSegment(cube); > > checkCubeIsPartitioned(cube); > > > > if (cube.getSegments().getFirstSegment().isOffsetCube()) { > > // offset cube, merge by date range? > > if (segRange == null && tsRange != null) { > > Pair<CubeSegment, CubeSegment> pair = cube.getSegments( > SegmentStatusEnum.READY) > > .findMergeOffsetsByDateRange(tsRange, Long.MAX_VALUE); > > if (pair == null) > > throw new IllegalArgumentException("Find no segments to merge > by " + tsRange + " for cube " + cube); > > segRange = new SegmentRange(pair.getFirst().getSegRange().start, > pair.getSecond().getSegRange().end); > > } > > tsRange = null; > > Preconditions.checkArgument(segRange != null); > > } else { > > segRange = null; > > Preconditions.checkArgument(tsRange != null); > > } > > > > In my case, this is getting invoked, > > else { > > segRange = null; > > Preconditions.checkArgument(tsRange != null); > > } > > > > and as we have passed in tsRange as NULL , so we get this exception: > > java.lang.IllegalArgumentException > > at com.google.common.base.Preconditions.checkArgument( > Preconditions.java:76) > > at org.apache.kylin.cube.CubeManager.mergeSegments( > CubeManager.java:544) > > at org.apache.kylin.rest.service.CubeService.mergeCubeSegment( > CubeService.java:601) > > at org.apache.kylin.rest.service.CubeService. > updateOnNewSegmentReady(CubeService.java:545) > > at org.apache.kylin.rest.service.CubeService$$ > FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>) > > at org.springframework.cglib.proxy.MethodProxy.invoke( > MethodProxy.java:204) > > at org.springframework.aop.framework.CglibAopProxy$ > DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669) > > at org.apache.kylin.rest.service.CubeService$$ > EnhancerBySpringCGLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > > at org.apache.kylin.rest.service.CacheService$1$1.run( > CacheService.java:85) > > > > And the auto merge fails. > > > > Let me know in case I am missing on something here, or a fix maybe. > > > > > > Thanks, > > Ketan > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋
