Hi Guys, Any update on this, as I see this is a bug or ? We are passing in null as TSRange and the check Preconditions.checkArgument(tsRange != null); is always failing.
Help would be appreciated. Thanks, Ketan > On 21-Dec-2017, at 3:06 PM, ketan dikshit <[email protected]> > wrote: > > Hi Guys, > I am using Kylin 2.2 and getting some issues with Kylin auto merge. > The error log in kylin.log is here: > > 2017-12-21 08:31:18,811 INFO [Thread-50937] service.CubeService:552 : > checking keepCubeRetention > 2017-12-21 08:31:18,811 DEBUG [Thread-50937] model.Segments:201 : Cube > publishers_v4 has 1 building segments > 2017-12-21 08:31:18,811 ERROR [Thread-50937] service.CacheService:87 : Error > in updateOnNewSegmentReady() > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:76) > at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:544) > at > org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:601) > at > org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:545) > at > org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>) > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669) > at > org.apache.kylin.rest.service.CubeService$$EnhancerBySpringCGLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > at > org.apache.kylin.rest.service.CacheService$1$1.run(CacheService.java:85) > > > Auto Merge Thresholds > 8 (Hours) > 1 (Days) > 7 (Days) > 15 (Days) > > > As I see inside the code, I see, > private void mergeCubeSegment(String cubeName) { > CubeInstance cube = getCubeManager().getCube(cubeName); > if (!cube.needAutoMerge()) { > return; > } > > synchronized (CubeService.class) { > try { > cube = getCubeManager().getCube(cubeName); > SegmentRange offsets = cube.autoMergeCubeSegments(); > if (offsets != null) { > CubeSegment newSeg = getCubeManager().mergeSegments(cube, > null, offsets, true); > logger.debug("Will submit merge job on " + newSeg); > DefaultChainedExecutable job = > EngineFactory.createBatchMergeJob(newSeg, "SYSTEM"); > getExecutableManager().addJob(job); > } else { > logger.debug("Not ready for merge on cube " + cubeName); > } > } catch (IOException e) { > logger.error("Failed to auto merge cube " + cubeName, e); > } > } > } > > ie; getCubeManager().mergeSegments(cube, null, offsets, true); is passing the > TSRange as ‘null’; > to CubeManager:public CubeSegment mergeSegments(CubeInstance cube, TSRange > tsRange, SegmentRange segRange, boolean force) method; > which is defined as: > if (cube.getSegments().isEmpty()) > throw new IllegalArgumentException("Cube " + cube + " has no segments"); > > checkInputRanges(tsRange, segRange); > checkBuildingSegment(cube); > checkCubeIsPartitioned(cube); > > if (cube.getSegments().getFirstSegment().isOffsetCube()) { > // offset cube, merge by date range? > if (segRange == null && tsRange != null) { > Pair<CubeSegment, CubeSegment> pair = > cube.getSegments(SegmentStatusEnum.READY) > .findMergeOffsetsByDateRange(tsRange, Long.MAX_VALUE); > if (pair == null) > throw new IllegalArgumentException("Find no segments to merge by " > + tsRange + " for cube " + cube); > segRange = new SegmentRange(pair.getFirst().getSegRange().start, > pair.getSecond().getSegRange().end); > } > tsRange = null; > Preconditions.checkArgument(segRange != null); > } else { > segRange = null; > Preconditions.checkArgument(tsRange != null); > } > > In my case, this is getting invoked, > else { > segRange = null; > Preconditions.checkArgument(tsRange != null); > } > > and as we have passed in tsRange as NULL , so we get this exception: > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:76) > at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:544) > at > org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:601) > at > org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:545) > at > org.apache.kylin.rest.service.CubeService$$FastClassBySpringCGLIB$$17a07c0e.invoke(<generated>) > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669) > at > org.apache.kylin.rest.service.CubeService$$EnhancerBySpringCGLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > at > org.apache.kylin.rest.service.CacheService$1$1.run(CacheService.java:85) > > And the auto merge fails. > > Let me know in case I am missing on something here, or a fix maybe. > > > Thanks, > Ketan > > > >
