It seems KYLIN-3165 <https://issues.apache.org/jira/browse/KYLIN-3165>. I think you can merge it to KYLIN-2.2.0 to solve the problem.
2017-12-26 14:23 GMT+08:00 ketan dikshit <[email protected]>: > Hi ShaoFeng, > We are using Hive as data source for pulling in fact data for cube > building, > Also the commit id as I see is: 0169dc0950fc278239413cfb71313ff9ae8edf7d > > Will also investigate myself, and try to resolve the same and share a > patch. > > Thanks, > Ketan > > > On 22-Dec-2017, at 7:41 PM, ShaoFeng Shi <[email protected]> wrote: > > > > Hi Ketan, > > > > Is your Cube data from Hive or Kafka? > > > > The line number seems couldn't match with the Kylin 2.2 source code; Can > > you double confirm the Kylin version? In Kylin installation folder, there > > is a commit id sha file, please provide that file content to us so we can > > exactly match the line numbers. Thanks. > > > > 2017-12-22 18:12 GMT+08:00 ShaoFeng Shi <[email protected]>: > > > >> It should be a bug; I took a quick look but didn't find the root cause > >> (very busy in the year-end). Could you please investigate it and then > >> contribute a patch? Thanks! > >> > >> 2017-12-22 16:41 GMT+08:00 ketan dikshit <[email protected]> > : > >> > >>> Hi Guys, > >>> Any update on this, as I see this is a bug or ? > >>> > >>> We are passing in null as TSRange and the check > >>> Preconditions.checkArgument(tsRange != null); is always failing. > >>> > >>> Help would be appreciated. > >>> > >>> Thanks, > >>> Ketan > >>> > >>>> On 21-Dec-2017, at 3:06 PM, ketan dikshit > <[email protected]> > >>> wrote: > >>>> > >>>> Hi Guys, > >>>> I am using Kylin 2.2 and getting some issues with Kylin auto merge. > >>>> The error log in kylin.log is here: > >>>> > >>>> 2017-12-21 08:31:18,811 INFO [Thread-50937] service.CubeService:552 : > >>> checking keepCubeRetention > >>>> 2017-12-21 08:31:18,811 DEBUG [Thread-50937] model.Segments:201 : Cube > >>> publishers_v4 has 1 building segments > >>>> 2017-12-21 08:31:18,811 ERROR [Thread-50937] service.CacheService:87 : > >>> Error in updateOnNewSegmentReady() > >>>> java.lang.IllegalArgumentException > >>>> at com.google.common.base.Preconditions.checkArgument(Precondit > >>> ions.java:76) > >>>> at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager. > >>> java:544) > >>>> at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(C > >>> ubeService.java:601) > >>>> at org.apache.kylin.rest.service.CubeService.updateOnNewSegment > >>> Ready(CubeService.java:545) > >>>> at org.apache.kylin.rest.service.CubeService$$FastClassBySpring > >>> CGLIB$$17a07c0e.invoke(<generated>) > >>>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodPro > >>> xy.java:204) > >>>> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvis > >>> edInterceptor.intercept(CglibAopProxy.java:669) > >>>> at org.apache.kylin.rest.service.CubeService$$EnhancerBySpringC > >>> GLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > >>>> at org.apache.kylin.rest.service.CacheService$1$1.run(CacheServ > >>> ice.java:85) > >>>> > >>>> > >>>> Auto Merge Thresholds > >>>> 8 (Hours) > >>>> 1 (Days) > >>>> 7 (Days) > >>>> 15 (Days) > >>>> > >>>> > >>>> As I see inside the code, I see, > >>>> private void mergeCubeSegment(String cubeName) { > >>>> CubeInstance cube = getCubeManager().getCube(cubeName); > >>>> if (!cube.needAutoMerge()) { > >>>> return; > >>>> } > >>>> > >>>> synchronized (CubeService.class) { > >>>> try { > >>>> cube = getCubeManager().getCube(cubeName); > >>>> SegmentRange offsets = cube.autoMergeCubeSegments(); > >>>> if (offsets != null) { > >>>> CubeSegment newSeg = getCubeManager(). > mergeSegments(cube, > >>> null, offsets, true); > >>>> logger.debug("Will submit merge job on " + newSeg); > >>>> DefaultChainedExecutable job = > >>> EngineFactory.createBatchMergeJob(newSeg, "SYSTEM"); > >>>> getExecutableManager().addJob(job); > >>>> } else { > >>>> logger.debug("Not ready for merge on cube " + cubeName); > >>>> } > >>>> } catch (IOException e) { > >>>> logger.error("Failed to auto merge cube " + cubeName, e); > >>>> } > >>>> } > >>>> } > >>>> > >>>> ie; getCubeManager().mergeSegments(cube, null, offsets, true); is > >>> passing the TSRange as ‘null’; > >>>> to CubeManager:public CubeSegment mergeSegments(CubeInstance cube, > >>> TSRange tsRange, SegmentRange segRange, boolean force) method; > >>>> which is defined as: > >>>> if (cube.getSegments().isEmpty()) > >>>> throw new IllegalArgumentException("Cube " + cube + " has no > >>> segments"); > >>>> > >>>> checkInputRanges(tsRange, segRange); > >>>> checkBuildingSegment(cube); > >>>> checkCubeIsPartitioned(cube); > >>>> > >>>> if (cube.getSegments().getFirstSegment().isOffsetCube()) { > >>>> // offset cube, merge by date range? > >>>> if (segRange == null && tsRange != null) { > >>>> Pair<CubeSegment, CubeSegment> pair = > >>> cube.getSegments(SegmentStatusEnum.READY) > >>>> .findMergeOffsetsByDateRange(tsRange, Long.MAX_VALUE); > >>>> if (pair == null) > >>>> throw new IllegalArgumentException("Find no segments to > >>> merge by " + tsRange + " for cube " + cube); > >>>> segRange = new SegmentRange(pair.getFirst(). > getSegRange().start, > >>> pair.getSecond().getSegRange().end); > >>>> } > >>>> tsRange = null; > >>>> Preconditions.checkArgument(segRange != null); > >>>> } else { > >>>> segRange = null; > >>>> Preconditions.checkArgument(tsRange != null); > >>>> } > >>>> > >>>> In my case, this is getting invoked, > >>>> else { > >>>> segRange = null; > >>>> Preconditions.checkArgument(tsRange != null); > >>>> } > >>>> > >>>> and as we have passed in tsRange as NULL , so we get this exception: > >>>> java.lang.IllegalArgumentException > >>>> at com.google.common.base.Preconditions.checkArgument(Precondit > >>> ions.java:76) > >>>> at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager. > >>> java:544) > >>>> at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(C > >>> ubeService.java:601) > >>>> at org.apache.kylin.rest.service.CubeService.updateOnNewSegment > >>> Ready(CubeService.java:545) > >>>> at org.apache.kylin.rest.service.CubeService$$FastClassBySpring > >>> CGLIB$$17a07c0e.invoke(<generated>) > >>>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodPro > >>> xy.java:204) > >>>> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvis > >>> edInterceptor.intercept(CglibAopProxy.java:669) > >>>> at org.apache.kylin.rest.service.CubeService$$EnhancerBySpringC > >>> GLIB$$e4f6b188.updateOnNewSegmentReady(<generated>) > >>>> at org.apache.kylin.rest.service.CacheService$1$1.run(CacheServ > >>> ice.java:85) > >>>> > >>>> And the auto merge fails. > >>>> > >>>> Let me know in case I am missing on something here, or a fix maybe. > >>>> > >>>> > >>>> Thanks, > >>>> Ketan > >>>> > >>>> > >>>> > >>>> > >>> > >>> > >> > >> > >> -- > >> Best regards, > >> > >> Shaofeng Shi 史少锋 > >> > >> > > > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > >
