The bug was fixed and all tests passed in my Jenkins; Please pull the latest code from 0.7-staging and try again; Thanks;
On 6/18/15, 3:01 PM, "Shi, Shaofeng" <[email protected]> wrote: >The unit test was broken with a commit in yesterday; we’re fixing it; Will >update you when it got fixed; > >On 6/18/15, 4:43 AM, "Srinivasan Hariharan" ><[email protected]> wrote: > >>Hi, >>I made the changes but Kylin-Query module unit tests fails in 0.7 staging >>branch code. Without my changes also unit tests fails for the query >>module. >> >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>CombinationTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAnd >>C >>ompQuery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>IIQueryTest.testDetailedQuery:59->KylinTestBase.execAndCompQuery:359->Kyl >>i >>nTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >>IIQueryTest>KylinQueryTest.testCommonQuery:165->KylinTestBase.execAndComp >>Q >>uery:359->KylinTestBase.executeQuery:204 type mismatch: >>type1: >>DECIMAL(19, 4) >>type2: >>DECIMAL(39, 16) NOT NULL >> >> >>Regards, >>Srinivasan Hariharan >> >> >> >>> Date: Wed, 17 Jun 2015 11:32:05 +0800 >>> Subject: Re: Hive external Table Dimension >>> From: [email protected] >>> To: [email protected] >>> >>> Srinivasan, >>> >>> You can checkout 0.7-staging branch as start; Look into >>> org.apache.kylin.dict.lookup.HiveTable, the implementation of >>> "getSignature()" and "getColumnDelimeter()" is not perfect: it calls >>> "getFileTable()", which will check the underlying HDFS file, as we know >>> this is not suitable for external table; >>> >>> To fix the problem, need re-write two methods; In the new >>>"getSignature()", >>> using Hive API to get the table's path, size and last modified time, >>>you >>> may need do some search here; For the new "getColumnDelimeter()", just >>> return DELIM_AUTO is okay; >>> >>> After finish the code and pass all unit test, please create a patch and >>> attache it in the JIRA for review ("pull request" is not accepted >>>anymore); >>> >>> Thanks for the contribution; >>> >>> >>> >>> 2015-06-17 1:10 GMT+08:00 Srinivasan Hariharan < >>> [email protected]>: >>> >>> > Hi , >>> > >>> > I am interested to contribute to this JIRA, could anyone help me out >>>where >>> > can I start. >>> > >>> > https://issues.apache.org/jira/browse/KYLIN-824 >>> > >>> > Regards, >>> > Srinivasan Hariharan >>> > >>> > >>> > >>> > From: [email protected] >>> > To: [email protected] >>> > Subject: RE: Hive external Table Dimension >>> > Date: Thu, 11 Jun 2015 21:51:08 +0530 >>> > >>> > >>> > >>> > >>> > >>> > Thanks, >>> > >>> > I have created JIRA. >>> > >>> > https://issues.apache.org/jira/browse/KYLIN-824 >>> > >>> > I am interested to contribute, i will see the code and update for >>>help. >>> > >>> > >>> > > From: [email protected] >>> > > To: [email protected] >>> > > Subject: Re: Hive external Table Dimension >>> > > Date: Thu, 11 Jun 2015 14:33:59 +0000 >>> > > >>> > > Kylin need take snapshot for lookup tables for runtime queries (to >>>derive >>> > > the dimensions that not on row key), that¹s why it try to seek the >>> > > underlying data file; >>> > > >>> > > So far without this it couldn¹t move ahead; For long run, Kylin can >>> > > consider to abstract this; Please open a JIRA as requirement if you >>>like; >>> > > >>> > > On 6/11/15, 5:45 PM, "Srinivasan Hariharan02" < >>> > [email protected]> >>> > > wrote: >>> > > >>> > > >Hi, >>> > > > >>> > > >I have a dimension external table in Hive which is created using >>>Hbase >>> > > >Storage handler. After creating the cube using this hive table >>>cube >>> > > >build job failed in the "Build Dimension Dictionary" with below >>>error >>> > > >java.lang.IllegalStateException: Expect 1 and only 1 non-zero file >>>under >>> > > >hdfs://host:8020/user/hive/warehouse/hbase.db/department/, but >>>find 0 >>> > > > at >>> > > >>>>org.apache.kylin.dict.lookup.HiveTable.findOnlyFile(HiveTable.java:123) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.dict.lookup.HiveTable.computeHDFSLocation(HiveTable.ja >>>>v >>>>a: >>> > > >107) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.dict.lookup.HiveTable.getHDFSLocation(HiveTable.java:8 >>>>3 >>>>) >>> > > > at >>> > > >>>>org.apache.kylin.dict.lookup.HiveTable.getFileTable(HiveTable.java:76) >>> > > > at >>> > > >>>>org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:71) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManag >>>>e >>>>r. >>> > > >java:164) >>> > > > at >>> > > >>>>org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:154) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Diction >>>>a >>>>ry >>> > > >GeneratorCLI.java:53) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Diction >>>>a >>>>ry >>> > > >GeneratorCLI.java:42) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictiona >>>>r >>>>yJ >>> > > >ob.java:53) >>> > > > at >>>org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>> > > > at >>>org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExe >>>>c >>>>ut >>> > > >able.java:63) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu >>>>t >>>>ab >>> > > >le.java:107) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultC >>>>h >>>>ai >>> > > >nedExecutable.java:50) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu >>>>t >>>>ab >>> > > >le.java:107) >>> > > > at >>> > > >>> > >>>>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Def >>>>a >>>>ul >>> > > >tScheduler.java:132) >>> > > > at >>> > > >>> > >>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja >>>>v >>>>a: >>> > > >1145) >>> > > > at >>> > > >>> > >>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >>>>a >>>>va >>> > > >:615) >>> > > > at java.lang.Thread.run(Thread.java:744) >>> > > > >>> > > >Since external table created from other sources like Hbase hive >>>doesn't >>> > > >store any data in their warehouse directory. So it should not >>>check for >>> > > >files under warehouse dir for external tables. Please help. >>> > > > >>> > > >Regards, >>> > > >Srinivasan Hariharan >>> > > >Mob +91-9940395830 >>> > > > >>> > > > >>> > > >**************** CAUTION - Disclaimer ***************** >>> > > >This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION >>>intended >>> > > >solely >>> > > >for the use of the addressee(s). If you are not the intended >>>recipient, >>> > > >please >>> > > >notify the sender by e-mail and delete the original message. >>>Further, >>> > you >>> > > >are not >>> > > >to copy, disclose, or distribute this e-mail or its contents to >>>any >>> > other >>> > > >person and >>> > > >any such actions are unlawful. This e-mail may contain viruses. >>>Infosys >>> > > >has taken >>> > > >every reasonable precaution to minimize this risk, but is not >>>liable for >>> > > >any damage >>> > > >you may sustain as a result of any virus in this e-mail. You >>>should >>> > carry >>> > > >out your >>> > > >own virus checks before opening the e-mail or attachment. Infosys >>> > > >reserves the >>> > > >right to monitor and review the content of all messages sent to or >>>from >>> > > >this e-mail >>> > > >address. Messages sent to or from this e-mail address may be >>>stored on >>> > > >the >>> > > >Infosys e-mail system. >>> > > >***INFOSYS******** End of Disclaimer ********INFOSYS*** >>> > > >>> > >>> > >>> > >> >
