[jira] [Created] (KYLIN-3891) Kylin jdbc throws NoHttpResponseException
zhao jintao created KYLIN-3891:
--
Summary: Kylin jdbc throws NoHttpResponseException
Key: KYLIN-3891
URL: https://issues.apache.org/jira/browse/KYLIN-3891
Project: Kylin
Issue Type: Bug
Components: Driver - JDBC
Affects Versions: v2.5.2
Environment: Huawei FusionInsight
Reporter: zhao jintao
Hi Team;
When I use kylin jdbc, I find that jdbc throws NoHttpResponseException.
Then I do jdbc test cases in "DriverTest.java", I use one jdbc connection
query 20 times, each time first wait 20 minutes. Unfortunately， I also find
that jdbc also throws NoHttpResponseException.
This is my test code:
try {
for (int j = 0; j < 20; j++) {
Thread.sleep(1000*20);
Statement state = conn.createStatement();
ResultSet resultSet = state.executeQuery("select count(*) from
KYLIN_SALES ;");
printResultSetMetaData(resultSet);
printResultSet(resultSet);
resultSet.close();
}
} catch (InterruptedException e){
e.printStackTrace();
}
This is NoHttpResponseException:
java.sql.SQLException: Error while executing SQL "select count(*) from
KYLIN_SALES ;": java.sql.SQLException: org.apache.http.NoHttpResponseException:
The target server failed to respond
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
at org.apache.kylin.jdbc.DriverTest.testHttpClient(DriverTest.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.RuntimeException: java.sql.SQLException:
org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(KylinMeta.java:115)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
... 24 more
Caused by: java.sql.SQLException: org.apache.http.NoHttpResponseException: The
target server failed to respond
at org.apache.kylin.jdbc.KylinResultSet.execute(KylinResultSet.java:71)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
at org.apache.kylin.jdbc.KylinMeta.prepareAndExecute(KylinMeta.java:111)
... 26 more
Caused by: org.apache.http.NoHttpResponseException: The target server failed to
respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
Re: kylin top-n query
And this may also help: http://kylin.apache.org/docs/tutorial/create_cube.html (go to the "TOP_N" Section) -- Regards! Aron Tao 黄云尧 于2019年3月18日周一 下午12:06写道： > someone has documents for top-n query in kylin ? > > > >
[RESULT][VOTE] Release apache-kylin-2.6.1 binary packages
Thanks to everyone who has verified the binary packages. The tally is as follows. 3 binding +1s: Shaofeng Shi Billy Liu Yang Li 1 non-binding +1s: Jianhua Peng No 0s or -1s. Therefore I am delighted to announce that the proposal to release Apache-Kylin-2.6.1 binary package has passed. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Email: shaofeng...@apache.org Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org
Re: [VOTE] Release apache-kylin-2.6.1 binary packages
Thanks to Billy, Yang, Jianhua for the verification! I will send out the result soon. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Email: shaofeng...@apache.org Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org Jianhua Peng 于2019年3月18日周一 下午8:55写道： > > +1 > On 2019/03/15 08:57:57, ShaoFeng Shi wrote: > > Hi all, > > > > The source code of apache-kylin-2.6.1 has been released on 3/8 on last > > week. Now we prepared the binary packages of v2.6.1 for users' > convenience. > > Please review the binary packages, and give your vote. > > > > The packages are in : > > https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.6.1-rc1/ > > > > The hash of the artifact is as follows: > > apache-kylin-2.6.1-bin-hbase1x.tar.gz - > > f91f3ff0d6426f84e752cc1178fd704895842e9464ce5cd31c099b1f31eb6b68 > > apache-kylin-2.6.1-bin-hadoop3.tar.gz - > > 6f06e94055d7639729f7879508669375a80eddd76c2a4880da38a0f7f223de44 > > apache-kylin-2.6.1-bin-cdh57.tar.gz - > > b5038da13bfbf7fbba9a46b4675b587c882a8e152d244b063c4a610d6000bd55 > > apache-kylin-2.6.1-bin-cdh60.tar.gz - > > d1ba39a6e288131a89e3c8e4d0959fd3c05c4ed42df1164df4d2ec9ddf55f92f > > > > The checking content should include: > > > >- sigs and hashes must be OK > >- the package must contain the correct NOTICE and LICENSE files for > the > >included content > >- the package must not contain any content not derived from the > source. > >- in the case of bundled binaries, reviewers must check that all > >contents are represented in the LICENSE (and NOTICE file if required). > >The bundle must not contain any files that are prohibited from > >distribution (category X). > > > > > > Here is my vote: > > +1 (binding) > > > > Thank you! > > > > Best regards, > > > > Shaofeng Shi 史少锋 > > Apache Kylin PMC > > Email: shaofeng...@apache.org > > > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > > Join Kylin user mail group: user-subscr...@kylin.apache.org > > Join Kylin dev mail group: dev-subscr...@kylin.apache.org > > >
Build buildSupportsSnappy Error When Doing Integration Testing
Hi all, I am currently running integration test. However, I met the following error. Could you please share some suggestions on this? *1. Command*: mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox *2. Error message from Yarn Container Attempt:* 2019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Accepting Mapper Key with ordinal: 1 2019-03-18 16:43:25,583 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do map, available memory: 322m 2019-03-18 16:43:25,596 INFO [main] org.apache.kylin.common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.cuboid.CuboidManager 2019-03-18 16:43:25,599 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2019-03-18 16:43:25,599 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false 2019-03-18 16:43:25,795 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63 ) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136 ) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150 ) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168 ) at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at org.apache.hadoop.io.SequenceFile$BlockCompressWriter .(SequenceFile.java:1552) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75) at org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:113) at org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:468) at org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:85 ) at org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:44 ) at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) 2019-03-18 16:43:25,797 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do cleanup, available memory: 318m 2019-03-18 16:43:25,813 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Total rows: 1 2019-03-18 16:43:25,813 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63 ) at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136 ) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150 ) at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168 ) at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1304) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:1192) at org.apache.hadoop.io.SequenceFile$BlockCompressWriter .(SequenceFile.java:1552) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:289) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:542) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getSequenceWriter(SequenceFileOutputFormat.java:64) at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:75) at org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat$LazyRecordWriter.write(LazyOutputFormat.java:113) at org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:468) at org.apache.kylin.engine.mr.steps.FilterRecommendCuboidDataMapper.doMap(FilterRecommendCuboidDataMapper.java:85 ) at
??????kylin top-n query
Hi ?? There is a blog abort the principle of TopN query[http://kylin.apache.org/blog/2016/03/19/approximate-topn-measure/]. I hope that would be useful for you. -- Best Regards, Chao Long -- -- ??: "??"; : 2019??3??18??(??) 8:06 ??: "dev"; : kylin top-n query someone has documents for top-n query in kylin ?
怎么修复cube，当我丢失了某个dict时
您好： 今天我在构建cube 时，遇到一个问题，就是在构建第四步 Build Dimension Dictionary ，一直都会抛出一个异常。问题是在构建今天早 3点到4点的segment出现的，我把2-3点的重新构建 也不行。怎么可以修复这个cube，其它cube 和 在3点之前的segment 都可以构建 org.apache.kylin.engine.mr.exception.HadoopShellException: java.lang.RuntimeException: java.io.IOException: Failed to read big resource /dict/AD_V.IMPRESSION_LOG/SCHEDULEID/cdfa5043-e699-fbd7-e8db-b3646d927a5f.dict at org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:108) at org.apache.kylin.dict.DictionaryManager.checkDupByContent(DictionaryManager.java:173) at org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:151) at org.apache.kylin.dict.DictionaryManager.saveDictionary(DictionaryManager.java:320) at org.apache.kylin.cube.CubeManager$DictionaryAssist.saveDictionary(CubeManager.java:1117) at org.apache.kylin.cube.CubeManager.saveDictionary(CubeManager.java:1079) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:74) at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:55) at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73) at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93) at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Failed to read big resource /dict/AD_V.IMPRESSION_LOG/SCHEDULEID/cdfa5043-e699-fbd7-e8db-b3646d927a5f.dict at org.apache.kylin.common.persistence.PushdownResourceStore.openPushdown(PushdownResourceStore.java:176) at org.apache.kylin.storage.hbase.HBaseResourceStore.getInputStream(HBaseResourceStore.java:256) at org.apache.kylin.storage.hbase.HBaseResourceStore.rawResource(HBaseResourceStore.java:226) at org.apache.kylin.storage.hbase.HBaseResourceStore.getResourceImpl(HBaseResourceStore.java:276) at org.apache.kylin.common.persistence.ResourceStore$3.call(ResourceStore.java:291) at org.apache.kylin.common.persistence.ResourceStore$3.call(ResourceStore.java:288) at org.apache.kylin.common.persistence.ExponentialBackoffRetry.doWithRetry(ExponentialBackoffRetry.java:52) at org.apache.kylin.common.persistence.ResourceStore.getResourceWithRetry(ResourceStore.java:288) at org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:265) at org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:256) at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:397) at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:80) at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:77) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257) at com.google.common.cache.LocalCache.get(LocalCache.java:4000) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874) at org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:101) ... 17 more Caused by: java.io.FileNotFoundException: /kylin/kylin_metadata/resources/dict/AD_V.IMPRESSION_LOG/SCHEDULEID/cdfa5043-e699-fbd7-e8db-b3646d927a5f.dict (FS: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_1042273900_27, ugi=root (auth:SIMPLE)]]) at org.apache.kylin.common.persistence.PushdownResourceStore.openPushdown(PushdownResourceStore.java:173) ... 37 more result code:2 at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70) at
Re: [VOTE] Release apache-kylin-2.6.1 binary packages
+1 On 2019/03/15 08:57:57, ShaoFeng Shi wrote: > Hi all, > > The source code of apache-kylin-2.6.1 has been released on 3/8 on last > week. Now we prepared the binary packages of v2.6.1 for users' convenience. > Please review the binary packages, and give your vote. > > The packages are in : > https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.6.1-rc1/ > > The hash of the artifact is as follows: > apache-kylin-2.6.1-bin-hbase1x.tar.gz - > f91f3ff0d6426f84e752cc1178fd704895842e9464ce5cd31c099b1f31eb6b68 > apache-kylin-2.6.1-bin-hadoop3.tar.gz - > 6f06e94055d7639729f7879508669375a80eddd76c2a4880da38a0f7f223de44 > apache-kylin-2.6.1-bin-cdh57.tar.gz - > b5038da13bfbf7fbba9a46b4675b587c882a8e152d244b063c4a610d6000bd55 > apache-kylin-2.6.1-bin-cdh60.tar.gz - > d1ba39a6e288131a89e3c8e4d0959fd3c05c4ed42df1164df4d2ec9ddf55f92f > > The checking content should include: > >- sigs and hashes must be OK >- the package must contain the correct NOTICE and LICENSE files for the >included content >- the package must not contain any content not derived from the source. >- in the case of bundled binaries, reviewers must check that all >contents are represented in the LICENSE (and NOTICE file if required). >The bundle must not contain any files that are prohibited from >distribution (category X). > > > Here is my vote: > +1 (binding) > > Thank you! > > Best regards, > > Shaofeng Shi 史少锋 > Apache Kylin PMC > Email: shaofeng...@apache.org > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > Join Kylin user mail group: user-subscr...@kylin.apache.org > Join Kylin dev mail group: dev-subscr...@kylin.apache.org >
kylin top-n query
someone has documents for top-n query in kylin ?
[jira] [Created] (KYLIN-3890) Add doc about usage of ./bin/metadata.sh
Yuzhang QIU created KYLIN-3890: -- Summary: Add doc about usage of ./bin/metadata.sh Key: KYLIN-3890 URL: https://issues.apache.org/jira/browse/KYLIN-3890 Project: Kylin Issue Type: Improvement Components: Documentation Affects Versions: v2.5.2 Reporter: Yuzhang QIU JIRA title descript the JIRA -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3889) The column of Count distinct bitmap disappear when create a cube
Wenquan Ha created KYLIN-3889: - Summary: The column of Count distinct bitmap disappear when create a cube Key: KYLIN-3889 URL: https://issues.apache.org/jira/browse/KYLIN-3889 Project: Kylin Issue Type: Bug Components: Measure - Count Distinct Affects Versions: v2.5.2 Reporter: Wenquan Ha I have found that count distinct bitmap measure will disappear in cube, which measure use the column of join key( which column connect the fact table and lookup table with inner join or left join). *Reproduction step(version: apache-kylin-2.6.1-bin-hbase1)* # create a model with one fact table and one lookup table through the A column to connect both # create a cube, A column will not be choosed as dimension column. And create a count distinct bitmap measure use A column. # Click Next button and you will find the measure which you just created already disappeard on web gui. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: 答复: How kylin store data in Hbase ?
Hi Rahul, Please check this slide, I made for last year's HBaseCon; Page 16/17 introduces how Kylin store cube in HBase; https://www.slideshare.net/ShiShaoFeng1/apache-kylin-on-hbase-extreme-olap-engine-for-big-data Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Email: shaofeng...@apache.org Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org rsanad...@gmail.com 于2019年3月18日周一 下午4:13写道： > Hello Na Zhai, > > 2 Mandatory Dimension and No Hierarchic Dims . > > Thanks, > Rahul S > > -- > Sent from: http://apache-kylin.74782.x6.nabble.com/ >
Re: 答复: How kylin store data in Hbase ?
Hello Na Zhai, 2 Mandatory Dimension and No Hierarchic Dims . Thanks, Rahul S -- Sent from: http://apache-kylin.74782.x6.nabble.com/
[jira] [Created] (KYLIN-3888) TableNotDisabledException when running "Convert Lookup Table to HFile"
Guangxu Cheng created KYLIN-3888:
Summary: TableNotDisabledException when running "Convert Lookup
Table to HFile"
Key: KYLIN-3888
URL: https://issues.apache.org/jira/browse/KYLIN-3888
Project: Kylin
Issue Type: Bug
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
{code}
org.apache.hadoop.hbase.TableNotDisabledException:
KYLIN_LOOKUP_DEFAULT.F_ADVERTISER_DIM_KYLIN_TEST_9WCVUDIXI5
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at
org.apache.hadoop.hbase.util.ForeignExceptionUtil.toIOException(ForeignExceptionUtil.java:45)
at
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.convertResult(HBaseAdmin.java:4756)
at
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:4714)
at
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:4647)
at
org.apache.hadoop.hbase.client.HBaseAdmin.deleteTable(HBaseAdmin.java:904)
at
org.apache.kylin.storage.hbase.lookup.LookupTableToHFileJob.removeSnapshotIfExist(LookupTableToHFileJob.java:170)
at
org.apache.kylin.storage.hbase.lookup.LookupTableToHFileJob.run(LookupTableToHFileJob.java:104)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
at
org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableNotDisabledException):
KYLIN_LOOKUP_DEFAULT.F_ADVERTISER_DIM_KYLIN_TEST_9WCVUDIXI5
at
org.apache.hadoop.hbase.master.HMaster.checkTableModifiable(HMaster.java:2266)
at
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.prepareDelete(DeleteTableProcedure.java:253)
at
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:102)
at
org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:59)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:119)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:498)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1147)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:942)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:895)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:77)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:497)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
Liu Shaohui created KYLIN-3887:
--
Summary: Query with decimal sum measure of double complied failed
after KYLIN-3703
Key: KYLIN-3887
URL: https://issues.apache.org/jira/browse/KYLIN-3887
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
After KYLIN-3703, Query with decimal sum measure of double complied failed.
{code:java}
Caused by: org.codehaus.commons.compiler.CompileException:
Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
"java.lang.Double"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3886) Missing argument for options for yarn command
Liu Shaohui created KYLIN-3886: -- Summary: Missing argument for options for yarn command Key: KYLIN-3886 URL: https://issues.apache.org/jira/browse/KYLIN-3886 Project: Kylin Issue Type: Bug Reporter: Liu Shaohui 2019-03-13 11:48:08,604 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : Missing argument for options 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : usage: application 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -appStates Works with -list to filter applications 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on input comma-separated list of 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : application states. The valid application 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : state can be one of the following: 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUN 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : NING,FINISHED,FAILED,KILLED 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -appTypes Works with -list to filter applications 2019-03-13 11:48:08,606 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : based on input comma-separated list of 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : application types. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -help Displays help for all commands. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -kill Kills the application. 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : -list List applications. Supports optional use 2019-03-13 11:48:08,607 INFO [Scheduler 542945608 Job f918877a-deb0-704c-ec6f-82f33f5e39a5-323] spark.SparkExecutable:38 : of -appTypes to filter applications based -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3885) Build dimension dictionary job costs too long when using Spark fact distinct
Liu Shaohui created KYLIN-3885:
--
Summary: Build dimension dictionary job costs too long when using
Spark fact distinct
Key: KYLIN-3885
URL: https://issues.apache.org/jira/browse/KYLIN-3885
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
Build dimension dictionary job costs less than 20 minutes when using mapreduce
fact distinct,but but it costs more than 3 hours when using spark fact distinct.
{code:java}
"Scheduler 542945608 Job 05c62aca-853f-396e-9653-f20c9ebd8ebc-329" #329 prio=5
os_prio=0 tid=0x7f312109c800 nid=0x2dc0b in Object.wait()
[0x7f30d8d24000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.hadoop.ipc.Client.call(Client.java:1482)
- locked <0x0005c3110fc0> (a org.apache.hadoop.ipc.Client$Call)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy33.delete(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:573)
at sun.reflect.GeneratedMethodAccessor193.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:249)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:107)
at com.sun.proxy.$Proxy34.delete(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2057)
at
org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:682)
at
org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:675)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:696)
at
org.apache.hadoop.fs.FilterFileSystem.delete(FilterFileSystem.java:232)
at
org.apache.hadoop.fs.viewfs.ChRootedFileSystem.delete(ChRootedFileSystem.java:198)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.delete(ViewFileSystem.java:334)
at
org.apache.hadoop.hdfs.FederatedDFSFileSystem.delete(FederatedDFSFileSystem.java:232)
at
org.apache.kylin.dict.global.GlobalDictHDFSStore.deleteSlice(GlobalDictHDFSStore.java:211)
at
org.apache.kylin.dict.global.AppendTrieDictionaryBuilder.flushCurrentNode(AppendTrieDictionaryBuilder.java:137)
at
org.apache.kylin.dict.global.AppendTrieDictionaryBuilder.addValue(AppendTrieDictionaryBuilder.java:97)
at
org.apache.kylin.dict.GlobalDictionaryBuilder.addValue(GlobalDictionaryBuilder.java:85)
at
org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:82)
at
org.apache.kylin.dict.DictionaryManager.buildDictFromReadableTable(DictionaryManager.java:303)
at
org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:290)
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildDictionary(CubeManager.java:1043)
at
org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:1012)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:72)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3883) Kylin supports column count aggregation
xiaodongzhang created KYLIN-3883: Summary: Kylin supports column count aggregation Key: KYLIN-3883 URL: https://issues.apache.org/jira/browse/KYLIN-3883 Project: Kylin Issue Type: New Feature Components: Job Engine Affects Versions: all Reporter: xiaodongzhang Assignee: xiaodongzhang Fix For: v3.0.0 Kylin目前只支持对常量1进行count聚合预计算，即count(1)，count(col_1)的查询内部全部重写到对count(1)的查询，这样会导致一个问题：当col_1列中存在null值时，count(col_1)的查询结果并不准确。这样就导致Kylin的查询结果和Hive、Spark等不一致。该Patch中，提供了对count(col_1)的支持。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3884) loading hfile to HBase failed for temporary dir in output path
Liu Shaohui created KYLIN-3884:
--
Summary: loading hfile to HBase failed for temporary dir in
output path
Key: KYLIN-3884
URL: https://issues.apache.org/jira/browse/KYLIN-3884
Project: Kylin
Issue Type: Bug
Reporter: Liu Shaohui
{code:java}
2019-03-14 20:18:46,591 DEBUG [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] steps.BulkLoadJob:77 : Start to run
LoadIncrementalHFiles
2019-03-14 20:18:46,642 WARN [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:197 :
Skipping non-directory
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_SUCCESS
2019-03-14 20:18:46,650 ERROR [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] mapreduce.LoadIncrementalHFiles:352 :
-
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/4170d772384144848c1c10cba66152c3
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/50ec331ff3c648e3b6e4f54a7b1fe7e9
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/703ade3b535b4fedab39ee183e22aa7c
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/82019f8ca00a4f16b9d2b45356a55a3a
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/8cc8844bced24cb88fda52fecc7224d5
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/cbac78e0c6d74b5c96a7b64f99e0d0b3
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/F1/e3844766a4d0486d89f287450034f378
hdfs://zjyprc-xiaomi/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
2019-03-14 20:18:46,651 ERROR [Scheduler 2084224398 Job
e48de76a-6e16-309f-a3a5-191c04071072-131] common.HadoopShellExecutable:65 :
error execute HadoopShellExecutable{id=e48de76a-6e16-309f-a3a5-191c04071072-08,
name=Load HFile to HBase Table, state=RUNNING}
java.io.FileNotFoundException: Path is not a file:
/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0
Caused by:
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is
not a file:
/user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-e48de76a-6e16-309f-a3a5-191c04071072/total_user_cube/hfile/_temporary/0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
