date:20170420

Re: how to make the kylin to chinese

2017-04-20 Thread Li Yang

Installation guide is here:
http://kylin.apache.org/docs20/install/index.html
If you want to build Kylin from source, guide is here:
http://kylin.apache.org/development/howto_package.html

Cheers
Yang

On Tue, Apr 18, 2017 at 8:25 PM,  wrote:

> Hello
> Boy!
> I want to know how to make the kylin to chinese .i have been
> download the resource of the kylin2.0 on the cdh5.7 from github.but I don’t
> know how to make it to the installed package. Do you help me !
>
>
>   A chinese boy
>
> 发送自 Windows 10 版邮件应用
>
>

Re: Fail to extract disctinct columns from fact tables

2017-04-20 Thread Li Yang

I recall checking if Kylin has got the right hive-site.xml on classpath may
fix this problem.

On Thu, Apr 20, 2017 at 8:13 PM, Hong Wei  wrote:

> Dear Kylin:
> I try to run Kylin (v1.6.0) on HuaWei FusionInsight with Kerberos
> authorization (Hadoop v2.7.0, Hive v1.3.0, HBase v1.0.2). I use Beeline
> mode to visit hive with config like:
> kylin.hive.beeline.params=-n [username] -u 'jdbc:hive2://ip:port;
> principal=[principal];auth=KERBEROS;sasl.qop=auto-conf'
>
> When I tried to build the sample cube, a mistake token place in the
> 3rd step: 'extract the distinct columns from fact tables'. The Part of
> error log like this:
> WARN hive.metastore:574 : set_ugi() not suuccessful, Likely cause: new
> client talking to old server. Continuing without it.
> org.apache.thrift.transport.TTransportException
>at org.apache.thrift.transport.TIOStreamTransport.read(
> TIOStreamTransport.java:132)
>at org.apache.thrift.transport.TTransport.readAll(TTransport.
> java:86)
>
>at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
> recv_set_ugi(ThriftHiveMetastore.java:3794)
>at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
> set_ugi(ThriftHiveMetastore.java:3780)
>at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> open(HiveMetaStoreClient:566)
>at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> reconnect(HiveMetaStoreClient:342)
>   ...
>at  org.apache.hive.hcatalog.common.HCatUtil.getTable(
> HCatUtil.java:180)
>   ...
>at  org.apache.kylin.source.hive.HiveMRInputFormat.
> configureJob(HiveMRInput.java:105)
>at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> setupMapper(FactDistinctColumnsJob.java:119)
>at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> run(FactDistinctColumnsJob.java:103)
>..
>INFOhive.metastore:602 :Connect to metastore.
>INFOcommon.AbstractHadoopJob:506 : tempMetaFileString is : null
>ERROR common.MapReduceExecutable:127 : error execute
> MapReduceExecutable{id=..,name=Extract Fact Table Distinct Columns,
> state=RUNNING}
>
>
> Unfortunatly, I have no idea to solve this problem, I need some help.
> Thanks.
>
>
>
>Sincerely yours

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

2017-04-20 Thread Li Yang

I've updated that JIRA's owner.

Yang

On Fri, Apr 21, 2017 at 12:00 PM, Luke Han  wrote:

> Hi Peng,
>  The community will check and review patches later, it's all volunteer
> work, please check that JIRA which will be updated when one committer has
> bandwidth to do that.
>
> Thanks for your contribution.
> Luke
>
>
>
>
>
> Best Regards!
> -
>
> Luke Han
>
> On Fri, Apr 21, 2017 at 9:18 AM,  wrote:
>
> >
>

how to make the kylin to chinese

2017-04-20 Thread zptang

Hello 
Boy!
I want to know how to make the kylin to chinese .i have been download 
the resource of the kylin2.0 on the cdh5.7 from github.but I don’t know how to 
make it to the installed package. Do you help me !


A chinese boy

发送自 Windows 10 版邮件应用

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

2017-04-20 Thread Luke Han

Hi Peng,
 The community will check and review patches later, it's all volunteer
work, please check that JIRA which will be updated when one committer has
bandwidth to do that.

Thanks for your contribution.
Luke





Best Regards!
-

Luke Han

On Fri, Apr 21, 2017 at 9:18 AM,  wrote:

>

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

2017-04-20 Thread Billy Liu

Currently, JIRA could not find the right user by name "peng.jianhua". Is
this the name you login JIRA system?

2017-04-21 9:18 GMT+08:00 :

>

Review Request 58554: KYLIN-2557:The kylin will start failed because of creating HBase table confict when multiple instances of the kylin are started at the same time.

2017-04-20 Thread pengjianhua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58554/
---

Review request for kylin, Dayue Gao, hongbin ma, and Wang Xiaoyu.


Bugs: KYLIN-2557
https://issues.apache.org/jira/browse/KYLIN-2557


Repository: kylin


Description
---

Muliple instances were started at the same time after we installed the kylin. A 
conflict will occurs because these instances cteate kylin_metadata_acl table at 
the same time. The result is that the kylin start failed. The error log is as 
following:
Caused by: org.apache.hadoop.hbase.TableExistsException: kylin_metadata_acl
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at 
org.apache.hadoop.hbase.util.ForeignExceptionUtil.toIOException(ForeignExceptionUtil.java:45)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.convertResult(HBaseAdmin.java:4448)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:4406)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:4339)
at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:604)
at 
org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:307)
at 
org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:262)
at 
org.apache.kylin.rest.security.RealAclHBaseStorage.prepareHBaseTable(RealAclHBaseStorage.java:50)
at 
org.apache.kylin.rest.security.MockAclHBaseStorage.prepareHBaseTable(MockAclHBaseStorage.java:53)
at org.apache.kylin.rest.service.AclService.init(AclService.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:344)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:295)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)
... 125 more


Diffs
-

  
storage-hbase/src/main/java/org/apache/kylin/storage/hbase/HBaseConnection.java 
7e2cefc69 
  
storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/ZookeeperCreateHTableLock.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/58554/diff/1/


Testing
---


Thanks,

pengjianhua

Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR and r

2017-04-20 Thread peng.jianhua

Re: 2.0.0 beta3 package updated

2017-04-20 Thread Li Yang

It is stable as latest test showed and a few known issues are resolved. I'm
starting the release in this weekend.

Yang

On Wed, Apr 19, 2017 at 5:05 PM, Luke Han  wrote:

>
>
>
>
>
>
>
>
> Please try to start release process for 2.0 if such tech
> preview packages helped stable this branch.
> Thanks
>
>
>
>
>
>
>
>
>
> On Tue, Apr 18, 2017 at 10:22 PM +0800, "Li Yang" 
> wrote:
>
>
>
>
>
>
>
>
>
>
> Hi all,
>
> FYI that I just updated the 2.0.0. beta3 package on the download page
> .
>
> Since the beta, we have received many feedback and the release branch 2.0.x
> has been active -- which is not exactly what I hoped. Let's see if things
> get stable and the branch cools down.
>
> Still think the 2.0.0 release will be out in April.
>
> Cheers
> Yang
>
>
>
>
>
>

Fail to extract disctinct columns from fact tables

2017-04-20 Thread Hong Wei

Dear Kylin:
I try to run Kylin (v1.6.0) on HuaWei FusionInsight with Kerberos 
authorization (Hadoop v2.7.0, Hive v1.3.0, HBase v1.0.2). I use Beeline mode to 
visit hive with config like:
kylin.hive.beeline.params=-n [username] -u 
'jdbc:hive2://ip:port;principal=[principal];auth=KERBEROS;sasl.qop=auto-conf' 

When I tried to build the sample cube, a mistake token place in the 3rd 
step: 'extract the distinct columns from fact tables'. The Part of error log 
like this:
WARN hive.metastore:574 : set_ugi() not suuccessful, Likely cause: new 
client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
   at 
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
   at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
   
   at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3794)
   at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3780)
   at  
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient:566)
   at  
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.reconnect(HiveMetaStoreClient:342)
  ...
   at  org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180)
  ...
   at  
org.apache.kylin.source.hive.HiveMRInputFormat.configureJob(HiveMRInput.java:105)
   at  
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:119)
   at  
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:103)
   ..
   INFOhive.metastore:602 :Connect to metastore.
   INFOcommon.AbstractHadoopJob:506 : tempMetaFileString is : null
   ERROR common.MapReduceExecutable:127 : error execute 
MapReduceExecutable{id=..,name=Extract Fact Table Distinct Columns, 
state=RUNNING}


Unfortunatly, I have no idea to solve this problem, I need some help. 
Thanks.



 Sincerely yours

[jira] [Created] (KYLIN-2558) ODBC: SQLPrepare(SQLPrepareW) should prepare query metadata

2017-04-20 Thread Yifan Zhang (JIRA)

Yifan Zhang created KYLIN-2558:
--

 Summary: ODBC: SQLPrepare(SQLPrepareW) should prepare query 
metadata
 Key: KYLIN-2558
 URL: https://issues.apache.org/jira/browse/KYLIN-2558
 Project: Kylin
  Issue Type: Bug
  Components: Driver - ODBC
Affects Versions: v1.6.0
Reporter: Yifan Zhang


When use SSRS BI to save query to dataset, error will be thrown out.

Root cause, SQLPrepareW does not prepare column info and SQLNumResultCols will 
fail.
{code:title=driver_debug.log}
[DEBUG]SQLPrepareW called on: 191574520
[INFO ]SQLFreeStmt called, 191574520 with option 0
[DEBUG]SQLFreeStmt called for Result
[DEBUG]The query being prepared is :
[DEBUG]SELECT   COUNT(*) AS AA

FROM  kylin_sales
[DEBUG]SQLNumResultCols called
[DEBUG]_SQLPutDiagRow called, handle type is 3
[DEBUG]DiagMsg: [Kylin][ODBC 1.0(w) Driver]no resultset or IRD
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

????: build cube?????????????????????????????????????????? unable to create new native thread

2017-04-20 Thread roger shi

Hi,

The first step is running some Hive command. It seems the default hive memory 
configuration is not enough. Please check the hive configuration in your 
environment. You can also adjust the configuration in Kylin's hive 
configuration file (conf/kylin_hive_conf.xml).

??: 35925138 <35925...@qq.com>
: 2017??4??21?? 8:16:51
??: dev
: build cube?? unable to create new 
native thread

build cube??


Exception in thread "main" java.lang.OutOfMemoryError: unable to create new 
native threadat java.lang.Thread.start0(Native Method)at 
java.lang.Thread.start(Thread.java:714)   at 
org.apache.hadoop.util.Shell.runCommand(Shell.java:521)   at 
org.apache.hadoop.util.Shell.run(Shell.java:455)  at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) 
at org.apache.hadoop.util.Shell.isSetsidSupported(Shell.java:390)at 
org.apache.hadoop.util.Shell.(Shell.java:380) at 
org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:2327)
at 
org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:365) 
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:58)   
 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:637)  at 
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
  at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)  at 
org.apache.hadoop.util.RunJar.run(RunJar.java:221)at 
org.apache.hadoop.util.RunJar.main(RunJar.java:136)

build cube?????????????????????????????????????????? unable to create new native thread

2017-04-20 Thread 35925138

build cube??


Exception in thread "main" java.lang.OutOfMemoryError: unable to create new 
native thread   at java.lang.Thread.start0(Native Method)   at 
java.lang.Thread.start(Thread.java:714)  at 
org.apache.hadoop.util.Shell.runCommand(Shell.java:521)  at 
org.apache.hadoop.util.Shell.run(Shell.java:455) at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)at 
org.apache.hadoop.util.Shell.isSetsidSupported(Shell.java:390)   at 
org.apache.hadoop.util.Shell.(Shell.java:380)at 
org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:2327)
   at org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:365) 
   at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:58)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:637) at 
org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483) at 
org.apache.hadoop.util.RunJar.run(RunJar.java:221)   at 
org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Fail to extract disctinct columns from fact tables

2017-04-20 Thread Hong Wei

Dear Kylin:
I try to run Kylin (v1.6.0) on HuaWei FusionInsight with Kerberos 
authorization (Hadoop v2.7.0, Hive v1.3.0, HBase v1.0.2). I use Beeline mode to 
visit hive with config like:
kylin.hive.beeline.params=-n [username] -u 
'jdbc:hive2://ip:port;principal=[principal];auth=KERBEROS;sasl.qop=auto-conf' 

When I tried to build the sample cube, a mistake token place in the 3rd 
step: 'extract the distinct columns from fact tables'. The Part of error log 
like this:
WARN hive.metastore:574 : set_ugi() not suuccessful, Likely cause: new 
client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
   at 
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
   at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
   
   at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3794)
   at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3780)
   at  
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient:566)
   at  
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.reconnect(HiveMetaStoreClient:342)
  ...
   at  org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180)
  ...
   at  
org.apache.kylin.source.hive.HiveMRInputFormat.configureJob(HiveMRInput.java:105)
   at  
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:119)
   at  
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:103)
   ..
   INFOhive.metastore:602 :Connect to metastore.
   INFOcommon.AbstractHadoopJob:506 : tempMetaFileString is : null
   ERROR common.MapReduceExecutable:127 : error execute 
MapReduceExecutable{id=..,name=Extract Fact Table Distinct Columns, 
state=RUNNING}


Unfortunatly, I have no idea to solve this problem, I need some help. 
Thanks.



 Sincerely yours

Re: 答复: 回复：答复: Error: java.io.IOException: Unable to initialize any output collector

2017-04-20 Thread ShaoFeng Shi

Cool, enjoy apache kylin!

2017-04-20 14:36 GMT+08:00 35925138 <35925...@qq.com>:

> 非常感谢，问题已经解决，原因是我的hadoop2.6.0在执行mr任务时，无法获取classpath，导致加载不到hadoop_lzo.jar
> 我看了一下yarn-site.xml，发现了一个配置mapreduce.application.classpath，可以用来指定classpath,
> 修改此项，或者将hadoop_lzo.jar复制到$HADOOP_HOME/share/hadoop/mapreduce/lib/下即可，
> 该目录是mapreduce.application.classpath配置的默认目录
>
>
>
>
> -- 原始邮件 --
> 发件人: "ShaoFeng Shi";;
> 发送时间: 2017年4月20日(星期四) 上午9:08
> 收件人: "dev";
>
> 主题: Re: 答复: 回复：答复: Error: java.io.IOException: Unable to initialize any
> output collector
>
>
>
> Hive is Kylin's source, not computing engine; Kylin starts its own MR jobs
> to build the cube. If need specify additional parameters into MR jobs, add
> them in conf/kylin_job*.xml, or you can customize them at cube level:
> https://kylin.apache.org/docs20/install/advance_settings.html
>
> 2017-04-19 16:24 GMT+08:00 35925138 <35925...@qq.com>:
>
> > kylin.hbase.default.compression.codec=lzo
> >
> > 
> > mapreduce.map.output.compress.codec
> > com.hadoop.compression.lzo.LzoCodec
> > 
> > 
> >
> > 
> > mapreduce.output.fileoutputformat.compress.codec
> > com.hadoop.compression.lzo.LzoCodec
> > 
> > 
> >
> >
> >
> > 应该是这样修改吧。
> >
> >
> >
> >
> > -- 原始邮件 --
> > 发件人: "ShaoFeng Shi";;
> > 发送时间: 2017年4月19日(星期三) 下午3:40
> > 收件人: "dev";
> >
> > 主题: Re: 答复: 回复：答复: Error: java.io.IOException: Unable to initialize any
> > output collector
> >
> >
> >
> > To change use lzo:
> > 1) Hive/MR compression: search "SnappyCodec" in conf/*.xml file, replace
> > them with lzo's codec
> > 2) HBase compression: in conf/kylin.properties, set
> > "kylin.hbase.default.compression.codec" to "lzo"
> >
> > 2017-04-19 10:56 GMT+08:00 35925138 <35925...@qq.com>:
> >
> > > 这个问题，我通过设置hive-site.xml里的hive.aux.jars.path，将hadoop-
> > lzo的那几个jar强行写进去，解决的，不过在build
> > > cube的第三步，有爆出了这样的错误，
> > > yarn里爆出的错误为：
> > > Error: java.io.IOException: Unable to initialize any output collector
> at
> > > org.apache.hadoop.mapred.MapTask.createSortingCollector(
> > MapTask.java:412)
> > > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81) at
> > > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<
> > init>(MapTask.java:695)
> > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767) at
> > > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
> > > java.security.AccessController.doPrivileged(Native Method) at
> > > javax.security.auth.Subject.doAs(Subject.java:422) at
> > > org.apache.hadoop.security.UserGroupInformation.doAs(
> > > UserGroupInformation.java:1628) at org.apache.hadoop.mapred.
> > > YarnChild.main(YarnChild.java:158) Container killed by the
> > > ApplicationMaster. Container killed on request. Exit code is 143
> > Container
> > > exited with a non-zero exit code 143
> > >
> > > 我查看日志，日志中有这样的信息
> > >
> > > 2017-04-19 10:48:15,875 WARN [main] org.apache.hadoop.mapred.MapTask:
> > > Unable to initialize MapOutputCollector org.apache.hadoop.mapred.
> > > MapTask$MapOutputBuffer
> > > java.lang.IllegalArgumentException: Compression codec
> > > com.hadoop.compression.lzo.LzoCodec was not found.
> > > at org.apache.hadoop.mapred.JobConf.
> getMapOutputCompressorClass(
> > > JobConf.java:798)
> > > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(
> > > MapTask.java:1019)
> > > at org.apache.hadoop.mapred.MapTask.createSortingCollector(
> > > MapTask.java:401)
> > > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.
> java:81)
> > > at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<
> > > init>(MapTask.java:695)
> > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.
> > java:767)
> > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.
> java:163)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at javax.security.auth.Subject.doAs(Subject.java:422)
> > > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > > UserGroupInformation.java:1628)
> > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > > Caused by: java.lang.ClassNotFoundException: Class
> > > com.hadoop.compression.lzo.LzoCodec not found
> > > at org.apache.hadoop.conf.Configuration.getClassByName(
> > > Configuration.java:1980)
> > > at org.apache.hadoop.mapred.JobConf.
> getMapOutputCompressorClass(
> > > JobConf.java:796)
> > > ... 11 more
> > >
> > > 我在hadoop中，执行hadoop/bin/hadoop checknative -a
> > > 给出的信息为：
> > > 17/04/19 09:55:10 WARN bzip2.Bzip2Factory: Failed to load/initialize
> > > native-bzip2 library system-native, will use pure-Java version
> > > 17/04/19 09:55:10 INFO

答复: [jira] [Created] (KYLIN-2557) The kylin will start failed because of creating HBase table confict when multiple instances of the kylin are started at the same time.

2017-04-20 Thread roger shi

Hi jianhua,

The short term workaround is to start an instance first and others later.

发件人: peng.jianhua (JIRA) 
发送时间: 2017年4月20日 13:43:04
收件人: dev@kylin.apache.org
主题: [jira] [Created] (KYLIN-2557) The kylin will start failed because of 
creating HBase table confict when multiple instances of the kylin are started 
at the same time.

peng.jianhua created KYLIN-2557:
---

 Summary: The kylin will start failed because of creating HBase 
table confict when multiple instances of the kylin are started at the same time.
 Key: KYLIN-2557
 URL: https://issues.apache.org/jira/browse/KYLIN-2557
 Project: Kylin
  Issue Type: Bug
  Components: Storage - HBase
Affects Versions: v2.0.0
Reporter: peng.jianhua
Assignee: liyang
Priority: Critical


Muliple instances were started at the same time after we installed the kylin. A 
conflict will occurs because these instances cteate kylin_metadata_acl table at 
the same time. The result is that the kylin start failed. The error log is as 
following:
Caused by: org.apache.hadoop.hbase.TableExistsException: kylin_metadata_acl
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at 
org.apache.hadoop.hbase.util.ForeignExceptionUtil.toIOException(ForeignExceptionUtil.java:45)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.convertResult(HBaseAdmin.java:4448)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:4406)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:4339)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:604)
at 
org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:307)
at 
org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:262)
at 
org.apache.kylin.rest.security.RealAclHBaseStorage.prepareHBaseTable(RealAclHBaseStorage.java:50)
at 
org.apache.kylin.rest.security.MockAclHBaseStorage.prepareHBaseTable(MockAclHBaseStorage.java:53)
at org.apache.kylin.rest.service.AclService.init(AclService.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:344)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:295)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)
... 125 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

?????? ????: ??????????: Error: java.io.IOException: Unable to initialize any output collector

2017-04-20 Thread 35925138

??hadoop2.6.0??mrclasspath??hadoop_lzo.jar
??yarn-site.xmlmapreduce.application.classpath??classpath,hadoop_lzo.jar??$HADOOP_HOME/share/hadoop/mapreduce/lib/mapreduce.application.classpath??




--  --
??: "ShaoFeng Shi";;
: 2017??4??20??(??) 9:08
??: "dev"; 

: Re: : ??: Error: java.io.IOException: Unable to initialize 
any output collector



Hive is Kylin's source, not computing engine; Kylin starts its own MR jobs
to build the cube. If need specify additional parameters into MR jobs, add
them in conf/kylin_job*.xml, or you can customize them at cube level:
https://kylin.apache.org/docs20/install/advance_settings.html

2017-04-19 16:24 GMT+08:00 35925138 <35925...@qq.com>:

> kylin.hbase.default.compression.codec=lzo
>
> 
> mapreduce.map.output.compress.codec
> com.hadoop.compression.lzo.LzoCodec
> 
> 
>
> 
> mapreduce.output.fileoutputformat.compress.codec
> com.hadoop.compression.lzo.LzoCodec
> 
> 
>
>
>
> ??
>
>
>
>
> --  --
> ??: "ShaoFeng Shi";;
> : 2017??4??19??(??) 3:40
> ??: "dev";
>
> : Re: : ??: Error: java.io.IOException: Unable to initialize 
> any
> output collector
>
>
>
> To change use lzo:
> 1) Hive/MR compression: search "SnappyCodec" in conf/*.xml file, replace
> them with lzo's codec
> 2) HBase compression: in conf/kylin.properties, set
> "kylin.hbase.default.compression.codec" to "lzo"
>
> 2017-04-19 10:56 GMT+08:00 35925138 <35925...@qq.com>:
>
> > hive-site.xmlhive.aux.jars.pathhadoop-
> lzojar??build
> > cube??
> > yarn
> > Error: java.io.IOException: Unable to initialize any output collector at
> > org.apache.hadoop.mapred.MapTask.createSortingCollector(
> MapTask.java:412)
> > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81) at
> > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<
> init>(MapTask.java:695)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767) at
> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
> > java.security.AccessController.doPrivileged(Native Method) at
> > javax.security.auth.Subject.doAs(Subject.java:422) at
> > org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1628) at org.apache.hadoop.mapred.
> > YarnChild.main(YarnChild.java:158) Container killed by the
> > ApplicationMaster. Container killed on request. Exit code is 143
> Container
> > exited with a non-zero exit code 143
> >
> > ??
> >
> > 2017-04-19 10:48:15,875 WARN [main] org.apache.hadoop.mapred.MapTask:
> > Unable to initialize MapOutputCollector org.apache.hadoop.mapred.
> > MapTask$MapOutputBuffer
> > java.lang.IllegalArgumentException: Compression codec
> > com.hadoop.compression.lzo.LzoCodec was not found.
> > at org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(
> > JobConf.java:798)
> > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(
> > MapTask.java:1019)
> > at org.apache.hadoop.mapred.MapTask.createSortingCollector(
> > MapTask.java:401)
> > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
> > at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<
> > init>(MapTask.java:695)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.
> java:767)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1628)
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.lang.ClassNotFoundException: Class
> > com.hadoop.compression.lzo.LzoCodec not found
> > at org.apache.hadoop.conf.Configuration.getClassByName(
> > Configuration.java:1980)
> > at org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(
> > JobConf.java:796)
> > ... 11 more
> >
> > hadoophadoop/bin/hadoop checknative -a
> > ??
> > 17/04/19 09:55:10 WARN bzip2.Bzip2Factory: Failed to load/initialize
> > native-bzip2 library system-native, will use pure-Java version
> > 17/04/19 09:55:10 INFO zlib.ZlibFactory: Successfully loaded &
> initialized
> > native-zlib library
> > Native library

Re: how to make the kylin to chinese

Re: Fail to extract disctinct columns from fact tables

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

how to make the kylin to chinese

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

Re: Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR an

Review Request 58554: KYLIN-2557:The kylin will start failed because of creating HBase table confict when multiple instances of the kylin are started at the same time.

Hi PMS, Do you can add me as contributor for kylin and assign the https://issues.apache.org/jira/browse/KYLIN-2557, which I reported, to me? For KYLIN-2557 I had submitted patch, created its RR and r

Re: 2.0.0 beta3 package updated

Fail to extract disctinct columns from fact tables

[jira] [Created] (KYLIN-2558) ODBC: SQLPrepare(SQLPrepareW) should prepare query metadata

????: build cube?????????????????????????????????????????? unable to create new native thread

build cube?????????????????????????????????????????? unable to create new native thread

Fail to extract disctinct columns from fact tables

Re: 答复: 回复：答复: Error: java.io.IOException: Unable to initialize any output collector

答复: [jira] [Created] (KYLIN-2557) The kylin will start failed because of creating HBase table confict when multiple instances of the kylin are started at the same time.

?????? ????: ??????????: Error: java.io.IOException: Unable to initialize any output collector

17 matches

Site Navigation

Mail list logo

Footer information