Re: [VOTE] Release apache-kylin-2.0.0 (RC1)

[Dong Li]

+1 (binding)

mvn test passed
md5/sha/gpg verified

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-11T00:41:47+08:00)
Maven home: /usr/local/Cellar/maven/3.3.9/libexec
Java version: 1.7.0_79, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.4", arch: "x86_64", family: "mac"

Thanks,
Dong Li

2017-04-22 13:26 GMT+08:00 Li Yang :

> Hi all,
>
> I have created a build for Apache Kylin 2.0.0, release candidate 1.
>
> Changes highlights:
>
> Support snowflake data model (KYLIN-1875)
> Support TPC-H queries (KYLIN-2467)
> Spark cubing engine (KYLIN-2331)
> Job engine HA (KYLIN-2006)
> Percentile measure (KYLIN-2396)
> Cloud tested (KYLIN-2351)
>
>
> Thanks to everyone who has contributed to this release. Here is release
> notes:
> http://kylin.apache.org/docs20/release_notes.html
>
> The commit to be voted upon (199f5db467c224b91c32632fca5e26c4745bdc66):
> https://github.com/apache/kylin/tree/kylin-2.0.0
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.0.0-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachekylin-1039/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/liyang.asc
>
> Please vote on releasing this package as Apache Kylin 2.0.0.
>
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PPMC votes are cast.
>
> [ ] +1 Release this package as Apache Kylin 2.0.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
>
> Cheers
> Yang
>

[jira] [Created] (KYLIN-2560) Fix license headers for 2.0.0 release

[liyang (JIRA)]

liyang created KYLIN-2560:
-

 Summary: Fix license headers for 2.0.0 release
 Key: KYLIN-2560
 URL: https://issues.apache.org/jira/browse/KYLIN-2560
 Project: Kylin
  Issue Type: Improvement
Reporter: liyang
Assignee: liyang






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Rowkey in Advance Setting of Cube

[rahulsingh]

Hi All,

I am totally confused about Rowkey in Advance Setting of Cube.

I am deleting some dimensions from Rowkey and save the Cube, then I see the
cube after saving, it remains same.

My question is what will happen if I remove the dimensions from the Rowkey.
Will everything work correctly? UI allows to delete sections of the Rowkey??

Thank You
Rahul Singh

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/Rowkey-in-Advance-Setting-of-Cube-tp7728.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

[jira] [Created] (KYLIN-2559) Failed to create HDFS directory because of the property 'kylin.env.hdfs-working-dir' can not be empty in check-env.sh but we didn't check it in this script.

[peng.jianhua (JIRA)]

peng.jianhua created KYLIN-2559:
---

 Summary: Failed to create HDFS directory because of the property 
'kylin.env.hdfs-working-dir' can not be empty in check-env.sh but we didn't 
check it in this script.
 Key: KYLIN-2559
 URL: https://issues.apache.org/jira/browse/KYLIN-2559
 Project: Kylin
  Issue Type: Bug
  Components: General
Affects Versions: v2.0.0
Reporter: peng.jianhua
Assignee: peng.jianhua
Priority: Minor
 Fix For: v2.0.0


Failed to create HDFS directory because of the property 
'kylin.env.hdfs-working-dir' can not be empty in check-env.sh but we didn't 
check it in this script.
The error log is as following:
-mkdir: Not enough arguments: expected 1 but got 0
Usage: hadoop fs [generic options] -mkdir [-p]  ...
Failed to create . Please make sure the user has right to access 

We should check if the property 'kylin.env.hdfs-working-dir' is empty in 
check-env.sh.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Fail to extract disctinct columns from fact tables

[ShaoFeng Shi]

Check this:  java.lan.IllegalArgumentException: Cannot modify
hive.security.authorization.sqlstd.confwhitelist.append at runtime. It is
not in list of params that are allowed to be modified at runtime (state=,
code=0)

FusionInsight has very strict security control, which doesn't allow client
to overwrite Hive/mapreduce/hbase configuration values; while Kylin
customizes many configurations for its performance and functionality.

To overcome such error, you need contact your FI administrator, and tell
him to add the parameter to FI's whitelist. You can do this in batch for
all parameters in KYLIN_HOME/conf/*.xml, but there might be other
parameters from application.

2017-04-21 14:19 GMT+08:00 Hong Wei :

> I do confirm kylin has got the right hive-site.xml on classpath.
> Additionally, when I add
> --hiveconf hive.security.authorization.sqlstd.confwhitelist.append='
> mapred.job.*|dfs.*'
> into beeline params which is configured in conf/kylin.properties, a error
> is reported while starting kylin:
>
>
> Error: Fail to open new session: org.apache.hive.service.cli.HiveSQLException:
> java.lan.IllegalArgumentException: Cannot modify
> hive.security.authorization.sqlstd.confwhitelist.append at runtime. It is
> not in list of params that are allowed to be modified at runtime (state=,
> code=0)
>
>
>
>
> -- 原始邮件 --
> 发件人: "Li Yang";;
> 发送时间: 2017年4月21日(星期五) 中午1:02
> 收件人: "dev";
>
> 主题: Re: Fail to extract disctinct columns from fact tables
>
>
>
> I recall checking if Kylin has got the right hive-site.xml on classpath may
> fix this problem.
>
> On Thu, Apr 20, 2017 at 8:13 PM, Hong Wei  wrote:
>
> > Dear Kylin:
> > I try to run Kylin (v1.6.0) on HuaWei FusionInsight with Kerberos
> > authorization (Hadoop v2.7.0, Hive v1.3.0, HBase v1.0.2). I use Beeline
> > mode to visit hive with config like:
> > kylin.hive.beeline.params=-n [username] -u 'jdbc:hive2://ip:port;
> > principal=[principal];auth=KERBEROS;sasl.qop=auto-conf'
> >
> > When I tried to build the sample cube, a mistake token place in the
> > 3rd step: 'extract the distinct columns from fact tables'. The Part of
> > error log like this:
> > WARN hive.metastore:574 : set_ugi() not suuccessful, Likely cause:
> new
> > client talking to old server. Continuing without it.
> > org.apache.thrift.transport.TTransportException
> >at org.apache.thrift.transport.TIOStreamTransport.read(
> > TIOStreamTransport.java:132)
> >at org.apache.thrift.transport.TTransport.readAll(TTransport.
> > java:86)
> >
> >at org.apache.hadoop.hive.metastore.api.
> ThriftHiveMetastore$Client.
> > recv_set_ugi(ThriftHiveMetastore.java:3794)
> >at org.apache.hadoop.hive.metastore.api.
> ThriftHiveMetastore$Client.
> > set_ugi(ThriftHiveMetastore.java:3780)
> >at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> > open(HiveMetaStoreClient:566)
> >at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> > reconnect(HiveMetaStoreClient:342)
> >   ...
> >at  org.apache.hive.hcatalog.common.HCatUtil.getTable(
> > HCatUtil.java:180)
> >   ...
> >at  org.apache.kylin.source.hive.HiveMRInputFormat.
> > configureJob(HiveMRInput.java:105)
> >at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> > setupMapper(FactDistinctColumnsJob.java:119)
> >at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> > run(FactDistinctColumnsJob.java:103)
> >..
> >INFOhive.metastore:602 :Connect to metastore.
> >INFOcommon.AbstractHadoopJob:506 : tempMetaFileString is :
> null
> >ERROR common.MapReduceExecutable:127 : error execute
> > MapReduceExecutable{id=..,name=Extract Fact Table Distinct Columns,
> > state=RUNNING}
> >
> >
> > Unfortunatly, I have no idea to solve this problem, I need some help.
> > Thanks.
> >
> >
> >
> >Sincerely yours
>



-- 
Best regards,

Shaofeng Shi 史少锋

?????? Fail to extract disctinct columns from fact tables

[Hong Wei]

I do confirm kylin has got the right hive-site.xml on classpath.
Additionally, when I add 
--hiveconf 
hive.security.authorization.sqlstd.confwhitelist.append='mapred.job.*|dfs.*' 
into beeline params which is configured in conf/kylin.properties, a error is 
reported while starting kylin:


Error: Fail to open new session: org.apache.hive.service.cli.HiveSQLException: 
java.lan.IllegalArgumentException: Cannot modify 
hive.security.authorization.sqlstd.confwhitelist.append at runtime. It is not 
in list of params that are allowed to be modified at runtime (state=, code=0)




--  --
??: "Li Yang";;
: 2017??4??21??(??) 1:02
??: "dev"; 

: Re: Fail to extract disctinct columns from fact tables



I recall checking if Kylin has got the right hive-site.xml on classpath may
fix this problem.

On Thu, Apr 20, 2017 at 8:13 PM, Hong Wei  wrote:

> Dear Kylin:
> I try to run Kylin (v1.6.0) on HuaWei FusionInsight with Kerberos
> authorization (Hadoop v2.7.0, Hive v1.3.0, HBase v1.0.2). I use Beeline
> mode to visit hive with config like:
> kylin.hive.beeline.params=-n [username] -u 'jdbc:hive2://ip:port;
> principal=[principal];auth=KERBEROS;sasl.qop=auto-conf'
>
> When I tried to build the sample cube, a mistake token place in the
> 3rd step: 'extract the distinct columns from fact tables'. The Part of
> error log like this:
> WARN hive.metastore:574 : set_ugi() not suuccessful, Likely cause: new
> client talking to old server. Continuing without it.
> org.apache.thrift.transport.TTransportException
>at org.apache.thrift.transport.TIOStreamTransport.read(
> TIOStreamTransport.java:132)
>at org.apache.thrift.transport.TTransport.readAll(TTransport.
> java:86)
>
>at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
> recv_set_ugi(ThriftHiveMetastore.java:3794)
>at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
> set_ugi(ThriftHiveMetastore.java:3780)
>at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> open(HiveMetaStoreClient:566)
>at  org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> reconnect(HiveMetaStoreClient:342)
>   ...
>at  org.apache.hive.hcatalog.common.HCatUtil.getTable(
> HCatUtil.java:180)
>   ...
>at  org.apache.kylin.source.hive.HiveMRInputFormat.
> configureJob(HiveMRInput.java:105)
>at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> setupMapper(FactDistinctColumnsJob.java:119)
>at  org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
> run(FactDistinctColumnsJob.java:103)
>..
>INFOhive.metastore:602 :Connect to metastore.
>INFOcommon.AbstractHadoopJob:506 : tempMetaFileString is : null
>ERROR common.MapReduceExecutable:127 : error execute
> MapReduceExecutable{id=..,name=Extract Fact Table Distinct Columns,
> state=RUNNING}
>
>
> Unfortunatly, I have no idea to solve this problem, I need some help.
> Thanks.
>
>
>
>Sincerely yours