[jira] [Resolved] (KYLIN-2668) Kylin JDBC Driver doesn't support Calcites Properties in URL

2017-06-28 Thread Shaofeng SHI (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI resolved KYLIN-2668.
-
Resolution: Fixed

> Kylin JDBC Driver doesn't support Calcites Properties in URL
> 
>
> Key: KYLIN-2668
> URL: https://issues.apache.org/jira/browse/KYLIN-2668
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC
>Affects Versions: 1.x-HBase1.1.3
>Reporter: Joe Swingle
>Assignee: Joe Swingle
> Fix For: v2.1.0
>
> Attachments: KYLIN-2668-v2.patch
>
>
> We currently access our OLAP Cubes in Kylin from our BI Tool over JDBC.  We 
> run Kylin over SSL.  When using our COTS BI Tool, we cannot programmatically 
> pass in property values such as ssl=True into a java.util.Properties.   We 
> can only specify a Username, Password and URL.
> According to Calcite we should be able to provide some properties in the JDBC 
> URL.  Line 142 of org.apache.calcite.avatica.UnregisteredDriver, builds java 
> properties based on the URL.   This works for Kylin.  For example, 
> "jdbc:kylin:ssl=True;//kylinserver.com/Projectname"   Will correctly 
> establish the connection as using SSL.
> The problem is in KylinConnection's Constructor.  It splits the jdbcURL on 
> "jdbc:kylin://".  So, it ignores the SSL. (KylinConnection.  Line 54-60.)
> The KylinConnection needs to be updated to support this method for setting 
> properties, so it can be used by 3rd party applications where programing the 
> connection is not possible.
> We have coded this locally, and will submit patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2668) Kylin JDBC Driver doesn't support Calcites Properties in URL

2017-06-28 Thread Shaofeng SHI (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI updated KYLIN-2668:

Fix Version/s: v2.1.0

> Kylin JDBC Driver doesn't support Calcites Properties in URL
> 
>
> Key: KYLIN-2668
> URL: https://issues.apache.org/jira/browse/KYLIN-2668
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC
>Affects Versions: 1.x-HBase1.1.3
>Reporter: Joe Swingle
>Assignee: Joe Swingle
> Fix For: v2.1.0
>
> Attachments: KYLIN-2668-v2.patch
>
>
> We currently access our OLAP Cubes in Kylin from our BI Tool over JDBC.  We 
> run Kylin over SSL.  When using our COTS BI Tool, we cannot programmatically 
> pass in property values such as ssl=True into a java.util.Properties.   We 
> can only specify a Username, Password and URL.
> According to Calcite we should be able to provide some properties in the JDBC 
> URL.  Line 142 of org.apache.calcite.avatica.UnregisteredDriver, builds java 
> properties based on the URL.   This works for Kylin.  For example, 
> "jdbc:kylin:ssl=True;//kylinserver.com/Projectname"   Will correctly 
> establish the connection as using SSL.
> The problem is in KylinConnection's Constructor.  It splits the jdbcURL on 
> "jdbc:kylin://".  So, it ignores the SSL. (KylinConnection.  Line 54-60.)
> The KylinConnection needs to be updated to support this method for setting 
> properties, so it can be used by 3rd party applications where programing the 
> connection is not possible.
> We have coded this locally, and will submit patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2668) Kylin JDBC Driver doesn't support Calcites Properties in URL

2017-06-28 Thread Shaofeng SHI (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067610#comment-16067610
 ] 

Shaofeng SHI commented on KYLIN-2668:
-

The commit subject shows good after merged in Git; Pushed to master at: 
https://github.com/apache/kylin/commit/2f084601b938d6051c0fe02a9fc075be549547f0

Thanks Joe!

> Kylin JDBC Driver doesn't support Calcites Properties in URL
> 
>
> Key: KYLIN-2668
> URL: https://issues.apache.org/jira/browse/KYLIN-2668
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC
>Affects Versions: 1.x-HBase1.1.3
>Reporter: Joe Swingle
>Assignee: Joe Swingle
> Attachments: KYLIN-2668-v2.patch
>
>
> We currently access our OLAP Cubes in Kylin from our BI Tool over JDBC.  We 
> run Kylin over SSL.  When using our COTS BI Tool, we cannot programmatically 
> pass in property values such as ssl=True into a java.util.Properties.   We 
> can only specify a Username, Password and URL.
> According to Calcite we should be able to provide some properties in the JDBC 
> URL.  Line 142 of org.apache.calcite.avatica.UnregisteredDriver, builds java 
> properties based on the URL.   This works for Kylin.  For example, 
> "jdbc:kylin:ssl=True;//kylinserver.com/Projectname"   Will correctly 
> establish the connection as using SSL.
> The problem is in KylinConnection's Constructor.  It splits the jdbcURL on 
> "jdbc:kylin://".  So, it ignores the SSL. (KylinConnection.  Line 54-60.)
> The KylinConnection needs to be updated to support this method for setting 
> properties, so it can be used by 3rd party applications where programing the 
> connection is not possible.
> We have coded this locally, and will submit patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (KYLIN-2668) Kylin JDBC Driver doesn't support Calcites Properties in URL

2017-06-28 Thread Shaofeng SHI (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-2668:
---

Assignee: Joe Swingle

> Kylin JDBC Driver doesn't support Calcites Properties in URL
> 
>
> Key: KYLIN-2668
> URL: https://issues.apache.org/jira/browse/KYLIN-2668
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC
>Affects Versions: 1.x-HBase1.1.3
>Reporter: Joe Swingle
>Assignee: Joe Swingle
> Attachments: KYLIN-2668-v2.patch
>
>
> We currently access our OLAP Cubes in Kylin from our BI Tool over JDBC.  We 
> run Kylin over SSL.  When using our COTS BI Tool, we cannot programmatically 
> pass in property values such as ssl=True into a java.util.Properties.   We 
> can only specify a Username, Password and URL.
> According to Calcite we should be able to provide some properties in the JDBC 
> URL.  Line 142 of org.apache.calcite.avatica.UnregisteredDriver, builds java 
> properties based on the URL.   This works for Kylin.  For example, 
> "jdbc:kylin:ssl=True;//kylinserver.com/Projectname"   Will correctly 
> establish the connection as using SSL.
> The problem is in KylinConnection's Constructor.  It splits the jdbcURL on 
> "jdbc:kylin://".  So, it ignores the SSL. (KylinConnection.  Line 54-60.)
> The KylinConnection needs to be updated to support this method for setting 
> properties, so it can be used by 3rd party applications where programing the 
> connection is not possible.
> We have coded this locally, and will submit patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2693) Should use overrideHiveConfig for LookupHiveViewMaterialization and RedistributeFlatHiveTable

2017-06-28 Thread Shaofeng SHI (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067600#comment-16067600
 ] 

Shaofeng SHI commented on KYLIN-2693:
-

+1

> Should use overrideHiveConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable
> -
>
> Key: KYLIN-2693
> URL: https://issues.apache.org/jira/browse/KYLIN-2693
> Project: Kylin
>  Issue Type: Bug
>  Components: Job Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2693.patch
>
>
> Currently,  we use KylinConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable step. We should use cubeOverrideHiveConfig.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2694) Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer

2017-06-28 Thread Shaofeng SHI (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067565#comment-16067565
 ] 

Shaofeng SHI commented on KYLIN-2694:
-

+1; Thanks for pointing it out, my mistake.

> Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer
> 
>
> Key: KYLIN-2694
> URL: https://issues.apache.org/jira/browse/KYLIN-2694
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>Priority: Minor
> Attachments: KYLIN-2694.patch
>
>
> cubeDesc.getBuildLevel() could be zero, so there will throw 
> ArrayIndexOutOfBoundsException in allRDDs[totalLevels - 1].unpersist().



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2653) Spark cubing support HBase cluster with kerberos

2017-06-28 Thread Shaofeng SHI (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067556#comment-16067556
 ] 

Shaofeng SHI commented on KYLIN-2653:
-

Cool, this is the right way; thanks Kaisen!

> Spark cubing support HBase cluster with kerberos
> 
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily,we could support HBase cluster with kerberos on Yarn client mode, 
> because which is easy.
> In the long term,we should avoid access HBase in Spark cubing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2695) Should allow user to override spark conf in cube

2017-06-28 Thread Shaofeng SHI (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067551#comment-16067551
 ] 

Shaofeng SHI commented on KYLIN-2695:
-

+1

> Should allow user to override spark conf in cube
> 
>
> Key: KYLIN-2695
> URL: https://issues.apache.org/jira/browse/KYLIN-2695
> Project: Kylin
>  Issue Type: Improvement
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2695.patch
>
>
> Currently, we could only get spark conf from kylin server config. We should 
> allow user to override spark conf in cube.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2695) Should allow user to override spark conf in cube

2017-06-28 Thread Billy Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067542#comment-16067542
 ] 

Billy Liu commented on KYLIN-2695:
--

+1

> Should allow user to override spark conf in cube
> 
>
> Key: KYLIN-2695
> URL: https://issues.apache.org/jira/browse/KYLIN-2695
> Project: Kylin
>  Issue Type: Improvement
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2695.patch
>
>
> Currently, we could only get spark conf from kylin server config. We should 
> allow user to override spark conf in cube.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (KYLIN-2670) CASE WHEN supporting problem in kylin2.0

2017-06-28 Thread gongliaoan (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066636#comment-16066636
 ] 

gongliaoan edited comment on KYLIN-2670 at 6/28/17 3:02 PM:


I meet the same problem and want to mode the code of kylin try to fix the 
problem


was (Author: gongliaoan):
I meet the same problem and want to mode the code of kylin to fix the problem

> CASE WHEN supporting problem in kylin2.0
> 
>
> Key: KYLIN-2670
> URL: https://issues.apache.org/jira/browse/KYLIN-2670
> Project: Kylin
>  Issue Type: Bug
>  Components: Query Engine
>Affects Versions: v2.0.0
>Reporter: zhou degao
>Assignee: liyang
>
> Following query failed in kylin 2.0 but succeeded in kylin 1.6
> select "fact_pv_data_alias"."PRODUCT_NAME" as "c0", 
> "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as 
> "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", 
> "fact_pv_data_alias"."PLATFORM" order by CASE WHEN 
> "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN 
> "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PLATFORM" ASC
> Reported error in kylin 2.0:
> Error while executing SQL "select "fact_pv_data_alias"."PRODUCT_NAME" as 
> "c0", "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as 
> "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", 
> "fact_pv_data_alias"."PLATFORM" order by CASE WHEN 
> "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN 
> "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PLATFORM" ASC LIMIT 5": index (2) must be less than 
> size (2) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2670) CASE WHEN supporting problem in kylin2.0

2017-06-28 Thread gongliaoan (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066636#comment-16066636
 ] 

gongliaoan commented on KYLIN-2670:
---

I meet the same problem and want to mode the code of kylin to fix the problem

> CASE WHEN supporting problem in kylin2.0
> 
>
> Key: KYLIN-2670
> URL: https://issues.apache.org/jira/browse/KYLIN-2670
> Project: Kylin
>  Issue Type: Bug
>  Components: Query Engine
>Affects Versions: v2.0.0
>Reporter: zhou degao
>Assignee: liyang
>
> Following query failed in kylin 2.0 but succeeded in kylin 1.6
> select "fact_pv_data_alias"."PRODUCT_NAME" as "c0", 
> "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as 
> "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", 
> "fact_pv_data_alias"."PLATFORM" order by CASE WHEN 
> "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN 
> "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PLATFORM" ASC
> Reported error in kylin 2.0:
> Error while executing SQL "select "fact_pv_data_alias"."PRODUCT_NAME" as 
> "c0", "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as 
> "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", 
> "fact_pv_data_alias"."PLATFORM" order by CASE WHEN 
> "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN 
> "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, 
> "fact_pv_data_alias"."PLATFORM" ASC LIMIT 5": index (2) must be less than 
> size (2) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2653) Spark cubing support HBase cluster with kerberos

2017-06-28 Thread kangkaisen (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066427#comment-16066427
 ] 

kangkaisen commented on KYLIN-2653:
---

I have finished the POC. I am testing with large-scale cube data.

> Spark cubing support HBase cluster with kerberos
> 
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily,we could support HBase cluster with kerberos on Yarn client mode, 
> because which is easy.
> In the long term,we should avoid access HBase in Spark cubing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2653) Spark cubing support HBase cluster with kerberos

2017-06-28 Thread kangkaisen (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066419#comment-16066419
 ] 

kangkaisen commented on KYLIN-2653:
---

As for as we all known, Yarn client mode couldn't be used in prod env. So, I 
want to resolve this issue absolutely.

This mainly idea is uploading the necessary metadata to HDFS before spark 
submit and using HDFSResourceStore manage the metadata.

> Spark cubing support HBase cluster with kerberos
> 
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily,we could support HBase cluster with kerberos on Yarn client mode, 
> because which is easy.
> In the long term,we should avoid access HBase in Spark cubing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2653) Spark cubing support HBase cluster with kerberos

2017-06-28 Thread kangkaisen (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066408#comment-16066408
 ] 

kangkaisen commented on KYLIN-2653:
---

Temporarily, we only need to add three line codes before {{new SparkConf() in 
SparkCubingByLayer}}

{code:java}
Configuration configuration = 
HBaseConnection.getCurrentHBaseConfiguration();
HConnection connection = 
HConnectionManager.createConnection(configuration);
TokenUtil.obtainAndCacheToken(connection, 
UserProvider.instantiate(configuration).create(UserGroupInformation.getCurrentUser()));
{code}

which could work well on Yarn client mode.

> Spark cubing support HBase cluster with kerberos
> 
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily,we could support HBase cluster with kerberos on Yarn client mode, 
> because which is easy.
> In the long term,we should avoid access HBase in Spark cubing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2653) Spark cubing support HBase cluster with kerberos

2017-06-28 Thread kangkaisen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kangkaisen updated KYLIN-2653:
--
Summary: Spark cubing support HBase cluster with kerberos  (was: Spark 
cubing support HBase cluster with kerberos on Yarn client mode)

> Spark cubing support HBase cluster with kerberos
> 
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily,we could support HBase cluster with kerberos on Yarn client mode, 
> because which is easy.
> In the long term,we should avoid access HBase in Spark cubing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2695) Should allow user to override spark conf in cube

2017-06-28 Thread kangkaisen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kangkaisen updated KYLIN-2695:
--
Attachment: KYLIN-2695.patch

This is the patch.

> Should allow user to override spark conf in cube
> 
>
> Key: KYLIN-2695
> URL: https://issues.apache.org/jira/browse/KYLIN-2695
> Project: Kylin
>  Issue Type: Improvement
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2695.patch
>
>
> Currently, we could only get spark conf from kylin server config. We should 
> allow user to override spark conf in cube.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (KYLIN-2695) Should allow user to override spark conf in cube

2017-06-28 Thread kangkaisen (JIRA)
kangkaisen created KYLIN-2695:
-

 Summary: Should allow user to override spark conf in cube
 Key: KYLIN-2695
 URL: https://issues.apache.org/jira/browse/KYLIN-2695
 Project: Kylin
  Issue Type: Improvement
  Components: Spark Engine
Affects Versions: v2.0.0
Reporter: kangkaisen
Assignee: kangkaisen


Currently, we could only get spark conf from kylin server config. We should 
allow user to override spark conf in cube.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2693) Should use overrideHiveConfig for LookupHiveViewMaterialization and RedistributeFlatHiveTable

2017-06-28 Thread Billy Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066391#comment-16066391
 ] 

Billy Liu commented on KYLIN-2693:
--

+1

> Should use overrideHiveConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable
> -
>
> Key: KYLIN-2693
> URL: https://issues.apache.org/jira/browse/KYLIN-2693
> Project: Kylin
>  Issue Type: Bug
>  Components: Job Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2693.patch
>
>
> Currently,  we use KylinConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable step. We should use cubeOverrideHiveConfig.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2691) Broken cube cannot be deleted

2017-06-28 Thread jiatao.tao (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao updated KYLIN-2691:
--
Description: Broken cube cannot be deleted.  (was: Broken cube cannot be 
deleted)

> Broken cube cannot be deleted
> -
>
> Key: KYLIN-2691
> URL: https://issues.apache.org/jira/browse/KYLIN-2691
> Project: Kylin
>  Issue Type: Bug
>  Components: Web 
>Reporter: jiatao.tao
>Assignee: luguosheng
>Priority: Minor
>
> Broken cube cannot be deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2694) Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer

2017-06-28 Thread kangkaisen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kangkaisen updated KYLIN-2694:
--
Attachment: KYLIN-2694.patch

This is the patch.

> Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer
> 
>
> Key: KYLIN-2694
> URL: https://issues.apache.org/jira/browse/KYLIN-2694
> Project: Kylin
>  Issue Type: Bug
>  Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>Priority: Minor
> Attachments: KYLIN-2694.patch
>
>
> cubeDesc.getBuildLevel() could be zero, so there will throw 
> ArrayIndexOutOfBoundsException in allRDDs[totalLevels - 1].unpersist().



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (KYLIN-2694) Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer

2017-06-28 Thread kangkaisen (JIRA)
kangkaisen created KYLIN-2694:
-

 Summary: Fix ArrayIndexOutOfBoundsException in SparkCubingByLayer
 Key: KYLIN-2694
 URL: https://issues.apache.org/jira/browse/KYLIN-2694
 Project: Kylin
  Issue Type: Bug
  Components: Spark Engine
Affects Versions: v2.0.0
Reporter: kangkaisen
Assignee: kangkaisen
Priority: Minor


cubeDesc.getBuildLevel() could be zero, so there will throw 
ArrayIndexOutOfBoundsException in allRDDs[totalLevels - 1].unpersist().



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2693) Should use overrideHiveConfig for LookupHiveViewMaterialization and RedistributeFlatHiveTable

2017-06-28 Thread kangkaisen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

kangkaisen updated KYLIN-2693:
--
Attachment: KYLIN-2693.patch

This is the patch.

> Should use overrideHiveConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable
> -
>
> Key: KYLIN-2693
> URL: https://issues.apache.org/jira/browse/KYLIN-2693
> Project: Kylin
>  Issue Type: Bug
>  Components: Job Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2693.patch
>
>
> Currently,  we use KylinConfig for LookupHiveViewMaterialization and 
> RedistributeFlatHiveTable step. We should use cubeOverrideHiveConfig.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (KYLIN-2693) Should use overrideHiveConfig for LookupHiveViewMaterialization and RedistributeFlatHiveTable

2017-06-28 Thread kangkaisen (JIRA)
kangkaisen created KYLIN-2693:
-

 Summary: Should use overrideHiveConfig for 
LookupHiveViewMaterialization and RedistributeFlatHiveTable
 Key: KYLIN-2693
 URL: https://issues.apache.org/jira/browse/KYLIN-2693
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v2.0.0
Reporter: kangkaisen
Assignee: kangkaisen


Currently,  we use KylinConfig for LookupHiveViewMaterialization and 
RedistributeFlatHiveTable step. We should use cubeOverrideHiveConfig.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (KYLIN-2638) Cannot install Kylin on CDH 5.11

2017-06-28 Thread Billy Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/KYLIN-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066280#comment-16066280
 ] 

Billy Liu commented on KYLIN-2638:
--

This happens when lower JDK version is configured. Kylin need JDK 7+.

> Cannot install Kylin on CDH 5.11
> 
>
> Key: KYLIN-2638
> URL: https://issues.apache.org/jira/browse/KYLIN-2638
> Project: Kylin
>  Issue Type: Bug
>  Components: Environment 
>Affects Versions: v2.0.0
>Reporter: Gergely
>Assignee: hongbin ma
>Priority: Blocker
>  Labels: newbie, scope
>
> Hi All,
> We have a blocking issue by installing Kylin on CDH 5.11. By executing such 
> lines on CentOS 7 we always getting empty strings.
> Could you please advise?
> bash $KYLIN_HOME/bin/get-properties.sh kylin.env.hdfs-working-dir
> KYLIN_HOME points to the right location, it also gives empty when we call 
> directly bash get-properties.sh kylin.env.hdfs-working-dir
> Many thanks in advance.
> Regards



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (KYLIN-2370) Refine unload table

2017-06-28 Thread Zhixiong Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen closed KYLIN-2370.

Resolution: Fixed

> Refine unload table
> ---
>
> Key: KYLIN-2370
> URL: https://issues.apache.org/jira/browse/KYLIN-2370
> Project: Kylin
>  Issue Type: Improvement
>  Components: Web 
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
> Attachments: 0001-KYLIN-2370-Refine-unload-and-reload-table.patch
>
>
> Move the Unload Button to the Table level .
> Add a reload button side by side.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (KYLIN-2691) Broken cube cannot be deleted

2017-06-28 Thread Zhixiong Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen reassigned KYLIN-2691:


Assignee: luguosheng  (was: Zhong,Jason)

> Broken cube cannot be deleted
> -
>
> Key: KYLIN-2691
> URL: https://issues.apache.org/jira/browse/KYLIN-2691
> Project: Kylin
>  Issue Type: Bug
>  Components: Web 
>Reporter: jiatao.tao
>Assignee: luguosheng
>Priority: Minor
>
> Broken cube cannot be deleted



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Closed] (KYLIN-2689) Only dimension columns can join when create a model

2017-06-28 Thread Zhixiong Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen closed KYLIN-2689.

Resolution: Fixed

This issue has fixed.

> Only dimension columns can join when create a model
> ---
>
> Key: KYLIN-2689
> URL: https://issues.apache.org/jira/browse/KYLIN-2689
> Project: Kylin
>  Issue Type: Improvement
>  Components: Web 
>Affects Versions: v2.1.0
>Reporter: luguosheng
>Assignee: luguosheng
> Fix For: v2.1.0
>
> Attachments: 
> 0001-KYLIN-2689-Only-dimension-columns-can-join-when-crea.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2689) Only dimension columns can join when create a model

2017-06-28 Thread Zhixiong Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen updated KYLIN-2689:
-
Attachment: 0001-KYLIN-2689-Only-dimension-columns-can-join-when-crea.patch

> Only dimension columns can join when create a model
> ---
>
> Key: KYLIN-2689
> URL: https://issues.apache.org/jira/browse/KYLIN-2689
> Project: Kylin
>  Issue Type: Improvement
>  Components: Web 
>Affects Versions: v2.1.0
>Reporter: luguosheng
>Assignee: luguosheng
> Fix For: v2.1.0
>
> Attachments: 
> 0001-KYLIN-2689-Only-dimension-columns-can-join-when-crea.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (KYLIN-2667) Ignore whitespace when caching query

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang resolved KYLIN-2667.
---
Resolution: Duplicate

> Ignore whitespace when caching query
> 
>
> Key: KYLIN-2667
> URL: https://issues.apache.org/jira/browse/KYLIN-2667
> Project: Kylin
>  Issue Type: Improvement
>Reporter: hongbin ma
>Assignee: hongbin ma
> Fix For: v2.1.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Deleted] (KYLIN-2692) Yang's Radar

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang deleted KYLIN-2692:
--


> Yang's Radar
> 
>
> Key: KYLIN-2692
> URL: https://issues.apache.org/jira/browse/KYLIN-2692
> Project: Kylin
>  Issue Type: Task
>Reporter: liyang
>
> Tracks some JIRAs of my interest.
> Job
> * {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there 
> is getSchedulerType(){color}
> Dictionary
> * KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
> * KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict
> * KYLIN-2662 A cell beyond 32KB?
> HBase
> * {color:#d04437}KYLIN-224 Support HBase namespace{color}
> * {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
> hbase 1.2.5{color}
>   
> Query
> * KYLIN-2666 Let queries with only spacing difference hit cache
> Source
> * {color:#14892c}KYLIN-1351 JDBC source needs UT{color}
> Install
> * {color:#14892c}KYLIN-2638 Cannot install Kylin on CDH 5.11{color}
> Misc
> * {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
> QueryServiceV2#execute(){color}
> * {color:#14892c}KYLIN-2612 Potential NPE accessing familyMap in 
> AclTableMigrationTool#getAllAceInfo{color}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2612) Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2612:
--
Labels: newbie scope  (was: )

> Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
> 
>
> Key: KYLIN-2612
> URL: https://issues.apache.org/jira/browse/KYLIN-2612
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>  Labels: newbie, scope
>
> Here is related code:
> {code}
> NavigableMap familyMap = 
> result.getFamilyMap(Bytes.toBytes(AclHBaseStorage.ACL_ACES_FAMILY));
> for (Map.Entry entry : familyMap.entrySet()) {
> {code}
> Good practice is to check whether familyMap is null / empty first.
> See the following class in hbase for example:
> hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2643) PreparedStatement should be closed in QueryServiceV2#execute()

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2643:
--
Labels: newbie scope  (was: )

> PreparedStatement should be closed in QueryServiceV2#execute()
> --
>
> Key: KYLIN-2643
> URL: https://issues.apache.org/jira/browse/KYLIN-2643
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yifan Zhang
>Priority: Minor
>  Labels: newbie, scope
>
> {code}
> PreparedStatement preparedState = 
> conn.prepareStatement(correctedSql);
> processStatementAttr(preparedState, sqlRequest);
> for (int i = 0; i < ((PrepareSqlRequest) 
> sqlRequest).getParams().length; i++) {
> setParam(preparedState, i + 1, ((PrepareSqlRequest) 
> sqlRequest).getParams()[i]);
> }
> resultSet = preparedState.executeQuery();
> {code}
> preparedState should be closed upon returning from the method.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-1351) Support common RDBMS as data source in Kylin (Pending UT)

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-1351:
--
Labels: newbie scope  (was: newbie)

> Support common RDBMS as data source in Kylin (Pending UT)
> -
>
> Key: KYLIN-1351
> URL: https://issues.apache.org/jira/browse/KYLIN-1351
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Shaofeng SHI
>Assignee: Cheng Yi
>  Labels: newbie, scope
>
> From v2.0, Kylin's plug-in architecture makes it possible to have multiple 
> data sources, cube engines and storages. Some users ever aksed that whether 
> Kylin support source data feeded from RDBMS like Oracle, MySQL, now it is 
> possible to do that. Some tools like Apache Sqoop can easily export data from 
> RDBMS to HDFS, that would help Kylin get the data and then build that into 
> cubes. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-1351) Support common RDBMS as data source in Kylin (Pending UT)

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-1351:
--
Summary: Support common RDBMS as data source in Kylin (Pending UT)  (was: 
Support common RDBMS as data source in Kylin)

> Support common RDBMS as data source in Kylin (Pending UT)
> -
>
> Key: KYLIN-1351
> URL: https://issues.apache.org/jira/browse/KYLIN-1351
> Project: Kylin
>  Issue Type: New Feature
>Reporter: Shaofeng SHI
>Assignee: Cheng Yi
>  Labels: newbie, scope
>
> From v2.0, Kylin's plug-in architecture makes it possible to have multiple 
> data sources, cube engines and storages. Some users ever aksed that whether 
> Kylin support source data feeded from RDBMS like Oracle, MySQL, now it is 
> possible to do that. Some tools like Apache Sqoop can easily export data from 
> RDBMS to HDFS, that would help Kylin get the data and then build that into 
> cubes. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2638) Cannot install Kylin on CDH 5.11

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2638:
--
Labels: newbie scope  (was: newbie)

> Cannot install Kylin on CDH 5.11
> 
>
> Key: KYLIN-2638
> URL: https://issues.apache.org/jira/browse/KYLIN-2638
> Project: Kylin
>  Issue Type: Bug
>  Components: Environment 
>Affects Versions: v2.0.0
>Reporter: Gergely
>Assignee: hongbin ma
>Priority: Blocker
>  Labels: newbie, scope
>
> Hi All,
> We have a blocking issue by installing Kylin on CDH 5.11. By executing such 
> lines on CentOS 7 we always getting empty strings.
> Could you please advise?
> bash $KYLIN_HOME/bin/get-properties.sh kylin.env.hdfs-working-dir
> KYLIN_HOME points to the right location, it also gives empty when we call 
> directly bash get-properties.sh kylin.env.hdfs-working-dir
> Many thanks in advance.
> Regards



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2666) Let queries with only spacing difference hit cache

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2666:
--
Labels: newbie scope  (was: newbie)

> Let queries with only spacing difference hit cache
> --
>
> Key: KYLIN-2666
> URL: https://issues.apache.org/jira/browse/KYLIN-2666
> Project: Kylin
>  Issue Type: Improvement
>Reporter: liyang
>  Labels: newbie, scope
>
> There are cases where the same query but having spacing difference get 
> re-submitted. However currently, cache check is string exactly equal, and 
> that will fail on queries with only spacing difference.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2618) Still dictionary cannot go over 2GB given ForestDict

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2618:
--
Labels: scope  (was: )

> Still dictionary cannot go over 2GB given ForestDict
> 
>
> Key: KYLIN-2618
> URL: https://issues.apache.org/jira/browse/KYLIN-2618
> Project: Kylin
>  Issue Type: Improvement
>Reporter: liyang
>  Labels: scope
>
> This is because dictionary persists through ResourceStore API and that goes 
> through a copying process based on byte array. The byte array is limited to 
> 2GB at most.
> The solution ideally should be storing dictionary/snapshot directly to HDFS.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2522) Compilation fails with Java 8 when upgrading to hbase 1.2.5

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2522:
--
Labels: newbie scope  (was: )

> Compilation fails with Java 8 when upgrading to hbase 1.2.5
> ---
>
> Key: KYLIN-2522
> URL: https://issues.apache.org/jira/browse/KYLIN-2522
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>  Labels: newbie, scope
>
> Compiling master branch with Java 8, I got:
> {code}
> [ERROR] 
> /Users/tyu/kylin/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/ZookeeperUtil.java:[23,23]
>  error: cannot find symbol
> [ERROR]   symbol:   class Nullable
>   location: package javax.annotation
> /Users/tyu/kylin/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/ZookeeperUtil.java:[45,13]
>  error: cannot find symbol
> [ERROR]   symbol: class Nullable
> /Users/tyu/kylin/storage-hbase/src/main/java/org/apache/kylin/storage/hbase/util/ZookeeperJobLock.java:[112,13]
>  error: cannot find symbol
> [INFO] 3 errors
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2567) DictionaryDimEnc.encode() swallows dictionary exception

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2567:
--
Labels: scope  (was: )

> DictionaryDimEnc.encode() swallows dictionary exception
> ---
>
> Key: KYLIN-2567
> URL: https://issues.apache.org/jira/browse/KYLIN-2567
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.0.0
>Reporter: liyang
>Assignee: liyang
>  Labels: scope
>
> This behavior has been there for a long time, but definitely is bad. Need to 
> move the exception handling to more appropriate place, rather than so deep 
> inside.
> Also this makes inconsistent behavior between DictionaryDimEnc.encode() and 
> DictionarySerializer.serialize().
> Finally same problem for decode().



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2662) NegativeArraySizeException in "Extract Fact Table Distinct Columns"

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2662:
--
Labels: scope  (was: )

> NegativeArraySizeException in "Extract Fact Table Distinct Columns"
> ---
>
> Key: KYLIN-2662
> URL: https://issues.apache.org/jira/browse/KYLIN-2662
> Project: Kylin
>  Issue Type: Bug
>Reporter: liyang
>  Labels: scope
>
> The full exception. The root cause suspect to be a cell goes over 32 KB. Need 
> verification and fix.
> {code}
> java.lang.NegativeArraySizeException
> at 
> org.apache.kylin.dict.TrieDictionary.getValueBytesFromIdWithoutCache(SourceFile:239)
> at 
> org.apache.kylin.dict.TrieDictionaryForestBuilder.addTree(SourceFile:134)
> at 
> org.apache.kylin.dict.TrieDictionaryForestBuilder.build(SourceFile:110)
> at 
> org.apache.kylin.dict.DictionaryGenerator$StringTrieDictForestBuilder.build(SourceFile:218)
> at 
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doCleanup(SourceFile:231)
> at 
> org.apache.kylin.engine.mr.KylinReducer.cleanup(SourceFile:71)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179)
> at 
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
> at 
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at 
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2626) Remove config getJobControllerLock() since there is getSchedulerType()

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2626:
--
Labels: newbie scope  (was: )

> Remove config getJobControllerLock() since there is getSchedulerType()
> --
>
> Key: KYLIN-2626
> URL: https://issues.apache.org/jira/browse/KYLIN-2626
> Project: Kylin
>  Issue Type: Bug
>  Components: Job Engine
>Affects Versions: v2.1.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>Priority: Minor
>  Labels: newbie, scope
> Fix For: v2.1.0
>
> Attachments: KYLIN-2626.patch
>
>
> KYLIN-2578 introduced this issue:
> {code:java}
>   Caused by: java.lang.RuntimeException: java.lang.InstantiationException: 
> org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock
> at org.apache.kylin.common.util.ClassUtil.newInstance(ClassUtil.java:95)
> at 
> org.apache.kylin.rest.service.JobService.afterPropertiesSet(JobService.java:110)
> at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
> at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
> ... 38 more
> Caused by: java.lang.InstantiationException: 
> org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock
> at java.lang.Class.newInstance(Class.java:427)
> at org.apache.kylin.common.util.ClassUtil.newInstance(ClassUtil.java:93)
> ... 41 more
> Caused by: java.lang.NoSuchMethodException: 
> org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.()
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.newInstance(Class.java:412)
> ... 42 more
> {code}
> which make the Kylin  job server cannot start.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-224) Leverage HBase Namespace to isolate Kylin HTables

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-224:
-
Labels: focus newbie scope  (was: github-import newbie)

> Leverage HBase Namespace to isolate Kylin HTables 
> --
>
> Key: KYLIN-224
> URL: https://issues.apache.org/jira/browse/KYLIN-224
> Project: Kylin
>  Issue Type: New Feature
>  Components: Storage - HBase
>Reporter: Luke Han
>Assignee: nichunen
>  Labels: focus, newbie, scope
> Fix For: Backlog
>
>
> To well manage and isolate Kylin relative htable in HBase, using namespace is 
> best way to isolate with other application's tables. 
> Due to [HBASE-9206|https://issues.apache.org/jira/browse/HBASE-9206], Kylin 
> do not support HBase namespace yet.
>  Imported from GitHub 
> Url: https://github.com/KylinOLAP/Kylin/issues/281
> Created by: [lukehan|https://github.com/lukehan]
> Labels: enhancement, 
> Milestone: Backlog
> Created at: Fri Dec 26 14:05:42 CST 2014
> State: open



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2692) Yang's Radar

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2692:
--
Description: 
Tracks some JIRAs of my interest.

Job
* {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there is 
getSchedulerType(){color}

Dictionary
* KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
* KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict
* KYLIN-2662 A cell beyond 32KB?

HBase
* {color:#d04437}KYLIN-224 Support HBase namespace{color}
* {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
hbase 1.2.5{color}

Query
* KYLIN-2666 Let queries with only spacing difference hit cache

Source
* {color:#14892c}KYLIN-1351 JDBC source needs UT{color}

Install
* {color:#14892c}KYLIN-2638 Cannot install Kylin on CDH 5.11{color}

Misc
* {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
QueryServiceV2#execute(){color}
* {color:#14892c}KYLIN-2612 Potential NPE accessing familyMap in 
AclTableMigrationTool#getAllAceInfo{color}



  was:
Tracks some JIRAs of my interest.

Job
* {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there is 
getSchedulerType(){color}

Dictionary
* KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
* KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict

HBase
* {color:#d04437}KYLIN-224 Support HBase namespace{color}
* {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
hbase 1.2.5{color}

Query
* KYLIN-2666 Let queries with only spacing difference hit cache
* {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
QueryServiceV2#execute(){color}




> Yang's Radar
> 
>
> Key: KYLIN-2692
> URL: https://issues.apache.org/jira/browse/KYLIN-2692
> Project: Kylin
>  Issue Type: Task
>Reporter: liyang
>
> Tracks some JIRAs of my interest.
> Job
> * {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there 
> is getSchedulerType(){color}
> Dictionary
> * KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
> * KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict
> * KYLIN-2662 A cell beyond 32KB?
> HBase
> * {color:#d04437}KYLIN-224 Support HBase namespace{color}
> * {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
> hbase 1.2.5{color}
>   
> Query
> * KYLIN-2666 Let queries with only spacing difference hit cache
> Source
> * {color:#14892c}KYLIN-1351 JDBC source needs UT{color}
> Install
> * {color:#14892c}KYLIN-2638 Cannot install Kylin on CDH 5.11{color}
> Misc
> * {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
> QueryServiceV2#execute(){color}
> * {color:#14892c}KYLIN-2612 Potential NPE accessing familyMap in 
> AclTableMigrationTool#getAllAceInfo{color}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2638) Cannot install Kylin on CDH 5.11

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2638:
--
Summary: Cannot install Kylin on CDH 5.11  (was: Cannot install Kylin)

> Cannot install Kylin on CDH 5.11
> 
>
> Key: KYLIN-2638
> URL: https://issues.apache.org/jira/browse/KYLIN-2638
> Project: Kylin
>  Issue Type: Bug
>  Components: Environment 
>Affects Versions: v2.0.0
>Reporter: Gergely
>Assignee: hongbin ma
>Priority: Blocker
>  Labels: newbie
>
> Hi All,
> We have a blocking issue by installing Kylin on CDH 5.11. By executing such 
> lines on CentOS 7 we always getting empty strings.
> Could you please advise?
> bash $KYLIN_HOME/bin/get-properties.sh kylin.env.hdfs-working-dir
> KYLIN_HOME points to the right location, it also gives empty when we call 
> directly bash get-properties.sh kylin.env.hdfs-working-dir
> Many thanks in advance.
> Regards



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (KYLIN-2692) Yang's Radar

2017-06-28 Thread liyang (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyang updated KYLIN-2692:
--
Description: 
Tracks some JIRAs of my interest.

Job
* {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there is 
getSchedulerType(){color}

Dictionary
* KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
* KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict

HBase
* {color:#d04437}KYLIN-224 Support HBase namespace{color}
* {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
hbase 1.2.5{color}

Query
* KYLIN-2666 Let queries with only spacing difference hit cache
* {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
QueryServiceV2#execute(){color}



  was:Tracks some JIRAs of my interest.


> Yang's Radar
> 
>
> Key: KYLIN-2692
> URL: https://issues.apache.org/jira/browse/KYLIN-2692
> Project: Kylin
>  Issue Type: Task
>Reporter: liyang
>
> Tracks some JIRAs of my interest.
> Job
> * {color:#14892c}KYLIN-2626 Remove config getJobControllerLock() since there 
> is getSchedulerType(){color}
> Dictionary
> * KYLIN-2567 DictionaryDimEnc.encode() swallows dictionary exception
> * KYLIN-2618 Still dictionary cannot go over 2GB given ForestDict
> HBase
> * {color:#d04437}KYLIN-224 Support HBase namespace{color}
> * {color:#14892c}KYLIN-2522 Compilation fails with Java 8 when upgrading to 
> hbase 1.2.5{color}
>   
> Query
> * KYLIN-2666 Let queries with only spacing difference hit cache
> * {color:#14892c}KYLIN-2643 PreparedStatement should be closed in 
> QueryServiceV2#execute(){color}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (KYLIN-2692) Yang's Radar

2017-06-28 Thread liyang (JIRA)
liyang created KYLIN-2692:
-

 Summary: Yang's Radar
 Key: KYLIN-2692
 URL: https://issues.apache.org/jira/browse/KYLIN-2692
 Project: Kylin
  Issue Type: Task
Reporter: liyang


Tracks some JIRAs of my interest.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (KYLIN-2691) Broken cube cannot be deleted

2017-06-28 Thread jiatao.tao (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao resolved KYLIN-2691.
---
Resolution: Fixed

> Broken cube cannot be deleted
> -
>
> Key: KYLIN-2691
> URL: https://issues.apache.org/jira/browse/KYLIN-2691
> Project: Kylin
>  Issue Type: Bug
>  Components: Web 
>Reporter: jiatao.tao
>Assignee: Zhong,Jason
>Priority: Minor
>
> Broken cube cannot be deleted



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (KYLIN-2681) Convert input sql's expression to computed column if computed colum defined

2017-06-28 Thread jiatao.tao (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao resolved KYLIN-2681.
---
Resolution: Fixed

> Convert input sql's expression to computed column if computed colum defined
> ---
>
> Key: KYLIN-2681
> URL: https://issues.apache.org/jira/browse/KYLIN-2681
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: v2.0.0
>Reporter: jiatao.tao
>Assignee: liyang
>Priority: Minor
> Fix For: Future
>
>
> convert user's input sql's expression to computed column if computed column 
> already defined.For example :
> *define computed column:*  
>   cc1:a*b+c
> *user's input sql*
>   select t1.a*t1.b+t1.c from table1 t1
> and then will be tranformed to:
>   select t1.cc1 from table1 t1



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (KYLIN-2686) The same project's computed column's definition can not be same.

2017-06-28 Thread jiatao.tao (JIRA)

 [ 
https://issues.apache.org/jira/browse/KYLIN-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jiatao.tao resolved KYLIN-2686.
---
Resolution: Fixed

> The same project's computed column's definition can not be same.
> 
>
> Key: KYLIN-2686
> URL: https://issues.apache.org/jira/browse/KYLIN-2686
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: Future
>Reporter: jiatao.tao
>Assignee: liyang
> Fix For: Future
>
>
> The computed columns defined in the same project's definition can not be the 
> same.Like in project "learn_kyiln",you may define computed columns like "cc1: 
> a + b","cc2:a + b" in one cube or in multiple cubes.This shall be forbidden 
> cuz cc1 ande cc2 's definition is same.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (KYLIN-2691) Broken cube cannot be deleted

2017-06-28 Thread jiatao.tao (JIRA)
jiatao.tao created KYLIN-2691:
-

 Summary: Broken cube cannot be deleted
 Key: KYLIN-2691
 URL: https://issues.apache.org/jira/browse/KYLIN-2691
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Reporter: jiatao.tao
Assignee: Zhong,Jason
Priority: Minor


Broken cube cannot be deleted



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)