Re: [DISCUSS] On removing Cube ACL

2017-08-19 Thread Li Yang 
(correct)  So it does not *simplify *anything.

A clear removal is still preferred, in my opinion.

On Sun, Aug 20, 2017 at 1:16 PM, Li Yang  wrote:

> I think we are looking for a clear attitude here. Leaving an extension
> point is the same to say "cube level ACL shall be kept in design". Because
> we still have to think about how the extension point works with
> project/model ACL etc. So it does not simply anything.
>
> On Tue, Aug 15, 2017 at 11:59 PM, Luke Han  wrote:
>
>> Data Model level is better than Cube level.
>>
>> How about to leave extend point there for people who want to keep Cube
>> Level ACL?
>>
>>
>>
>>
>> Best Regards!
>> -
>>
>> Luke Han
>>
>> On Tue, Aug 15, 2017 at 5:30 PM, Li Yang  wrote:
>>
>> > So we can still control who can (or cannot) access a cube by defining
>> ACL
>> > on tables, right?
>> >
>> > On Mon, Aug 7, 2017 at 2:26 PM, ShaoFeng Shi 
>> > wrote:
>> >
>> > > It makes sense for me. Will the Cube ACL info be migrated (from Cube
>> to
>> > > Table) when it is done?
>> > >
>> > > 2017-08-07 10:37 GMT+08:00 hongbin ma :
>> > >
>> > > > Current ACL design is based on very early versions where kylin
>> merely
>> > had
>> > > > simple concepts like projects and cubes.
>> > > > With the advent of Kylin v2.0, and several new concepts like Data
>> > Model (
>> > > > http://kylin.apache.org/docs/gettingstarted/concepts.html) and
>> Query
>> > > > Pushdown (KYLIN-2515), the original cube-centric ACL design is
>> becoming
>> > > > outdated. The reasons are two-fold: 1. cubes are no longer the only
>> > > > entities we want to take control within each projects.  2.
>> Cube-level
>> > ACL
>> > > > cannot protect underlying tables from being queries by unwanted
>> users.
>> > > >
>> > > > In fact, cubes is merely a special kind of index on the original
>> table.
>> > > > It's not straightforward to apply ACL on indexes rather than
>> original
>> > > > tables. That said, we need table-level ACL instead of cube-level
>> ACL.
>> > We
>> > > > have elaborated the detailed plans in KYLIN-2760 and KYLIN-2761.
>> Please
>> > > > comment on those issues or reply this email if you have any
>> concerns.
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > *Bin Mahone | 马洪宾*
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > > Shaofeng Shi 史少锋
>> > >
>> >
>>
>
>

Re: [DISCUSS] On removing Cube ACL

2017-08-19 Thread Li Yang 
I think we are looking for a clear attitude here. Leaving an extension
point is the same to say "cube level ACL shall be kept in design". Because
we still have to think about how the extension point works with
project/model ACL etc. So it does not simply anything.

On Tue, Aug 15, 2017 at 11:59 PM, Luke Han  wrote:

> Data Model level is better than Cube level.
>
> How about to leave extend point there for people who want to keep Cube
> Level ACL?
>
>
>
>
> Best Regards!
> -
>
> Luke Han
>
> On Tue, Aug 15, 2017 at 5:30 PM, Li Yang  wrote:
>
> > So we can still control who can (or cannot) access a cube by defining ACL
> > on tables, right?
> >
> > On Mon, Aug 7, 2017 at 2:26 PM, ShaoFeng Shi 
> > wrote:
> >
> > > It makes sense for me. Will the Cube ACL info be migrated (from Cube to
> > > Table) when it is done?
> > >
> > > 2017-08-07 10:37 GMT+08:00 hongbin ma :
> > >
> > > > Current ACL design is based on very early versions where kylin merely
> > had
> > > > simple concepts like projects and cubes.
> > > > With the advent of Kylin v2.0, and several new concepts like Data
> > Model (
> > > > http://kylin.apache.org/docs/gettingstarted/concepts.html) and Query
> > > > Pushdown (KYLIN-2515), the original cube-centric ACL design is
> becoming
> > > > outdated. The reasons are two-fold: 1. cubes are no longer the only
> > > > entities we want to take control within each projects.  2. Cube-level
> > ACL
> > > > cannot protect underlying tables from being queries by unwanted
> users.
> > > >
> > > > In fact, cubes is merely a special kind of index on the original
> table.
> > > > It's not straightforward to apply ACL on indexes rather than original
> > > > tables. That said, we need table-level ACL instead of cube-level ACL.
> > We
> > > > have elaborated the detailed plans in KYLIN-2760 and KYLIN-2761.
> Please
> > > > comment on those issues or reply this email if you have any concerns.
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > *Bin Mahone | 马洪宾*
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > Shaofeng Shi 史少锋
> > >
> >
>

Re: Get data from bigint column error

2017-08-19 Thread Li Yang 
What is your Kylin version? I think a similar issue was reported so it is
likely fixed in latest 2.1 release. Mind test it again on 2.1?

On Tue, Aug 15, 2017 at 3:42 PM, jimmy  wrote:

> Hi,
>
>
>
> I want to get data from kylin sample data.
>
> But,It return error message.
>
>
>
> SQL:
>
>
>
> select KYLIN_ACCOUNT.ACCOUNT_BUYER_LEVEL from KYLIN_ACCOUNT
>
>
>
> Error Message:
>
>
>
> Error while executing SQL "select KYLIN_ACCOUNT.ACCOUNT_BUYER_LEVEL from
> KYLIN_ACCOUNT LIMIT 5": java.lang.Integer cannot be cast to
> java.lang.Long
>
>
>
> Has any suggestion?
>
>
>
> Thanks a lot.
>
>
>
> Jimmy.
>

[jira] [Created] (KYLIN-2795) Improve REST API document, add get/list jobs

2017-08-19 Thread liyang (JIRA) 
liyang created KYLIN-2795:
-

 Summary: Improve REST API document, add get/list jobs
 Key: KYLIN-2795
 URL: https://issues.apache.org/jira/browse/KYLIN-2795
 Project: Kylin
  Issue Type: Improvement
Reporter: liyang


All job REST API requires jobid, but currently there is no API documented to 
get/list existing Jobs.

http://kylin.apache.org/docs20/howto/howto_use_restapi.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: how to get jobid

2017-08-19 Thread Li Yang 
Good point. JIRA logged.

https://issues.apache.org/jira/browse/KYLIN-2795

On Mon, Aug 14, 2017 at 7:27 PM, 为岁月留白 <215521...@qq.com> wrote:

> All of public job RESRAAPI new use jobid,but didn't have API to get
> jobid,so,how to get the jobid.

Re: [BadQueryDetector] service.BadQueryDetector:169 : Problemat ic thread 0x151

2017-08-19 Thread Li Yang 
The service outage is not expected, it could be slow but not outage.

> where hotel <> 'A' and hotel = 'A'

However this deserves a JIRA to optimize for the "always false" condition.
Please log a JIRA.


On Mon, Aug 14, 2017 at 4:58 PM, wangzy24  wrote:

> When conditions are mutually exclusive in SQL statements, the service goes
> down, such as : /select * from test where hotel <> 'A' and hotel = 'A'/,
> The
> expectation of this situation is that you can return no data, What is the
> reason for the outage?
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/BadQueryDetector-service-BadQueryDetector-169-
> Problemat-ic-thread-0x151-tp8720.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: Kylin often hangs up

2017-08-19 Thread Li Yang 
> 2017-08-10 10:14:35,013 WARN  [localhost-startStop-1-EventThread]
client.HConnectionManager$HConnectionImplementation:2468 : This client just
lost it's session with ZooKeeper, closing it. It will be recreated next
time someone needs it
org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired

Obviously the ZK connection is not stable.

On Thu, Aug 10, 2017 at 3:39 PM, shendandan 
wrote:

>
>
>
> Thanks for your respond,but Zk is working normally.
>
>
>
>
>
> At 2017-08-10 15:21:49, "Billy Liu"  wrote:
> >The first WARN happened during ZK timeout. Please check the ZK connection
> >first.
> >
> >2017-08-10 10:58 GMT+08:00 shendandan :
> >
> >> Hi!
> >>The process of kylin often hangs up,and the erroe in kylin,log:
> >>
> >>
> >> 2017-08-10 10:14:22,292 INFO  [BadQueryDetector]
> >> service.BadQueryDetector:151 : System free memory less than 100 MB. 1
> >> queries running.
> >> 2017-08-10 10:14:34,498 INFO  [Thread-11-EventThread]
> >> state.ConnectionStateManager:228 : State change: SUSPENDED
> >> 2017-08-10 10:14:34,506 INFO  [pool-7-thread-1]
> >> threadpool.DefaultScheduler:118 : Job Fetcher: 0 should running, 0
> actual
> >> running, 0 ready, 63 already succeed, 3 error, 13 discarded, 0 others
> >> 2017-08-10 10:14:34,999 INFO  [localhost-startStop-1-SendThread(
> >> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening
> socket
> >> connection to server hadoop-senior01.ctcf.com/10.1.8.90:2181. Will not
> >> attempt to authenticate using SASL (unknown error)
> >> 2017-08-10 10:14:34,999 INFO  [Thread-11-SendThread(hadoop-
> >> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening socket
> >> connection to server hadoop-senior03.ctcf.com/10.1.8.92:2181. Will not
> >> attempt to authenticate using SASL (unknown error)
> >> 2017-08-10 10:14:34,999 INFO  [localhost-startStop-1-SendThread(
> >> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket
> >> connection established, initiating session, client: /10.1.8.90:38229,
> >> server: hadoop-senior01.ctcf.com/10.1.8.90:2181
> >> 2017-08-10 10:14:34,999 INFO  [Thread-11-SendThread(hadoop-
> >> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket connection
> >> established, initiating session, client: /10.1.8.90:32962, server:
> >> hadoop-senior03.ctcf.com/10.1.8.92:2181
> >> 2017-08-10 10:14:35,001 INFO  [Thread-11-SendThread(hadoop-
> >> senior03.ctcf.com:2181)] zookeeper.ClientCnxn:1094 : Unable to
> reconnect
> >> to ZooKeeper service, session 0x25dc4bd90cc002a has expired, closing
> socket
> >> connection
> >> 2017-08-10 10:14:35,002 INFO  [localhost-startStop-1-SendThread(
> >> hadoop-senior01.ctcf.com:2181)] zookeeper.ClientCnxn:1094 : Unable to
> >> reconnect to ZooKeeper service, session 0x35dc4bd90460028 has expired,
> >> closing socket connection
> >> 2017-08-10 10:14:35,013 WARN  [localhost-startStop-1-EventThread]
> >> client.HConnectionManager$HConnectionImplementation:2468 : This client
> >> just lost it's session with ZooKeeper, closing it. It will be recreated
> >> next time someone needs it
> >> org.apache.zookeeper.KeeperException$SessionExpiredException:
> >> KeeperErrorCode = Session expired
> >> at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(
> >> ZooKeeperWatcher.java:517)
> >> at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.
> >> process(ZooKeeperWatcher.java:435)
> >> at org.apache.zookeeper.ClientCnxn$EventThread.
> >> processEvent(ClientCnxn.java:522)
> >> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> >> 2017-08-10 10:14:35,013 INFO  [Thread-11-EventThread]
> >> state.ConnectionStateManager:228 : State change: LOST
> >> 2017-08-10 10:14:35,014 INFO  [localhost-startStop-1-EventThread]
> >> client.HConnectionManager$HConnectionImplementation:1837 : Closing
> >> zookeeper sessionid=0x35dc4bd90460028
> >> 2017-08-10 10:14:35,014 WARN  [Thread-11-EventThread]
> >> curator.ConnectionState:289 : Session expired event received
> >> 2017-08-10 10:14:35,014 INFO  [localhost-startStop-1-EventThread]
> >> zookeeper.ClientCnxn:512 : EventThread shut down
> >> 2017-08-10 10:14:35,015 INFO  [Thread-11-EventThread]
> >> zookeeper.ZooKeeper:438 : Initiating client connection, connectString=
> >> hadoop-senior01.ctcf.com:2181,hadoop-senior02.ctcf.com:2181,
> >> hadoop-senior03.ctcf.com:2181 sessionTimeout=6
> >> watcher=org.apache.curator.ConnectionState@4d6f4fa7
> >> 2017-08-10 10:14:35,017 INFO  [Thread-11-SendThread(hadoop-
> >> senior02.ctcf.com:2181)] zookeeper.ClientCnxn:975 : Opening socket
> >> connection to server hadoop-senior02.ctcf.com/10.1.8.91:2181. Will not
> >> attempt to authenticate using SASL (unknown error)
> >> 2017-08-10 10:14:35,017 INFO  [Thread-11-SendThread(hadoop-
> >> senior02.ctcf.com:2181)] zookeeper.ClientCnxn:852 : Socket connection
> >> established, initiating session, client: /10.1.8.90:57503, server:

LAST_DAY function?

2017-08-19 Thread apache_...@163.com 
Hi,

   There are a lot of surprises in kylin 2.1 version.It's amazing.

   
   Ask a question:

  What functions can be used to calculate the date of the end of the month  in 
kylin?

  Like oracle  LAST_DAY function



apache_...@163.com

Re: [Announce] Apache Kylin 2.1.0 released

2017-08-19 Thread Han Luke 
Thanks Shaofeng to be the release manager.

Luke

From: ShaoFeng Shi 
Sent: Friday, August 18, 2017 3:13:19 PM
To: annou...@apache.org; user; dev
Subject: [Announce] Apache Kylin 2.1.0 released

The Apache Kylin team is pleased to announce the immediate availability of the 
2.1.0 release.

This is a major release after 2.0, with more than 100 bug fixes and 
enhancements; All of the changes in this release can be found in:
https://kylin.apache.org/docs21/release_notes.html

You can download the source release and binary packages from Apache Kylin's 
download page: https://kylin.apache.org/download/

Apache Kylin is an open source Distributed Analytics Engine designed to provide 
SQL interface and multi-dimensional analysis (OLAP) on Apache Hadoop, 
supporting extremely large datasets.

Apache Kylin lets you query massive data set at sub-second latency in 3 steps:
1. Identify a star schema or snowflake schema data set on Hadoop.
2. Build Cube on Hadoop.
3. Query data with ANSI-SQL and get results in sub-second, via ODBC, JDBC or 
RESTful API.

Thanks everyone who have contributed to the 2.1.0 release.

We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at
https://kylin.apache.org/

--
Best regards,

Shaofeng Shi 史少锋