Re: [Discuss] Apache Kylin Component Owner Plan

2018-02-04 Thread yu feng
may I take JDBC components, and backup with @Dong Li , Thanks. 2018-02-05 9:11 GMT+08:00 Jianhua Peng : > +1. > > I would like to take the ownership of "Security" components. > I also could be a backup owner for "Job Engine" components. > > Best regards, > Jianhua Peng >

Re: kylin job block hbase

2017-10-15 Thread yu feng
<shaofeng...@apache.org>: > Did you configure "kylin.hbase.cluster.fs", pointing to your HBase HDFS? > > Check this blog for more: > https://kylin.apache.org/blog/2016/06/10/standalone-hbase-cluster/ > > 2017-10-16 9:51 GMT+08:00 yu feng <olaptes...@gmail.com>

Re: kylin job block hbase

2017-10-15 Thread yu feng
t; is an MR job, won't block HBase normal tasks. > > The HBase BulkLoad on HDFS should be very fast (second level), as it is > just a move operation. > > For your case, is your HBase running with another HDFS other than the > default HDFS? > > > 2017-10-13 16:16 GMT+08:00 yu f

kylin job block hbase

2017-10-13 Thread yu feng
A very big cube, such as cube size id bigger than 1TB will block hbase's normal operation when doing the BulkLoad job (because the job will write to much data to HDFS), such as kylin metadata operation/ query. especially when the cube's merge job maybe write to hbase N TB in a mr job. Has anyone

Re: different mr config for different project or cube

2017-09-06 Thread yu feng
How to do it in GUI? by cube level configuration ? show me an example please. 2017-09-06 10:18 GMT+08:00 Billy Liu <billy...@apache.org>: > You could override the default MR config on the project level or cube level > through the GUI, not the file. > > 2017-09-06 10:11

different mr config for different project or cube

2017-09-05 Thread yu feng
I remember Kylin support use different mr config file for different project, like KYLIN-1706 and KYLIN-1706 However, I do not know how to use it in kylin-2.0.0. it would be appreciated if

Re: About kylin ACL

2017-09-01 Thread yu feng
you want to grant permission of > Click Access, then you can grant access at project level > > > 何京珂 > Joanna He > > > On 01/09/2017, 9:29 AM, "yu feng" <olaptes...@gmail.com> wrote: > > How to grant permission at project level, I just find the way that >

Re: About kylin ACL

2017-08-31 Thread yu feng
How to grant permission at project level, I just find the way that grant to cube. 2017-08-31 10:24 GMT+08:00 Li Yang <liy...@apache.org>: > The ADMIN can grant permission to the MODELER at project level. After > creating the new project. > > On Tue, Aug 22, 2017 at 4:26 PM

About kylin ACL

2017-08-22 Thread yu feng
In kylin, we have three role: ADMIN/ MODELER and ANALYST I find only ADMIN have Permission to create new project, But as a new normal MODELER user, What can I do in kylin? If ADMIN create a project for me, How to grant it to a MODELER user?

Kylin query return empty after upgrade to 2.0

2017-08-21 Thread yu feng
Hi, After I upgrade our env. from 1.5.2.1 to 2.0.0, building things go well, However, every query to older(generated before upgrading) return empty(0 result), and newly build segment return result success. I add some debug log and find in GTFilterScanner, I get 0 input from inputIterator and

Re: [VOTE] Release apache-kylin-2.1.0 (RC2)

2017-08-16 Thread yu feng
+1(no binding) build success md5 verified Best regards, 2017-08-15 23:57 GMT+08:00 Luke Han : > +1 (binding) > > mvn test passed > gpg/md5/sha1 verified > > > Best Regards! > - > > Luke Han > > On Tue, Aug 15, 2017 at 9:13 PM, 康凯森 wrote:

Re: [VOTE] Release apache-kylin-2.0.0 (RC3)

2017-04-28 Thread yu feng
+1(no binding) mvn test passed build(mvn clean install -DskipTests) success md5 verified 2017-04-29 0:30 GMT+08:00 Dong Li : > +1 binding > > mvn test passed > > Thanks, > Dong Li > > 2017-04-28 22:12 GMT+08:00 ShaoFeng Shi : > > > +1 binding; > > > >

Re: Unable to connect to Kylin Web UI

2016-10-30 Thread yu feng
we always use kylin index page : http://hostname:7070/kylin/login 2016-10-30 16:36 GMT+08:00 BigdataGR : > Hi, > > Can anyone please help me to move forward ? I am stuck at accessing kylin > web UI even kylin is started successfully as shown in the logs enclosed. > >

normal user can not load table

2016-06-29 Thread yu feng
when load table in a project, the user must has ADMIN role to syncTableToProject, is it some error about this logical

Re: id-name

2016-06-14 Thread yu feng
derived dimension is a better choice. 2016-06-14 23:00 GMT+08:00 耳东 <775620...@qq.com>: > hi allwhen I build the cube, there are always dimension like > (city_id,city_name) or (terminal_id,terminal_name) and so on, which is one > to one relation. I don't want to let both id and name be

TOPN group by column can not defined in dimension

2016-06-03 Thread yu feng
I create a cube with kylin sample data, cube defination is below : { "uuid": "e6cf2ccc-edca-41c6-b637-b3bc50894b5e", "version": "1.5.1", "name": "kylin_sales_cube_desc_2_clone", "description": null, "dimensions": [ { "name": "CAL_DT", "table": "DEFAULT.KYLIN_CAL_DT",

TOPN group by column can not defined in dimension

2016-06-02 Thread yu feng
I create a cube with kylin sample data, cube defination is below : { "uuid": "e6cf2ccc-edca-41c6-b637-b3bc50894b5e", "version": "1.5.1", "name": "kylin_sales_cube_desc_2_clone", "description": null, "dimensions": [ { "name": "CAL_DT", "table": "DEFAULT.KYLIN_CAL_DT",

Re: How to Build Binary Package in 1.5.x

2016-05-04 Thread yu feng
the script has moved to ${source_code}/build/script/package.sh 2016-05-05 11:12 GMT+08:00 zeLiu : > 1.5.x has not script/package.sh,How to Build Binary Package? thanks > > -- > View this message in context: >

Re: [VOTE] Release apache-kylin-1.5.1 (release candidate 1)

2016-04-10 Thread yu feng
+1 (binding) signature verified mvn test passed 2016-04-09 17:38 GMT+08:00 Luke Han : > +1 (binding) > > mvn test passed > signature verified > > > Best Regards! > - > > Luke Han > > On Sat, Apr 9, 2016 at 5:15 PM, 周千昊 wrote: > > > +1

Re: cube build failed if the fact_table is not in default database

2016-03-09 Thread yu feng
FAILED: SemanticException 1:567 AS clause has an invalid number of aliases. I don't think it is error of hive, you should post you hive command here, and let's talk about it. 2016-03-10 1:07 GMT+08:00 Sarnath : > Its hard to believe that select * from db.table does not

Re: building cube error in step 2 (StandbyException)

2016-02-28 Thread yu feng
you can deploy kylin at any node, you need ensure in that node you can run 'hadoop/hive/hbase' command in shell and get corrent outoput. you can try to move kylin to another node and config you hadoop configuration with HA mode which makes you can access it with nameservice. 2016-02-26 14:16

Re: when i use curl with Restful Api , HTTP Status 500 - Cannot create a session after the response has been committed

2016-02-28 Thread yu feng
Hi, I have encountered this problem, this is caused by "Authorization: Basic QURNSU46S1lMSU4K" is incorrent, you should use "echo -n 'ADMIN:KYLIN'| base64" to generate base64 encoding, which output is "QURNSU46S1lMSU4=", and you can try it again. wish it can halp you ~ 2016-02-26 20:47 GMT+08:00

How to use Hybrid model in Kylin

2016-02-23 Thread yu feng
Hi All : I find kylin-1.0 can generate Hybrid model which is composed by multiple cubes, jira is here : https://issues.apache.org/jira/browse/KYLIN-867 My question is how to use this feature in web UI. BTW, if I release some segments, but the cuboid files in HDFS and htable in Hbase of those

Re: How to use kylin with high cardinality dimensions.

2016-02-18 Thread yu feng
org/jira/browse/KYLIN-1313 for more details. 3. > > Adopt > > scalable dictionary solutions to replace current in-memory dictionary > > building (TBD) > > > > I'm also answering your questions inline with this pen > > > > On Thu, Feb 18, 2016 at 11:03 AM,

How to use kylin with high cardinality dimensions.

2016-02-17 Thread yu feng
Hi All: We are encounting some problems while supporting a demand that a cube with some high cardinality dimensions, those dimensions are URLs and user want to use those dimensions in where clause and filter with like function. besides, the cube has one distinct count measure. We has such

Re: [DISCUSS]Apache Kylin 2.0 Release Features & Criteria

2016-02-02 Thread yu feng
We are looking forward the release of 2.0, fast cubing algorithm, spark supporting and streaming cube is very useful to us. I have test 2.0-rc in our environment and it works fine, wish the release comes soon. 2016-02-02 18:02 GMT+08:00 Yerui Sun : > We’ve been looking

Re: only one reducer in job

2016-02-02 Thread yu feng
Hi, as far as I know, KYLIN-1066 is not about what you are talking about, this job "Kylin Hive Column Cardinality Job" is submitted after you loading table, it's function is calculate Cardinality of every column using hyperLogLog, so it have to

Re: kylin concurrency test

2016-01-27 Thread yu feng
icture for convenience, bottleneck of kylin query throughput rely on hbase scan performance, which will related to regionserver number and machine configuration, network etc. 2016-01-28 10:08 GMT+08:00 Luke Han <luke...@gmail.com>: > It's QPS, please contact Yu Feng (kylin committer) from N

Re: Convert Cuboid Data to HFile failed when hbase in different HDFS

2016-01-22 Thread yu feng
Tue, Jan 19, 2016 at 10:23 PM, yu feng <olaptes...@gmail.com> wrote: > > > In the step 'Convert Cuboid Data to HFile' execute failed, error log is : > > > > > > java.io.IOException: Failed to run job : Unable to map logical > nameservice > > URI 'hdfs://A' t

Re: internal hive table and build the cube backward

2016-01-19 Thread yu feng
Hi Yu and Everyone, > > > > Just a little bit supplement, Hive definitely involves in the step of > > Create > > Intermediate Flat Hive Table and Build Dimension Dictionary. The question > > is that does Hive involve in the following steps of building cuboids? > > >

Re: internal hive table and build the cube backward

2016-01-17 Thread yu feng
Firstly, kylin do not distinguish which kind table in hive, if only you can query it in hive, so the table can be normal table, external table, view or table with some serdes. then I think it is hard to build cube backward along the time in kylin. maybe someone has some good ideas at this point.

Re: Exception on building cube at 2nd step (IncompatibleClassChangeError)

2016-01-15 Thread yu feng
I find there are some hadoop jar of 2.2.0 in /home/hadoop/hbase/hbase-0.98.15-hadoop2/lib, and your hadoop is 2.5.2, this will bring collision, please check your env again. 2016-01-16 4:26 GMT+08:00 Eric Engstfeld : > Hi, > > I have this exception on the 2nd step of cube build. I

Re: Kylin capabilities

2016-01-15 Thread yu feng
; - > > Luke Han > > On Sat, Jan 16, 2016 at 8:58 AM, yu feng <olaptes...@gmail.com> wrote: > > > Does Kylin offer nesting dimensions or hierarchies? > > > > > Kylin does not support Snowflake schema, It is currently restricted to >

Re: Kylin capabilities

2016-01-15 Thread yu feng
Does Kylin offer nesting dimensions or hierarchies? > Kylin does not support Snowflake schema, It is currently restricted to >Star. Generally speaking, we always transform snowflake schema to star schema by creating some view. 2016-01-16 3:31 GMT+08:00 Adunuthula, Seshu :

Re: can we support adding mapping cube columns to hive table columns

2016-01-12 Thread yu feng
My suggestion is adding this mapping by creating view, So, you can change the column name in hive table, and recreate the view, that will do not have any effect to cube. and need not sync hive table. this view only used at the first step, so can recreate it in any time except doing this step.

Re: Relationship between rowkey column length and cube size

2016-01-09 Thread yu feng
Let me have a try to explain it. Cube size determines how to split region for table in hbase after generate all cuboid files, for example, If all of your cuboid file size is 100GB, your cube size set to "SMALL", and the property for SMALL is 10GB, kylin will create hbase table with 10 regions.

Re: build a cube with two ultra high cardinality columns

2016-01-08 Thread yu feng
assume average size of this column is 32 bytes, 50 millions cardinality means 1.5GB, in the step of 'Fact Table Distinct Columns.' mapper need read from intermediate table and remove duplicate values(do it in Combiner), however, this job will startup more than one mapper and just one reducer,

Re: restful interface

2016-01-08 Thread yu feng
p-outgoing-0 << Date: > Fri, 08 Jan 2016 10:41:50 GMT > 18:43:03 MainClientExec - Connection can be kept alive indefinitely > 18:43:03 ResponseProcessCookies - Cookie accepted > [JSESSIONID="1B48E4EE1F327464EA6B32449346BBB7", version:0, > domain:192

Re: Re: How to improve the performance of job!

2016-01-07 Thread yu feng
.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > My profile kylin_job_conf.xml: > > mapreduce.input.fileinputformat.split.maxsize >

Re: Error when scan from lower key

2016-01-06 Thread yu feng
This means your one of hbase scanner execute failed, you can check logs in regionserver for more detailed infomation. 2016-01-07 3:46 GMT+08:00 sdangi : > Hi Kylin Team -- > > > > I'm hitting the below error when executing query with the time dimension in > the where clause

Re: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_learn_kylin

2016-01-06 Thread yu feng
gt; > You can config the "hive.metastore.uris" property in hive-site.xml and > start hive metastore. so HCatalog can connect to it. > > > > 在 2016年01月07日 12:52, yu feng 写道: > >> you can check whether the table "default.kylin_intermediate_ >> learn_kylin_f

Re: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_learn_kylin

2016-01-06 Thread yu feng
I have encountered this problem, it is more likely because your current hive metastore config is error, could you tell some more detailed infomation about your env. 2016-01-07 10:23 GMT+08:00 和风 <363938...@qq.com>: > hi: > i build cube have a error. > logs: > java.io.IOException: >

Re: aboutthe; parameter 'acceptPartial'

2016-01-04 Thread yu feng
In my opinion, acceptPartial means whether return partial result if your query result has more rows than limit value , and it will always set to true no matter query from web UI or jdbc. However, if you query from web UI, you can set the limit (default is 5), if you query from jdbc, the

Feature about taking more than one hive source as kylin input in kylin-1.x

2016-01-01 Thread yu feng
Hi all: I have been submit a patch on kylin-1.0(KYLIN-1172), it can take more hive as kylin input. even those hive based on different hadoop cluster. However, After I consult with @Luke Han and @Shi, Shaofeng, they do not plan to add new big fearures to kylin-1.x which is in a stable state.

Re: FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.ConditionalWork cannot be cast to org.apache.hadoop.hive.ql.plan.MapredWork

2015-12-28 Thread yu feng
execute this sql with hive -e 'sql' and check the result, I think this is something error with you hive env, maybe hive version problem... 2015-12-28 18:51 GMT+08:00 taylor zhang : > hi all > > I got below error when ran example cude 'kylin_sales_cube', it seems that > hive

Re: cube model will be overridden while creating a new cube with the same name

2015-12-25 Thread yu feng
I'll fix this. > > > > https://issues.apache.org/jira/browse/KYLIN-1254 > > > > On Fri, Dec 25, 2015 at 3:17 PM, yu feng <olaptes...@gmail.com> wrote: > > > >> Hi, I find a Bug like this in kylin-1.0: > >> I build a cube named TEST successfully,

cube model will be overridden while creating a new cube with the same name

2015-12-24 Thread yu feng
Hi, I find a Bug like this in kylin-1.0: I build a cube named TEST successfully, and then build another cube named TEST too, those two cube has different fact table, the second cube will be error becacuse "The cube named TEST already exists", But the model of first cube will be overridden. this

error in calculate cardinality for view.

2015-12-23 Thread yu feng
I load a view in onew project, However, calculat cardinality always error like this: java.io.IOException: java.lang.NullPointerException at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at

Re: error in calculate cardinality for view.

2015-12-23 Thread yu feng
, already recorded in > https://issues.apache.org/jira/browse/KYLIN-916 > The workaround is to load and then calculate cardinality for the > underlying table which is not a view; > > On 12/24/15, 3:11 PM, "yu feng" <olaptes...@gmail.com> wrote: > > >I load a vi

build dictionary for timestamp column with DateStrDictionary

2015-12-15 Thread yu feng
Hi All : I build a cube, fact table like this : hive> describe testtimestamp; OK ts timestamp fname string lname string type int cost int I build a cube with dimensions 'ts', 'fname', 'lname' and type, However,