How to increase map and reduce task number during #3 Step Name: Extract Fact Table Distinct Columns

2017-02-13 Thread peter zhang
Hi all, I have a cube build very slowly, especially the step 3* Extract Fact Table Distinct Columns*. I checked job log at YARN, found the map and task number is small then previous 2 steps. Could any one tell me how to determine the map and reduce task number to increase job parallel. Thanks!

关于kylin查询问题咨询

2017-02-13 Thread 仇同心
大家好： 目前在使用中遇到一个kylin查询慢的场景: Cube 设计如下： { "uuid": "dfb77a08-f51d-4088-b559-1da67c28a068", "last_modified": 1486997875953, "version": "1.6.0", "name": "insu_t", "model_name": "insu_jdmall_model_test", "description": "", "null_string": null, "dimensions": [ { "name":

Re: kylin job stop accidentally and can resume success!

2017-02-13 Thread Alberto Ramón
Do you have the Resource Manager in a dedicated node ?(without container or Node Manager) 2017-02-13 17:38 GMT+01:00 不清 <452652...@qq.com>: > I check the configure in CM。 > > Java Heap Size of ResourceManager in Bytes =1536 MiB > Container Memory Minimum =1GiB > > Container Memory Increment

Re: kylin job stop accidentally and can resume success!

2017-02-13 Thread Alberto Ramón
check this : "Basically, it means RM can only allocate memory to containers in increments of . . . " TIP: is your RM in a work node? If this is true, this can be the problem (Its good idea put yarn master, RM, in a dedicated

?????? kylin job stop accidentally and can resume success!

2017-02-13 Thread ????
how can i get this heap size?? -- -- ??: "Alberto Ram??n";; : 2017??2??14??(??) 0:17 ??: "user"; : Re: kylin job stop accidentally and can resume success! Sounds like a

Re: kylin job stop accidentally and can resume success!

2017-02-13 Thread Alberto Ramón
Sounds like a problem of Resource Manager (RM) of YARN, check the Heap size for RM Kylin loose connectivity whit RM 2017-02-13 17:00 GMT+01:00 不清 <452652...@qq.com>: > hello,kylin community! > > sometimes my jobs stop accidenttly.It is can stop by any step. > > kylin log is like : > 2017-02-13

Re: 求助有一个超大维度

2017-02-13 Thread ShaoFeng Shi
root cause: java.lang.OutOfMemoryError: GC overhead How many dimensions in the cube? if you can share the cube JSON, that would be helpful for trouble shooting. 2017-02-13 10:53 GMT+08:00 不清 <452652...@qq.com>: > kylin社区,您好！ > > 是手机号作为维度，这个维度的去重值在500w~1500w。 > 我是使用的integer 编码 然后length设置为8.

Re: 求助有一个超大维度

2017-02-13 Thread Alberto Ramón
for B: its a java option, (. . . java.opts) Check if your JVM isn't very old, there are a lot of optimizacions for GC in last versions of Java 8 TIP 1: Check if you can reduce dimensionality of cube or use AGG to make lighter the build process You canTake some ideas from this

Re: New document: "How to optimize cube build"

2017-02-13 Thread ShaoFeng Shi
correct. Get Outlook for iOS On Mon, Feb 13, 2017 at 3:52 PM +0800, "Ajay Chitre" wrote: In this case, if user runs a query with a WHERE clause that has 2 dimensions from the "aggregation group" & 2 dimensions from the "other 5 dimensions", Kylin will