Do we have JIRAs to track the issue? it's easy to forget On Sat, Sep 3, 2016 at 5:57 PM, Yiming Liu <[email protected]> wrote:
> No progress, pending. > > 2016-09-03 17:36 GMT+08:00 hongbin ma <[email protected]>: > > > do we have any progress on such documents? > > > > On Tue, Aug 2, 2016 at 8:50 AM, Yiming Liu <[email protected]> > > wrote: > > > > > Thanks, Shaofeng. It makes sense to grant enough privileges to Kylin > for > > > Cube building. Just in some extreme cases, the privilege issue will be > a > > > show stop. > > > > > > The privilege document is great. It's very helpful for Hadoop system > > > administrator. > > > > > > 2016-08-01 9:38 GMT+08:00 ShaoFeng Shi <[email protected]>: > > > > > > > Hi Yiming, > > > > > > > > The "mapreduce.job.reduces" need by set at runtime, whose number is > > > > calculated based on user tables' size, it couldn't be pre-configured. > > > > > > > > The "hive.merge.mapredfiles=false" can be externalized to the conf > > file; > > > > The hive merge is not needed since 1.5.3, I set in code to ensure it > > will > > > > be not be enabled (config files before 1.5.3 has this param set to > > true). > > > > > > > > For other parameters, I think they're optional, but it is better to > > keep > > > as > > > > they're good for performance, like dfs.replication=2, compress.codec > > etc. > > > > > > > > Usually in a hadoop cluster, Apache Kylin should be treated as a > > > > priviledged user (instead of a normal user like analyst), which can > > > execute > > > > necessary hadoop/hdfs/hbase/hive actions (like mkdir, create htable, > > > etc); > > > > To achieve this, the administartor need do some configurations and > > > > authorizations; What we need do is to compose a document to list > > > > these privileges, what's your opinion? > > > > > > > > Thanks for the comment! > > > > > > > > > > > > 2016-07-30 14:03 GMT+08:00 Yiming Liu <[email protected]>: > > > > > > > > > Hi Kylin dev, > > > > > > > > > > The first step is building cube is to CreateFlatHiveTable, it will > > > call a > > > > > few hive configuration commands, such as > > > > > CreateFlatHiveTableStep line 78 and 79. > > > > > set mapreduce.job.reduces=numReduces > > > > > set hive.merge.mapredfiles=false > > > > > > > > > > Are these commands necessary for the cube building? Could we > > configure > > > > them > > > > > in files? I met some cases, where the hiveserver would say > > > "Configuration > > > > > is not allowed to modify at runtime". It will break the build. > > > > > > > > > > Maybe there are some other hard code hadoop commands still. It will > > be > > > > more > > > > > friendly if they could turn off on demand. > > > > > > > > > > -- > > > > > With Warm regards > > > > > > > > > > Yiming Liu (刘一鸣) > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > > > > > Shaofeng Shi > > > > > > > > > > > > > > > > -- > > > With Warm regards > > > > > > Yiming Liu (刘一鸣) > > > > > > > > > > > -- > > Regards, > > > > *Bin Mahone | 马洪宾* > > > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) > -- Regards, *Bin Mahone | 马洪宾*
