do we have any progress on such documents? On Tue, Aug 2, 2016 at 8:50 AM, Yiming Liu <[email protected]> wrote:
> Thanks, Shaofeng. It makes sense to grant enough privileges to Kylin for > Cube building. Just in some extreme cases, the privilege issue will be a > show stop. > > The privilege document is great. It's very helpful for Hadoop system > administrator. > > 2016-08-01 9:38 GMT+08:00 ShaoFeng Shi <[email protected]>: > > > Hi Yiming, > > > > The "mapreduce.job.reduces" need by set at runtime, whose number is > > calculated based on user tables' size, it couldn't be pre-configured. > > > > The "hive.merge.mapredfiles=false" can be externalized to the conf file; > > The hive merge is not needed since 1.5.3, I set in code to ensure it will > > be not be enabled (config files before 1.5.3 has this param set to true). > > > > For other parameters, I think they're optional, but it is better to keep > as > > they're good for performance, like dfs.replication=2, compress.codec etc. > > > > Usually in a hadoop cluster, Apache Kylin should be treated as a > > priviledged user (instead of a normal user like analyst), which can > execute > > necessary hadoop/hdfs/hbase/hive actions (like mkdir, create htable, > etc); > > To achieve this, the administartor need do some configurations and > > authorizations; What we need do is to compose a document to list > > these privileges, what's your opinion? > > > > Thanks for the comment! > > > > > > 2016-07-30 14:03 GMT+08:00 Yiming Liu <[email protected]>: > > > > > Hi Kylin dev, > > > > > > The first step is building cube is to CreateFlatHiveTable, it will > call a > > > few hive configuration commands, such as > > > CreateFlatHiveTableStep line 78 and 79. > > > set mapreduce.job.reduces=numReduces > > > set hive.merge.mapredfiles=false > > > > > > Are these commands necessary for the cube building? Could we configure > > them > > > in files? I met some cases, where the hiveserver would say > "Configuration > > > is not allowed to modify at runtime". It will break the build. > > > > > > Maybe there are some other hard code hadoop commands still. It will be > > more > > > friendly if they could turn off on demand. > > > > > > -- > > > With Warm regards > > > > > > Yiming Liu (刘一鸣) > > > > > > > > > > > -- > > Best regards, > > > > Shaofeng Shi > > > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) > -- Regards, *Bin Mahone | 马洪宾*
