thanks hongbin~ 2015-03-27 10:01 GMT+08:00 hongbin ma <[email protected]>:
> Can you let me know the reason for number 2? is it because the version > you're using does not include a "setenv.sh" in bin folder? > > If yes, simply adding "export ..." will not work, because in old versions > the variable KYLIN_LD_LIBRARY_PATH might not get used. > > If you can use the latest version on > http://kylin.incubator.apache.org/download/, please update that. > Otherwise, you'll need to manually add -Djava.library.path > "path/to/native/Linux-amd64-64" in kylin.sh's "hbase > {...} org.apache.hadoop.util.RunJar ${tomcat_root}/bin/bootstrap.jar" as a > startup parameter. > > Kylin will run in a tomcat instance, so we'll need to pass the native > library path to tomcat via java.library.path. > > And please make sure your native lzo libaries are installed in > path/to/native/Linux-amd64-64 > on your environment > > > On Thu, Mar 26, 2015 at 2:15 PM, dong wang <[email protected]> wrote: > > > 1, I configured all the compression properties in core-site.xml, and > > mapred-site.xml > > > > 2, since I cannot find "Before starting Kylin again, you'll need to > > uncomment KYLIN_LD_LIBRARY_PATH in ${KYLIN_HOME}/bin/setenv.sh", thus, I > > add "export KYLIN_LD_LIBRARY_PATH" to the very beginning of the file > > $KYLIN_HOME/bin/kylin.sh > > > > 3, as tested, it seems that the HIVE table will use lzo compression, > > however, the cube table in HBase does't apply lzo compression since when > > desc the hbase table for the cube, it shows "COMPRESSION => 'NONE'" for > its > > detailed information > > > > 4, note that when run commands such as "create 'mytable', > {NAME=>'colfam:', > > COMPRESSION=>'lzo'} " and "hbase > > org.apache.hadoop.hbase.util.CompressionTest abc.txt lzo", it works OK > > > > > > > > 2015-03-26 13:36 GMT+08:00 dong wang <[email protected]>: > > > > > as tested, it seems that the HIVE table will use lzo compression, and > the > > > cube table in HBase does't apply lzo compression > > > > > > 2015-03-26 13:03 GMT+08:00 dong wang <[email protected]>: > > > > > >> i c, what's more, if I configure the ALL the compression properties in > > >> core-site.xml and mapred-site.xml, is it still neccessary for me to > do > > the > > >> change like "export > > >> > > > KYLIN_LD_LIBRARY_PATH=::/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native" > > >> if I want to use the compression feature? > > >> > > >> 2015-03-25 21:17 GMT+08:00 hongbin ma <[email protected]>: > > >> > > >>> Usually it's not necessary to change hbase's configuration. > > >>> > > >>> However, we do have a place(*conf/kylin_job_conf.xml*) for users to > > >>> modify > > >>> MR jobs' configuration. One example would be adding lzo compression > to > > >>> Mappers' and Reducers' outputs. *This will reduce the final cube > size a > > >>> lot*. > > >>> Take a look at > > >>> > > >>> > > > https://github.com/KylinOLAP/Kylin/blob/master/docs/Operations/Advanced%20settings%20of%20Kylin%20environment.md > > >>> for how to. > > >>> > > >>> In a word, as your data size grows, if your cube building is taking > > >>> significant time or space, you'd better start to think about > optimizing > > >>> by > > >>> adding parameters in kylin_job_conf.xml > > >>> > > >>> On Wed, Mar 25, 2015 at 5:33 PM, dong wang <[email protected]> > > >>> wrote: > > >>> > > >>> > Is it necessary for user to do some configuration for > hbase-site.xml > > >>> when > > >>> > deploy KYLIN? > > >>> > > > >>> > > >>> > > >>> > > >>> -- > > >>> Regards, > > >>> > > >>> *Bin Mahone | 马洪宾* > > >>> Apache Kylin: http://kylin.io > > >>> Github: https://github.com/binmahone > > >>> > > >> > > >> > > > > > > > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone >
