Can you let me know the reason for number 2? is it because the version you're using does not include a "setenv.sh" in bin folder?
If yes, simply adding "export ..." will not work, because in old versions the variable KYLIN_LD_LIBRARY_PATH might not get used. If you can use the latest version on http://kylin.incubator.apache.org/download/, please update that. Otherwise, you'll need to manually add -Djava.library.path "path/to/native/Linux-amd64-64" in kylin.sh's "hbase {...} org.apache.hadoop.util.RunJar ${tomcat_root}/bin/bootstrap.jar" as a startup parameter. Kylin will run in a tomcat instance, so we'll need to pass the native library path to tomcat via java.library.path. And please make sure your native lzo libaries are installed in path/to/native/Linux-amd64-64 on your environment On Thu, Mar 26, 2015 at 2:15 PM, dong wang <[email protected]> wrote: > 1, I configured all the compression properties in core-site.xml, and > mapred-site.xml > > 2, since I cannot find "Before starting Kylin again, you'll need to > uncomment KYLIN_LD_LIBRARY_PATH in ${KYLIN_HOME}/bin/setenv.sh", thus, I > add "export KYLIN_LD_LIBRARY_PATH" to the very beginning of the file > $KYLIN_HOME/bin/kylin.sh > > 3, as tested, it seems that the HIVE table will use lzo compression, > however, the cube table in HBase does't apply lzo compression since when > desc the hbase table for the cube, it shows "COMPRESSION => 'NONE'" for its > detailed information > > 4, note that when run commands such as "create 'mytable', {NAME=>'colfam:', > COMPRESSION=>'lzo'} " and "hbase > org.apache.hadoop.hbase.util.CompressionTest abc.txt lzo", it works OK > > > > 2015-03-26 13:36 GMT+08:00 dong wang <[email protected]>: > > > as tested, it seems that the HIVE table will use lzo compression, and the > > cube table in HBase does't apply lzo compression > > > > 2015-03-26 13:03 GMT+08:00 dong wang <[email protected]>: > > > >> i c, what's more, if I configure the ALL the compression properties in > >> core-site.xml and mapred-site.xml, is it still neccessary for me to do > the > >> change like "export > >> > KYLIN_LD_LIBRARY_PATH=::/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native" > >> if I want to use the compression feature? > >> > >> 2015-03-25 21:17 GMT+08:00 hongbin ma <[email protected]>: > >> > >>> Usually it's not necessary to change hbase's configuration. > >>> > >>> However, we do have a place(*conf/kylin_job_conf.xml*) for users to > >>> modify > >>> MR jobs' configuration. One example would be adding lzo compression to > >>> Mappers' and Reducers' outputs. *This will reduce the final cube size a > >>> lot*. > >>> Take a look at > >>> > >>> > https://github.com/KylinOLAP/Kylin/blob/master/docs/Operations/Advanced%20settings%20of%20Kylin%20environment.md > >>> for how to. > >>> > >>> In a word, as your data size grows, if your cube building is taking > >>> significant time or space, you'd better start to think about optimizing > >>> by > >>> adding parameters in kylin_job_conf.xml > >>> > >>> On Wed, Mar 25, 2015 at 5:33 PM, dong wang <[email protected]> > >>> wrote: > >>> > >>> > Is it necessary for user to do some configuration for hbase-site.xml > >>> when > >>> > deploy KYLIN? > >>> > > >>> > >>> > >>> > >>> -- > >>> Regards, > >>> > >>> *Bin Mahone | 马洪宾* > >>> Apache Kylin: http://kylin.io > >>> Github: https://github.com/binmahone > >>> > >> > >> > > > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
