Opps, I think Hongbin already opened JIRA already: https://issues.apache.org/jira/browse/KYLIN-956
Please generate patch and attach to this JIRA. Thanks. Best Regards! --------------------- Luke Han On Sun, Aug 23, 2015 at 11:42 PM, Luke Han <[email protected]> wrote: > Hi Meng, > Thank you very much to add Snappy support, could you please follow > this flow to submit your patch: > 1. Open JIRA first: > https://issues.apache.org/jira/browse/KYLIN > <https://issues.apache.org/jira/browse/KYLIN> > 2. Generate patch, for 0.7-staging branch > 3. Attach patch to that JIRA > > Committers will review and merge your patch to branch if there's no > issue. > > More detail, please refer to "How to contribute": > http://kylin.incubator.apache.org/development/howto_contribute.html > <http://kylin.incubator.apache.org/development/howto_contribute.html> > > Thank you very much and looking forward for your submission:) > > Luke > > > Best Regards! > --------------------- > > Luke Han > > On Sun, Aug 23, 2015 at 11:02 PM, [email protected] <[email protected] > > wrote: > >> i had fixed this, and submitted to the master branch, is that right? >> >> add kylin.hbase.default.compression.codec to kylin.properties and >> modified CreateHTableJob.java >> >> #default compression codec for htable,snappy,lzo,gzip,lz4 available >> kylin.hbase.default.compression.codec=snappy >> >> >> >> >> ------------------ 原始邮件 ------------------ >> 发件人: liangmeng <[email protected]>; >> 发送时间: 2015-08-19 13:38:02 >> 收件人: dev <[email protected]>; >> 抄送: (无); >> 主题: kylin force to use lzo in hbase? >> in the doc, it says lzo compression is not used by default, but >> actually, if cluster is configured with lzo, kylin will use it, so i >> review the source code, and find that kylin determine to use lzo in hbase >> table by compression test result, not user's configuration; >> also, we prefer snappy as default compression, will kylin support it ? >> >> ////////////////////////////////////////////////// >> this is the source code in CreateHTableJob.java: >> >> for (HBaseColumnFamilyDesc cfDesc : >> cubeDesc.getHBaseMapping().getColumnFamily()) { >> HColumnDescriptor cf = new >> HColumnDescriptor(cfDesc.getName()); >> cf.setMaxVersions(1); >> >> if (LZOSupportnessChecker.getSupportness()) { >> logger.info("hbase will use lzo to compress data"); >> cf.setCompressionType(Algorithm.LZO); >> } else { >> logger.info("hbase will not use lzo to compress >> data"); >> } >> >> cf.setDataBlockEncoding(DataBlockEncoding.FAST_DIFF); >> cf.setInMemory(false); >> cf.setBlocksize(4 * 1024 * 1024); // set to 4MB >> tableDesc.addFamily(cf); >> } >> >> >> >> public class LZOSupportnessChecker { >> private static final Logger log = >> LoggerFactory.getLogger(LZOSupportnessChecker.class); >> >> public static boolean getSupportness() { >> try { >> File temp = File.createTempFile("test", ".tmp"); >> CompressionTest.main(new String[] { "file://" + >> temp.getAbsolutePath(), "lzo" }); >> } catch (Exception e) { >> log.error("Fail to compress file with lzo", e); >> return false; >> } >> return true; >> } >> >> public static void main(String[] args) throws Exception { >> System.out.println("LZO supported by current env? " + >> getSupportness()); >> } >> } >> >> >> >> >> >> >> >> >
