it seems yang has already ported changes to the 1.x-staging branch, thanks yang.
周千昊 <[email protected]>于2015年10月6日周二 下午7:26写道： > OK, I will re-port the changes to 1.x-staging > > Li Yang <[email protected]>于2015年10月6日周二 下午3:28写道： > >> Qianhao, pls don't patch 1.x-HBase1.x directly. All changes should go into >> 1.x-staging first, then apply an HBase patch to become 1.x-HBase1.x >> >> Ported the fix to 1.x-staging and promoted 1.x-HBase1.x branch again. The >> result should be same. The effective commons-codec is 1.9 as shown in >> eclipse. >> >> On Mon, Oct 5, 2015 at 11:47 PM, 周千昊 <[email protected]> wrote: >> >> > Hi, Benjamin, >> > It is a bug, a jira ticket is created as follow: >> > https://issues.apache.org/jira/browse/KYLIN-1059 >> > and it should have been fixed. since I don't have the hdp 2.3 env >> > currently, it would be appreciated if you can help to verify with the >> > latest code(branch 1.x-HBase1.x) >> > >> > Luke Han <[email protected]>于2015年10月5日周一 下午8:35写道： >> > >> > > Hi Benjamin, >> > > Did you generate binary package by yourself with below >> instruction? >> > or >> > > just compile war/jar from maven? >> > > >> > > http://kylin.incubator.apache.org/development/howto_package.html >> > > <http://kylin.incubator.apache.org/development/howto_package.html> >> > > >> > > Thanks. >> > > >> > > >> > > Best Regards! >> > > --------------------- >> > > >> > > Luke Han >> > > >> > > On Thu, Oct 1, 2015 at 1:30 PM, Benjamin Leonhardi < >> > > [email protected]> wrote: >> > > >> > > > Hello All, >> > > > >> > > > I try to get Kylin to run with HDP 2.3 sandbox image. Using the >> > HBase1.x >> > > > branch. >> > > > >> > > > https://github.com/apache/incubator-kylin/tree/1.x-HBase1.x >> > > > >> > > > It compiles fine and the tests run through until they reach the job >> > > > creation. I followed the steps below. >> > > > >> > > > http://kylin.incubator.apache.org/development/dev_env.html >> > > > >> > > > When I run the tests he reaches the job part and then fails because >> he >> > > > misses a method in commons-codec.jar. The method is >> > encodeBase64(byte[], >> > > > isChunk boolean, isUrlSafe boolean) and it is in commons-codec1.4+ >> but >> > > not >> > > > in older versions. I checked but my environment only has codec jars >> > that >> > > > are newer. >> > > > >> > > > However when I build kylin he for some reason compiled an older >> codec >> > > into >> > > > the kylin-job-1.1-incubating-SNAPSHOT-job.jar. If I unzip this I >> see a >> > > > Base64 class that only has the method encodeBase64(byte[], isChunk >> > > boolean >> > > > ) but not the urlsafe method. ( so I suppose codec 1.3 or older ) I >> > > tried >> > > > to check the pom.xml because I thought I might find an older codec >> > > > dependency inside but did not find anything. >> > > > >> > > > I also have to say I am not the best Maven person so I might be >> > > > overlooking something easy. One question I would have is why he >> > compiles >> > > > the code into the -job.jar at all. >> > > > >> > > > >> > > > Any help welcome >> > > > >> > > > Best regards, >> > > > >> > > > Benjamin >> > > > >> > > > >> > > > Starting: >> > > > Kylin_Fact_Distinct_Columns_test_kylin_cube_without_slr_empty_Step >> > > > L4J [2015-10-01 >> > > > 03:51:24,166][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > append >> > > > job jar: >> > > > >> /root/kylin_build/incubator-kylin/job/../job/target/kylin-job-1.1-incu >> > > > bating-SNAPSHOT-job.jar >> > > > L4J [2015-10-01 >> > > > 03:51:24,169][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > append >> > > > kylin.hive.dependency: null and kylin.hbase.dependency: null to >> > > > mapreduce.appli >> > > > cation.classpath >> > > > L4J [2015-10-01 >> > > > 03:51:24,167][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > append >> > > > job jar: >> > > > >> /root/kylin_build/incubator-kylin/job/../job/target/kylin-job-1.1-incu >> > > > bating-SNAPSHOT-job.jar >> > > > L4J [2015-10-01 >> > > > 03:51:24,170][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > append >> > > > kylin.hive.dependency: null and kylin.hbase.dependency: null to >> > > > mapreduce.appli >> > > > cation.classpath >> > > > L4J [2015-10-01 >> > > > 03:51:24,170][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > Hadoop >> > > > job classpath is: >> > > > >> > > >> > >> /tmp/kylin/*,$HADOOP_CONF_DIR,/usr/hdp/2.3.0.0-2530/hbase/lib/hbase-common.jar,/usr/hdp/current/hive-client/conf/,/usr/hdp/2.3.0.0-2530/hive/lib/hive-metastore.jar,/usr/hdp/2.3.0.0-2530/hive/lib/hive-exec.jar,/usr/hdp/2.3.0.0-2530/hive-hcatalog/share/hcatalog/*,$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/2.3.0.0-2530/hadoop/lib/hadoop-lzo-0.6.0.2.3.0.0-2530.jar:/etc/hadoop/conf/secure >> > > > L4J [2015-10-01 >> > > > 03:51:24,169][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > Hadoop >> > > > job classpath is: >> > > > >> > > >> > >> /tmp/kylin/*,$HADOOP_CONF_DIR,/usr/hdp/2.3.0.0-2530/hbase/lib/hbase-common.jar,/usr/hdp/current/hive-client/conf/,/usr/hdp/2.3.0.0-2530/hive/lib/hive-metastore.jar,/usr/hdp/2.3.0.0-2530/hive/lib/hive-exec.jar,/usr/hdp/2.3.0.0-2530/hive-hcatalog/share/hcatalog/*,$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/2.3.0.0-2530/hadoop/lib/hadoop-lzo-0.6.0.2.3.0.0-2530.jar:/etc/hadoop/conf/secure >> > > > L4J [2015-10-01 >> > 03:51:24,229][WARN][org.apache.hadoop.hive.conf.HiveConf] >> > > > - HiveConf of name hive.heapsize does not exist >> > > > L4J [2015-10-01 >> > 03:51:24,230][WARN][org.apache.hadoop.hive.conf.HiveConf] >> > > > - HiveConf of name hive.server2.enable.impersonation does not exist >> > > > L4J [2015-10-01 >> > 03:51:24,236][WARN][org.apache.hadoop.hive.conf.HiveConf] >> > > > - HiveConf of name hive.heapsize does not exist >> > > > L4J [2015-10-01 >> > 03:51:24,237][WARN][org.apache.hadoop.hive.conf.HiveConf] >> > > > - HiveConf of name hive.server2.enable.impersonation does not exist >> > > > L4J [2015-10-01 >> > > > 03:51:24,265][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > > tempMetaFileString is : null >> > > > L4J [2015-10-01 >> > > > 03:51:24,266][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob] - >> > > > tempMetaFileString is : null >> > > > L4J [2015-10-01 >> > > > >> > 03:51:24,267][ERROR][org.apache.kylin.job.execution.AbstractExecutable] >> - >> > > > error running Executable >> > > > java.lang.NoSuchMethodError: >> > > > org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.encodeBytes(HCatUtil.java:125) >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.serialize(HCatUtil.java:104) >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.getHiveConf(HCatUtil.java:585) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:100) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:101) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:77) >> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> > > > at >> > > > >> > > >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> > > > at >> > > > >> > > >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> > > > at java.lang.Thread.run(Thread.java:745) >> > > > L4J [2015-10-01 >> > > > >> > 03:51:24,267][ERROR][org.apache.kylin.job.execution.AbstractExecutable] >> - >> > > > error running Executable >> > > > java.lang.NoSuchMethodError: >> > > > org.apache.commons.codec.binary.Base64.encodeBase64([BZZ)[B >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.encodeBytes(HCatUtil.java:125) >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.serialize(HCatUtil.java:104) >> > > > at >> > > > >> org.apache.hive.hcatalog.common.HCatUtil.getHiveConf(HCatUtil.java:585) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:100) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) >> > > > at >> > > > >> > > >> > >> org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:101) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:77) >> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> > > > at >> > > > >> > > >> > >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> > > > at >> > > > >> > > >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> > > > at >> > > > >> > > >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> > > > >> > > > at java.lang.Thread.run(Thread.java:745) >> > > > >> > > >> > >> >
