btw, please prefer to use English, apache is a global community On Tue, Dec 22, 2015 at 4:33 PM, hongbin ma <mahong...@apache.org> wrote:
> hi, how's your hive built? is it compiled against hadoop 1.x? JobContext > was a class in hadoop 1.x, but became an interface in hadoop 2.x > > On Tue, Dec 22, 2015 at 4:23 PM, 胡志华(万里通科技及数据中心商务智能团队数据分析组) < > huzhihua...@pingan.com.cn> wrote: > >> 是的，我的很有可能是版本问题 >> >> -----邮件原件----- >> 发件人: TTS2沉默天使 [mailto:85546...@qq.com] >> 发送时间: 2015年12月22日 16:20 >> 收件人: dev >> 主题: 回复：答复: building cube , stop at step 2,help >> >> 看了下错误 你的错误和我的不一样 但是都是第二步 >> >> >> >> >> ------------------ 原始邮件 ------------------ >> 发件人: " 胡志华(万里通科技及数据中心商务智能团队数据分析组)";<huzhihua...@pingan.com.cn>; >> 发送时间: 2015年12月22日(星期二) 下午4:17 >> 收件人: "'dev@kylin.apache.org'"<dev@kylin.apache.org>; >> >> 主题: 答复: building cube , stop at step 2,help >> >> >> >> Thanks, you mean my hive version is incompatible with hadoop2.7.1 >> >> My hive is pre-build binary package, so I will check the hadoop dependecy >> >> >> -- >> ---邮件原件----- >> 发件人: 250635...@qq.com [mailto:250635...@qq.com] >> 发送时间: 2015年12月22日 16:02 >> 收件人: dev >> 主题: Re: building cube , stop at step 2,help >> >> Hi, >> I think there could be a hadoop version mismatch in your binary package. >> If you use of hive and hbase, maybe the default hadoop dependency is not >> as latest as your current use 2.7.1 , thus may lead to the >> IncompatibleClassChangeError. >> >> One possible workaround is to recompile and package your hive and hbase >> from souce code with targeting hadoop version to 2.7.1. >> >> >> >> 250635...@qq.com >> >> From: 胡志华(万里通科技及数据中心商务智能团队数据分析组) >> Date: 2015-12-22 15:54 >> To: 'dev@kylin.apache.org' >> Subject: building cube , stop at step 2,help Hi, >> >> I downloaded “apache-kylin-1.1.1-incubating-bin”, and use >> sample.sh to build a cube “kylin_sales_cube”, when I started the job , it >> stoped at step 2, with error >> >> java.lang.IncompatibleClassChangeError: Found interface >> org.apache.hadoop.mapreduce.JobContext, but class was expected >> at >> org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) >> at >> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) >> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) >> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) >> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) >> at >> org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:120) >> at >> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:83) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> at >> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> My environment lists as following >> >> Hadoop2.7.1 >> Hbase0.98.13 >> Hive0.13.1 >> >> So ,from the error code above, my hive environment is >> incompatible? And what does the parameter mean at step 2? >> >> -conf >> /home/hadoop/apache-kylin-1.1.1-incubating-bin/conf/kylin_job_conf.xml >> -cubename kylin_sales_cube -output >> /kylin/kylin_metadata/kylin-79ec4e09-9128-42bd-a455-9dcf3c16a42a/kylin_sales_cube/fact_distinct_columns >> -jobname Kylin_Fact_Distinct_Columns_kylin_sales_cube_Step -tablename >> default.kylin_intermediate_kylin_sales_cube_desc_19700101000000_20151217000000_79ec4e09_9128_42bd_a455_9dcf3c16a42a >> >> >> -----邮件原件----- >> 发件人: Shi, Shaofeng [mailto:shao...@ebay.com] >> 发送时间: 2015年12月22日 15:39 >> 收件人: dev@kylin.apache.org >> 主题: Re: about assigning different mr job queue to different user groups >> inside one kylin instances >> >> Thanks Dong; I put a comment in your JIRA: >> https://issues.apache.org/jira/browse/KYLIN-1241, we can discuss there. >> >> On 12/21/15, 10:29 AM, "dong wang" <eloquentb...@gmail.com> wrote: >> >> >I have added the logic ( https://github.com/apache/kylin/pull/17 ) to >> >auto detect the kylin_job_conf_medium/small/large_projectname.xml >> >first, then kylin_job_conf_projectname.xml, then >> >kylin_job_conf_medium/small/large.xml, >> >finally kylin_job_conf.xml, if still not found, throws an exception >> > >> >2015-12-14 16:22 GMT+08:00 yu feng <olaptes...@gmail.com>: >> > >> >> Maybe a quick solution is just create a config file for every project >> >>named ''kylin_job_conf_${projectname}.xml", it will just modify code >> >>that selecting config file for a MR job, and you can control every MR >> >>config property in project level. >> >> >> >> 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <shao...@ebay.com>: >> >> >> >> > This is a valid scenario; So far Kylin doesn¹t have project or cube >> >>level >> >> > job configurations, if it be implemented, you problem will be solved. >> >> > >> >> > On 12/11/15, 7:09 PM, "dong wang" <eloquentb...@gmail.com> wrote: >> >> > >> >> > >Currently, we have different business groups, we want to assign >> >> different >> >> > >mr job queue to different businiess user groups inside ONLY ONE >> >> > >kylin instance, do we have this feature? and as searched, I find >> >> > >the >> >> following >> >> > >piece: >> >> > > >> >> > >public static final String KYLIN_MAP_JOB_QUEUE = >> >>"mapred.job.queue.name >> >> "; >> >> > > >> >> > > >> >> > >If we can pass a parameter to indicate different users to use >> >>different >> >> mr >> >> > >job queue when building their own jobs with updating the codes, a >> >> question >> >> > >is that is there any potential problems to conduct the action? >> >> > >> >> > >> >> >> >> >> >> >> >> >> ******************************************************************************************************************************** >> The information in this email is confidential and may be legally >> privileged. If you have received this email in error or are not the >> intended recipient, please immediately notify the sender and delete this >> message from your computer. Any use, distribution, or copying of this email >> other than by the intended recipient is strictly prohibited. All messages >> sent to and from us may be monitored to ensure compliance with internal >> policies and to protect our business. >> Emails are not secure and cannot be guaranteed to be error free as they >> can be intercepted, amended, lost or destroyed, or contain viruses. Anyone >> who communicates with us by email is taken to accept these risks. >> >> 收发邮件者请注意： >> 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 >> 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 >> >> ******************************************************************************************************************************** >> >> >> >> >> >> ******************************************************************************************************************************** >> The information in this email is confidential and may be legally >> privileged. If you have received this email in error or are not the >> intended recipient, please immediately notify the sender and delete this >> message from your computer. Any use, distribution, or copying of this email >> other than by the intended recipient is strictly prohibited. All messages >> sent to and from us may be monitored to ensure compliance with internal >> policies and to protect our business. >> Emails are not secure and cannot be guaranteed to be error free as they >> can be intercepted, amended, lost or destroyed, or contain viruses. Anyone >> who communicates with us by email is taken to accept these risks. >> >> 收发邮件者请注意： >> 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 >> 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 >> >> ******************************************************************************************************************************** >> >> >> >> >> >> ******************************************************************************************************************************** >> The information in this email is confidential and may be legally >> privileged. If you have received this email in error or are not the >> intended recipient, please immediately notify the sender and delete this >> message from your computer. Any use, distribution, or copying of this email >> other than by the intended recipient is strictly prohibited. All messages >> sent to and from us may be monitored to ensure compliance with internal >> policies and to protect our business. >> Emails are not secure and cannot be guaranteed to be error free as they >> can be intercepted, amended, lost or destroyed, or contain viruses. Anyone >> who communicates with us by email is taken to accept these risks. >> >> 收发邮件者请注意： >> 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 >> 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 >> >> ******************************************************************************************************************************** >> > > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
