Hi,
I think there could be a hadoop version mismatch in your binary package. If you
use
pre-built binary package of hive and hbase, maybe the default hadoop dependency
is not as latest as your current use 2.7.1 , thus may lead to the
IncompatibleClassChangeError.
One possible workaround is to recompile and package your hive and hbase from
souce code with
targeting hadoop version to 2.7.1.
250635...@qq.com
From: 胡志华(万里通科技及数据中心商务智能团队数据分析组)
Date: 2015-12-22 15:54
To: 'dev@kylin.apache.org'
Subject: building cube , stop at step 2,help
Hi,
I downloaded “apache-kylin-1.1.1-incubating-bin”, and use sample.sh to
build a cube “kylin_sales_cube”, when I started the job , it stoped at step 2,
with error
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at
org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:120)
at
org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:83)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
My environment lists as following
Hadoop2.7.1
Hbase0.98.13
Hive0.13.1
So ,from the error code above, my hive environment is incompatible? And
what does the parameter mean at step 2?
-conf /home/hadoop/apache-kylin-1.1.1-incubating-bin/conf/kylin_job_conf.xml
-cubename kylin_sales_cube -output
/kylin/kylin_metadata/kylin-79ec4e09-9128-42bd-a455-9dcf3c16a42a/kylin_sales_cube/fact_distinct_columns
-jobname Kylin_Fact_Distinct_Columns_kylin_sales_cube_Step -tablename
default.kylin_intermediate_kylin_sales_cube_desc_19700101000000_20151217000000_79ec4e09_9128_42bd_a455_9dcf3c16a42a
-----邮件原件-----
发件人: Shi, Shaofeng [mailto:shao...@ebay.com]
发送时间: 2015年12月22日 15:39
收件人: dev@kylin.apache.org
主题: Re: about assigning different mr job queue to different user groups inside
one kylin instances
Thanks Dong; I put a comment in your JIRA:
https://issues.apache.org/jira/browse/KYLIN-1241, we can discuss there.
On 12/21/15, 10:29 AM, "dong wang" <eloquentb...@gmail.com> wrote:
>I have added the logic ( https://github.com/apache/kylin/pull/17 ) to
>auto detect the kylin_job_conf_medium/small/large_projectname.xml
>first, then kylin_job_conf_projectname.xml, then
>kylin_job_conf_medium/small/large.xml,
>finally kylin_job_conf.xml, if still not found, throws an exception
>
>2015-12-14 16:22 GMT+08:00 yu feng <olaptes...@gmail.com>:
>
>> Maybe a quick solution is just create a config file for every project
>>named ''kylin_job_conf_${projectname}.xml", it will just modify code
>>that selecting config file for a MR job, and you can control every MR
>>config property in project level.
>>
>> 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng <shao...@ebay.com>:
>>
>> > This is a valid scenario; So far Kylin doesn¹t have project or cube
>>level
>> > job configurations, if it be implemented, you problem will be solved.
>> >
>> > On 12/11/15, 7:09 PM, "dong wang" <eloquentb...@gmail.com> wrote:
>> >
>> > >Currently, we have different business groups, we want to assign
>> different
>> > >mr job queue to different businiess user groups inside ONLY ONE
>> > >kylin instance, do we have this feature? and as searched, I find
>> > >the
>> following
>> > >piece:
>> > >
>> > >public static final String KYLIN_MAP_JOB_QUEUE =
>>"mapred.job.queue.name
>> ";
>> > >
>> > >
>> > >If we can pass a parameter to indicate different users to use
>>different
>> mr
>> > >job queue when building their own jobs with updating the codes, a
>> question
>> > >is that is there any potential problems to conduct the action?
>> >
>> >
>>
********************************************************************************************************************************
The information in this email is confidential and may be legally privileged. If
you have received this email in error or are not the intended recipient, please
immediately notify the sender and delete this message from your computer. Any
use, distribution, or copying of this email other than by the intended
recipient is strictly prohibited. All messages sent to and from us may be
monitored to ensure compliance with internal policies and to protect our
business.
Emails are not secure and cannot be guaranteed to be error free as they can be
intercepted, amended, lost or destroyed, or contain viruses. Anyone who
communicates with us by email is taken to accept these risks.
收发邮件者请注意：
本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。
进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。
********************************************************************************************************************************
