在build cube时，报
java.lang.NoSuchMethodError:
com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
错误，看起来像是版本问题，请问有人遇到吗？应该怎样解决？
版本：2.5.1-cdh47
Exception in thread "main" java.lang.NoSuchMethodError:
com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
at
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
at
com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:538)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:105)
at com.sun.proxy.$Proxy19.close(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2084)
at com.sun.proxy.$Proxy19.close(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:357)
at org.apache.hadoop.hive.ql.metadata.Hive.access$000(Hive.java:153)
at org.apache.hadoop.hive.ql.metadata.Hive$1.remove(Hive.java:173)
at org.apache.hadoop.hive.ql.metadata.Hive.closeCurrent(Hive.java:326)
at
org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1643)
at
org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:701)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
谢谢
—郭鹏飞
完整日志如下：
java.io.IOException: OS command error exit with return code: 1, error message:
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M;
support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated
and will likely be removed in a future release
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/apache-kylin-2.5.1-bin-cdh57/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M;
support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/logging-assembly-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive/lib/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Logging initialized using configuration in
jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-common-1.1.0-cdh5.12.1.jar!/hive-log4j.properties
OK
Time taken: 2.905 seconds
OK
Time taken: 0.099 seconds
OK
Time taken: 0.584 seconds
OK
Time taken: 0.151 seconds
Query ID = olap_20181113192727_70d7dc6c-f4ad-4033-aadf-e10e3ece920e
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-exec-1.1.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hive-jdbc-1.1.0-cdh5.12.1-standalone.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-hadoop-bundle-1.5.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-pig-bundle-1.5.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/parquet-format-2.1.0-cdh5.12.1.jar!/shaded/parquet/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type
[shaded.parquet.org.slf4j.helpers.NOPLoggerFactory]
Starting Job = job_1541920145982_4712, Tracking URL =
http://c3-d07-136-38-2.yidian.com:8088/proxy/application_1541920145982_4712/
Kill Command =
/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/bin/../lib/hadoop/bin/hadoop
job -kill job_1541920145982_4712
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2018-11-13 19:27:25,312 Stage-1 map = 0%, reduce = 0%
2018-11-13 19:27:31,921 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.73
sec
MapReduce Total cumulative CPU time: 2 seconds 730 msec
Ended Job = job_1541920145982_4712
Loading data to table
default.kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
Table
default.kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
stats: [numFiles=1, numRows=0, totalSize=130, rawDataSize=0]
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 2.73 sec HDFS Read: 10336 HDFS Write:
235 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 730 msec
OK
Time taken: 17.454 seconds
WARN: The method class
org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
Exception in thread "main" java.lang.NoSuchMethodError:
com.facebook.fb303.FacebookService$Client.sendBaseOneway(Ljava/lang/String;Lorg/apache/thrift/TBase;)V
at
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:436)
at
com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:430)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:538)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:105)
at com.sun.proxy.$Proxy19.close(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2084)
at com.sun.proxy.$Proxy19.close(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.close(Hive.java:357)
at org.apache.hadoop.hive.ql.metadata.Hive.access$000(Hive.java:153)
at org.apache.hadoop.hive.ql.metadata.Hive$1.remove(Hive.java:173)
at org.apache.hadoop.hive.ql.metadata.Hive.closeCurrent(Hive.java:326)
at
org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1643)
at
org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:701)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.codec.CodecConfig: Compression set
to false
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet block
size to 134217728
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet page
size to 1048576
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Parquet
dictionary page size to 1048576
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Dictionary is
on
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Validation is
off
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Writer
version is: PARQUET_1_0
Nov 13, 2018 7:27:16 PM INFO: parquet.hadoop.ParquetOutputFormat: Maximum row
group padding size is 8388608 bytes
Nov 13, 2018 7:27:17 PM INFO: parquet.hadoop.InternalParquetRecordWriter:
Flushing mem columnStore to file. allocated memory: 0
The command is:
hive -e "USE default;
DROP TABLE IF EXISTS
kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06;
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06
(
ETL_ODW_EVENT_PLATFORM string
,ETL_ODW_EVENT_APP_ID string
,ETL_ODW_EVENT_MASH_TYPE string
,ETL_ODW_EVENT_FACTOR string
,ETL_ODW_EVENT_BATCH string
,ETL_ODW_EVENT_POSITION string
,ETL_ODW_EVENT_PAGE_TYPE string
,ETL_ODW_EVENT_ACTION_METHOD string
,ETL_ODW_EVENT_ACTION_SRC string
,ETL_ODW_EVENT_P_DAY string
,ETL_ODW_EVENT_P_HOUR string
,ETL_ODW_EVENT_P_TYPE string
,ETL_ODW_EVENT_DWELL bigint
,ETL_ODW_EVENT_DAU_UID string
,ETL_ODW_EVENT_CLICK_DOC_UID string
)
STORED AS SEQUENCEFILE
LOCATION
'hdfs://yellow/data/kylin/kylin_metadata/kylin-b4b541bb-4abf-c44c-29ed-664b7655e67c/kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06';
ALTER TABLE
kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06 SET
TBLPROPERTIES('auto.purge'='true');
INSERT OVERWRITE TABLE
kylin_intermediate_etl_odw_event_fe68595d_86f2_3f32_b068_51390782dd06 SELECT
ETL_ODW_EVENT.PLATFORM as ETL_ODW_EVENT_PLATFORM
,ETL_ODW_EVENT.APP_ID as ETL_ODW_EVENT_APP_ID
,ETL_ODW_EVENT.MASH_TYPE as ETL_ODW_EVENT_MASH_TYPE
,ETL_ODW_EVENT.FACTOR as ETL_ODW_EVENT_FACTOR
,ETL_ODW_EVENT.BATCH as ETL_ODW_EVENT_BATCH
,ETL_ODW_EVENT.POSITION as ETL_ODW_EVENT_POSITION
,ETL_ODW_EVENT.PAGE_TYPE as ETL_ODW_EVENT_PAGE_TYPE
,ETL_ODW_EVENT.ACTION_METHOD as ETL_ODW_EVENT_ACTION_METHOD
,ETL_ODW_EVENT.ACTION_SRC as ETL_ODW_EVENT_ACTION_SRC
,ETL_ODW_EVENT.P_DAY as ETL_ODW_EVENT_P_DAY
,ETL_ODW_EVENT.P_HOUR as ETL_ODW_EVENT_P_HOUR
,ETL_ODW_EVENT.P_TYPE as ETL_ODW_EVENT_P_TYPE
,ETL_ODW_EVENT.DWELL as ETL_ODW_EVENT_DWELL
,ETL_ODW_EVENT.DAU_UID as ETL_ODW_EVENT_DAU_UID
,ETL_ODW_EVENT.CLICK_DOC_UID as ETL_ODW_EVENT_CLICK_DOC_UID
FROM WORKING_DW.ETL_ODW_EVENT as ETL_ODW_EVENT
WHERE 1=1 AND (ETL_ODW_EVENT.P_DAY >= '2018-10-25' AND ETL_ODW_EVENT.P_DAY <
'2018-10-25')
;
" --hiveconf hive.merge.mapredfiles=false --hiveconf
hive.auto.convert.join=true --hiveconf dfs.replication=2 --hiveconf
hive.exec.compress.output=true --hiveconf
hive.auto.convert.join.noconditionaltask=true --hiveconf
mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false
--hiveconf hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf
hive.stats.autogather=true
at
org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:95)
at
org.apache.kylin.source.hive.CreateFlatHiveTableStep.createFlatHiveTable(CreateFlatHiveTableStep.java:62)
at
org.apache.kylin.source.hive.CreateFlatHiveTableStep.doWork(CreateFlatHiveTableStep.java:99)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:70)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:164)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
