dixingxing0 opened a new issue #2198:
URL: https://github.com/apache/iceberg/issues/2198
I got an error while join iceberg table and hive table at same time, most of
mappers succeed, but some mappers got an `cannot find field` error:
```java
Caused by: java.lang.RuntimeException: cannot find field log_src from
[org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector$IcebergRecordStructField@8838736]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:442)
at
org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldRef(IcebergRecordObjectInspector.java:78)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:139)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:980)
at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:1006)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:77)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:355)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:498)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:115)
... 22 more
```
Environment:
```java
Iceberg : 0.11
hive : 2.0.1
hadoop : 2.7.2
JDK : 1.8
```
I found some clues:
```java
2021-02-02 15:57:35,036 INFO [main] org.apache.hadoop.mapred.MapTask:
Processing split:
Paths:/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10005/0/emptyFile:0+0InputFormatClass:
org.apache.hadoop.mapred.TextInputFormat
```
The split file is for `hive_table1`(which is an empty table for test), but
still the mapper using `org.apache.iceberg.mr.hive.HiveIcebergSerDe`, the table
properties is also from `iceberg_table1`.
It seems like hive use wrong serd to deserialize data, I am not an expert in
hive, hope someone could give me some clues .
SQL:
```sql
select count(1),count(a.dt),count(b.dt)
from (
select dt,concat_ws('###',wx_source, log_src, dt, hour) as str
from flink_fdm_iceberg.iceberg_table1
where dt='2021-01-29' and hour='16') a
full outer join (
select dt,concat_ws('###',wx_source, log_src, dt, hour) as str
from flink_fdm_iceberg.hive_table1
where dt='2021-01-29' and hour='16') b on a.str=b.str
where a.str is null or b.str is null;
```
The full log by the failed mapper:
```java
2021-02-02 15:57:30,060 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties2021-02-02 15:57:30,060 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties2021-02-02 15:57:30,118 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at
10 second(s).2021-02-02 15:57:30,118 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
started2021-02-02 15:57:30,131 INFO [main] org.apache.hadoop.mapred.YarnChild:
Executing with tokens:2021-02-02 15:57:30,131 INFO [main]
org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service:
job_1605975810748_1427, Ident:
(org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@4b2bac3f)2021-02-02
15:57:30,167 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind:
HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns2, Ident: (HDFS_DELEGATION_TOKEN
token 151282235 for flink)2021-02
-02 15:57:30,168 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind:
HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns1, Ident: (HDFS_DELEGATION_TOKEN
token 151249944 for flink)2021-02-02 15:57:30,168 INFO [main]
org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service:
ha-hdfs:ns4, Ident: (HDFS_DELEGATION_TOKEN token 151235141 for flink)2021-02-02
15:57:30,169 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind:
HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns3, Ident: (HDFS_DELEGATION_TOKEN
token 151183885 for flink)2021-02-02 15:57:30,169 INFO [main]
org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service:
ha-hdfs:ns6, Ident: (HDFS_DELEGATION_TOKEN token 168039224 for flink)2021-02-02
15:57:30,169 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind:
HDFS_DELEGATION_TOKEN, Service: ha-hdfs:ns5, Ident: (HDFS_DELEGATION_TOKEN
token 150801000 for flink)2021-02-02 15:57:30,881 INFO [main]
org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying agai
n. Got null now.2021-02-02 15:57:31,032 INFO [main]
org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child:
/data1/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data2/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data4/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data5/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data6/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data7/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data8/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data9/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data10/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data11/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427,/data12/nm-local-dir/usercache/f
link/appcache/application_1605975810748_14272021-02-02 15:57:31,937 INFO
[main] org.apache.hadoop.conf.Configuration.deprecation: session.id is
deprecated. Instead, use dfs.metrics.session-id2021-02-02 15:57:34,143 INFO
[main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [
]2021-02-02 15:57:35,036 INFO [main] org.apache.hadoop.mapred.MapTask:
Processing split:
Paths:/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10005/0/emptyFile:0+0InputFormatClass:
org.apache.hadoop.mapred.TextInputFormat
2021-02-02 15:57:35,198 INFO [main]
org.apache.hadoop.hive.ql.exec.Utilities: PLAN PATH =
hdfs://ns6/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10006/33cfad1a-7384-4c27-b2d4-2f894ecb845a/map.xml2021-02-02
15:57:35,209 WARN [main] org.apache.hadoop.ipc.Client: Exception encountered
while connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category READ is not supported in state standby2021-02-02
15:57:35,221 INFO [main] org.apache.hadoop.hive.ql.exec.SerializationUtilities:
Deserializing MapWork using kryo2021-02-02 15:57:35,384 INFO [main]
com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library from
the embedded binaries2021-02-02 15:57:35,386 INFO [main]
com.hadoop.compression.lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev
5e360bdefd8923280b1a0234b845448050bf0caa]2021-02-02 15:57:35,857 INFO [main] or
g.apache.hadoop.conf.Configuration.deprecation: map.input.file is deprecated.
Instead, use mapreduce.map.input.file2021-02-02 15:57:35,857 INFO [main]
org.apache.hadoop.conf.Configuration.deprecation: map.input.start is
deprecated. Instead, use mapreduce.map.input.start2021-02-02 15:57:35,857 INFO
[main] org.apache.hadoop.conf.Configuration.deprecation: map.input.length is
deprecated. Instead, use mapreduce.map.input.length2021-02-02 15:57:35,857 INFO
[main] org.apache.hadoop.mapred.MapTask: numReduceTasks: 10092021-02-02
15:57:37,109 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi
268435452(1073741808)2021-02-02 15:57:37,110 INFO [main]
org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 10242021-02-02
15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at
8589934722021-02-02 15:57:37,110 INFO [main] org.apache.hadoop.mapred.MapTask:
bufstart = 0; bufvoid = 10737418242021-02-02 15:57:37,110 INFO [main]
org.apache.hadoop.mapred.MapTask: kvsta
rt = 268435452; length = 671088642021-02-02 15:57:37,114 INFO [main]
org.apache.hadoop.mapred.MapTask: Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer2021-02-02 15:57:37,119 INFO
[main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: conf classpath =
[file:/data12/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/container_e03_1605975810748_1427_01_000559/,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/etc/hadoop/,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-nfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-lzo-0.4.20-SNAPSHOT.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/mysql-connector-java-5.1.31-bin.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/spark-2.0.2-yarn-shuffle.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/comm
on/data-hadoop-patch.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-api-2.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-client-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/junit-4.11.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/avro-1.7.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jets3t-0.9.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-httpclie
nt-3.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-math3-3.1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-logging-1.1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpclient-4.2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/mockito-all-1.8.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hamcrest-core-1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-recipes-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-collections-3.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/ha
doop/common/lib/htrace-core-3.1.0-incubating.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpcore-4.2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsp-api-2.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-json-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/zookeeper-3.4.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-auth-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/had
oop/common/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-api-1.7.10.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xmlenc-0.52.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jettison-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-util-1.0.0-M20.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/snappy-java-1.0.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsch-0.1.42.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common
/lib/stax-api-1.0-2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-xc-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/paranamer-2.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-configuration-1.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/gson-2.2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-1.7.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yar
n/share/hadoop/common/lib/java-xmlbuilder-0.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-net-3.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-digester-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-framework-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/activation-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7
.2-yarn/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/ser
vers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xmlenc-0.52.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/sh
are/hadoop/hdfs/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-client-2.7.2.
jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-api-2.2.2.
jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-logging-1.1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-servlet-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-collections-3.2.2.jar,
file:
/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/aopalliance-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-json-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-client-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jettison-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/sha
re/hadoop/yarn/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/stax-api-1.0-2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/javax.inject-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-guice-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.j
ar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/activation-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-
client-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/junit-4.11.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/avro-1.7.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar,
file
:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/aopalliance-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/paranamer-2.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/javax.
inject-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/job.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/classes,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/lib/*,
file:/data2/nm-local-dir/usercache/flink/filecache/299/UserAgentUtils-1.19.jar,
file:/data9/nm-local-dir/usercache/flink/filecache/294/iceberg-hive-runtime-022dc46.dirty.jar,
file:/data7/nm-local-dir/usercache/flink/filecache/292/hive-jdbc-1.2.1.jar,
file:/data12/nm-local-di
r/usercache/flink/filecache/297/dc-ua-util-2.0.0-SNAPSHOT.jar,
file:/data11/nm-local-dir/usercache/flink/filecache/296/check-hive-db.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/,
file:/data10/nm-local-dir/usercache/flink/filecache/295/data-udf-0.0.1.jar,
file:/data6/nm-local-dir/usercache/flink/filecache/291/sqljdbc4-4.0.jar]2021-02-02
15:57:37,121 INFO [main] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: thread
classpath =
[file:/data12/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/container_e03_1605975810748_1427_01_000559/,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/etc/hadoop/,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-nfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-lzo-0.4.20-SNAPSHOT.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/mysql-connector-java-5.1.31-bin.jar,
file:/data/sysdir/servers/hadoop-2.7.2
-yarn/share/hadoop/common/spark-2.0.2-yarn-shuffle.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/data-hadoop-patch.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/hadoop-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-api-2.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-client-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/junit-4.11.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yar
n/share/hadoop/common/lib/avro-1.7.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jets3t-0.9.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-httpclient-3.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-math3-3.1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-logging-1.1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpclient-4.2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/mockito-all-1.8.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hamcrest-core-1.3.jar,
file:/data/sysdir/servers/hadoop-2
.7.2-yarn/share/hadoop/common/lib/curator-recipes-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-collections-3.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/httpcore-4.2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsp-api-2.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-json-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/zookeeper-3.4.6.jar,
file:/data/sysdir/ser
vers/hadoop-2.7.2-yarn/share/hadoop/common/lib/hadoop-auth-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-api-1.7.10.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xmlenc-0.52.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jettison-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-util-1.0.0-M20.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/snappy-java-1.0.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar,
file:/data/sysdir/servers/hadoop-2.7.2
-yarn/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsch-0.1.42.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/stax-api-1.0-2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-xc-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/paranamer-2.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-configuration-1.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/gson-2.2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2
.7.2-yarn/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-beanutils-1.7.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/java-xmlbuilder-0.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-net-3.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-digester-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/curator-framework-2.7.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/activation-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/common/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-logging-1.1.3.
jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jersey-core-1.9.jar,
file:
/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/xmlenc-0.52.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/hdfs/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2
.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop
/yarn/hadoop-yarn-server-tests-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jetty-util-6.1.26.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/servlet-api-2.5.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-logging-1.1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-
core-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guice-servlet-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-collections-3.2.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/guava-11.0.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/aopalliance-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-json-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-client-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/zookeeper-3.4.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-cli-1.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-core-1.9.jar,
file:/d
ata/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-lang-2.6.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jettison-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/stax-api-1.0-2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/javax.inject-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jsr305-3.0.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jersey-guice-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/ha
doop/yarn/lib/zookeeper-3.4.6-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/activation-1.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/yarn/lib/commons-codec-1.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapredu
ce-examples-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/junit-4.11.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/avro-1.7.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-c
ore-asl-1.9.13.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-server-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/aopalliance-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-io-2.4.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-core-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/asm-3.2.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/paranamer-2.3.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share
/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/log4j-1.2.17.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/javax.inject-1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/xz-1.0.jar,
file:/data/sysdir/servers/hadoop-2.7.2-yarn/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/job.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/classes,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/lib/*,
file:/data2/nm-local-dir/usercache/flink/filecache/299/UserAgen
tUtils-1.19.jar,
file:/data9/nm-local-dir/usercache/flink/filecache/294/iceberg-hive-runtime-022dc46.dirty.jar,
file:/data7/nm-local-dir/usercache/flink/filecache/292/hive-jdbc-1.2.1.jar,
file:/data12/nm-local-dir/usercache/flink/filecache/297/dc-ua-util-2.0.0-SNAPSHOT.jar,
file:/data11/nm-local-dir/usercache/flink/filecache/296/check-hive-db.jar,
file:/data3/nm-local-dir/usercache/flink/appcache/application_1605975810748_1427/filecache/10/job.jar/,
file:/data10/nm-local-dir/usercache/flink/filecache/295/data-udf-0.0.1.jar,
file:/data6/nm-local-dir/usercache/flink/filecache/291/sqljdbc4-4.0.jar]2021-02-02
15:57:37,121 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: PLAN PATH =
hdfs://ns6/tmp/hive/flink/25369d0b-110e-4eca-8553-f2f3953f0303/hive_2021-02-02_15-56-22_060_2459779186626852210-1/-mr-10006/33cfad1a-7384-4c27-b2d4-2f894ecb845a/map.xml2021-02-02
15:57:37,123 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator:
Initializing operator MAP[0]2021-02-02 15:57:37,128 INFO
[main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: serDeProperties :
{columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string,
location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1,
columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_refer,visits_ref,visits_keyword,visits_ar
ea_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour,
metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json,
previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json,
numFiles=59177, serialization.ddl=struct icebert_table1 { string version,
string lo
ad_id, string pre_load_id, string sign, string visit_time, string ip, string
province_id, string city_id, string county_id, string appkey, string
session_id, string user_id, string ua, string visits_id, string
first_visit_time, string first_visit_load_id, string first_visit_ref_domain,
string first_visit_subdomain_time, string first_visit_subdomain_load_id, string
first_visit_subdomain_refer, string visits_ref, string visits_keyword, string
visits_area_id, string bt_visits_ref, string last_visit_subdomain_refer, string
site_ref, string scene_type, string show_id, string ref, string refer_domain,
string key_word, string area_id, string current_url, string current_domain,
string site_id, string category_id, string sub_category_id, string object_id,
string series_id, string spec_id, string dealer_id, string level_id, string
doc_title, string page_custom_vars, string platform, string screen, string
color_depth, string browser_charset, string browser_language, string
browser_cookie_enabl
e, string extend, string create_time, string id, string custom1, string
custom2, string customer3, string customer4, string longitude, string latitude,
string browse_type, string ac_userid, string ac_app_key, string
ac_app_platform, string ac_app_ver, string ac_app_deviceid, string
cur_visit_ref_domain, string cookie_content, string pre_visits_ref, string
lt_visits_ref, string abtest, string wx_openid, string wx_source, string
log_src, string dt, string hour}, transient_lastDdlTime=1611904166,
iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","requi
red":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visit
s_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"strin
g"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"
type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"nam
e":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]},
columns.comments=, totalSize=302458258710, bucket_count=0,
file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat,
serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe,
storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler,
table_type=ICEBERG,
file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat,
EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,906
INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from
serDeProperties
{"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type"
:"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","require
d":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":
false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_
language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name
":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02
15:57:37,907 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe:
selectedColumns 1, [wx_source]2021-02-02 15:57:37,926 INFO [main]
org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema
{"type":"struct","fields":[{"id":72,"name":"wx_source","required":false,"type":"string"}]}2021-02-02
15:57:37,969 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe:
serDeProperties :
{columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string,
location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1,
columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_refer,visits_ref,visit
s_keyword,visits_area_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour,
metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json,
previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json,
numFiles=59177, serialization.ddl=struct icebert_table1 { string
version, string load_id, string pre_load_id, string sign, string visit_time,
string ip, string province_id, string city_id, string county_id, string appkey,
string session_id, string user_id, string ua, string visits_id, string
first_visit_time, string first_visit_load_id, string first_visit_ref_domain,
string first_visit_subdomain_time, string first_visit_subdomain_load_id, string
first_visit_subdomain_refer, string visits_ref, string visits_keyword, string
visits_area_id, string bt_visits_ref, string last_visit_subdomain_refer, string
site_ref, string scene_type, string show_id, string ref, string refer_domain,
string key_word, string area_id, string current_url, string current_domain,
string site_id, string category_id, string sub_category_id, string object_id,
string series_id, string spec_id, string dealer_id, string level_id, string
doc_title, string page_custom_vars, string platform, string screen, string
color_depth, string browser_charset, string browser_language, string b
rowser_cookie_enable, string extend, string create_time, string id, string
custom1, string custom2, string customer3, string customer4, string longitude,
string latitude, string browse_type, string ac_userid, string ac_app_key,
string ac_app_platform, string ac_app_ver, string ac_app_deviceid, string
cur_visit_ref_domain, string cookie_content, string pre_visits_ref, string
lt_visits_ref, string abtest, string wx_openid, string wx_source, string
log_src, string dt, string hour}, transient_lastDdlTime=1611904166,
iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"nam
e":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"i
d":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":
false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1"
,"required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"str
ing"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]},
columns.comments=, totalSize=302458258710, bucket_count=0,
file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat,
serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe,
storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler,
table_type=ICEBERG,
file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat,
EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,973
INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from
serDeProperties
{"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","req
uired":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdo
main_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"curre
nt_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":
49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"stri
ng"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02
15:57:37,974 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe:
selectedColumns 1, [wx_source]2021-02-02 15:57:37,974 INFO [main]
org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema
{"type":"struct","fields":[{"id":72,"name":"wx_source","required":false,"type":"s
tring"}]}2021-02-02 15:57:37,998 INFO [main]
org.apache.iceberg.mr.hive.HiveIcebergSerDe: serDeProperties :
{columns.types=string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string:string,
location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1,
columns=version,load_id,pre_load_id,sign,visit_time,ip,province_id,city_id,county_id,appkey,session_id,user_id,ua,visits_id,first_visit_time,first_visit_load_id,first_visit_ref_domain,first_visit_subdomain_time,first_visit_subdomain_load_id,first_visit_subdomain_ref
er,visits_ref,visits_keyword,visits_area_id,bt_visits_ref,last_visit_subdomain_refer,site_ref,scene_type,show_id,ref,refer_domain,key_word,area_id,current_url,current_domain,site_id,category_id,sub_category_id,object_id,series_id,spec_id,dealer_id,level_id,doc_title,page_custom_vars,platform,screen,color_depth,browser_charset,browser_language,browser_cookie_enable,extend,create_time,id,custom1,custom2,customer3,customer4,longitude,latitude,browse_type,ac_userid,ac_app_key,ac_app_platform,ac_app_ver,ac_app_deviceid,cur_visit_ref_domain,cookie_content,pre_visits_ref,lt_visits_ref,abtest,wx_openid,wx_source,log_src,dt,hour,
metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00753-b2322cf0-ac32-4eb0-957e-95d2189e0272.metadata.json,
previous_metadata_location=viewfs://AutoLfCluster/team/db/hive_db/flink_fdm_iceberg/icebert_table1/metadata/00752-6949a171-403e-4853-93a3-666df4305603.metadata.json,
numFiles=59177, serialization.ddl=struct iceb
ert_table1 { string version, string load_id, string pre_load_id, string sign,
string visit_time, string ip, string province_id, string city_id, string
county_id, string appkey, string session_id, string user_id, string ua, string
visits_id, string first_visit_time, string first_visit_load_id, string
first_visit_ref_domain, string first_visit_subdomain_time, string
first_visit_subdomain_load_id, string first_visit_subdomain_refer, string
visits_ref, string visits_keyword, string visits_area_id, string bt_visits_ref,
string last_visit_subdomain_refer, string site_ref, string scene_type, string
show_id, string ref, string refer_domain, string key_word, string area_id,
string current_url, string current_domain, string site_id, string category_id,
string sub_category_id, string object_id, string series_id, string spec_id,
string dealer_id, string level_id, string doc_title, string page_custom_vars,
string platform, string screen, string color_depth, string browser_charset,
string browser
_language, string browser_cookie_enable, string extend, string create_time,
string id, string custom1, string custom2, string customer3, string customer4,
string longitude, string latitude, string browse_type, string ac_userid, string
ac_app_key, string ac_app_platform, string ac_app_ver, string ac_app_deviceid,
string cur_visit_ref_domain, string cookie_content, string pre_visits_ref,
string lt_visits_ref, string abtest, string wx_openid, string wx_source, string
log_src, string dt, string hour}, transient_lastDdlTime=1611904166,
iceberg.mr.table.schema={"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"name":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"st
ring"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name":"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"
type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"id":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"obj
ect_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"type":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":
54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required":false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required
":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]},
columns.comments=, totalSize=302458258710, bucket_count=0,
file.outputformat=org.apache.iceberg.mr.hive.HiveIcebergOutputFormat,
serialization.lib=org.apache.iceberg.mr.hive.HiveIcebergSerDe,
storage_handler=org.apache.iceberg.mr.hive.HiveIcebergStorageHandler,
table_type=ICEBERG,
file.inputformat=org.apache.iceberg.mr.hive.HiveIcebergInputFormat,
EXTERNAL=TRUE, name=flink_fdm_iceberg.icebert_table1}2021-02-02 15:57:37,999
INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using schema from
serDeProperties
{"type":"struct","fields":[{"id":1,"name":"version","required":false,"type":"string"},{"id":2,"n
ame":"load_id","required":false,"type":"string"},{"id":3,"name":"pre_load_id","required":false,"type":"string"},{"id":4,"name":"sign","required":false,"type":"string"},{"id":5,"name":"visit_time","required":false,"type":"string"},{"id":6,"name":"ip","required":false,"type":"string"},{"id":7,"name":"province_id","required":false,"type":"string"},{"id":8,"name":"city_id","required":false,"type":"string"},{"id":9,"name":"county_id","required":false,"type":"string"},{"id":10,"name":"appkey","required":false,"type":"string"},{"id":11,"name":"session_id","required":false,"type":"string"},{"id":12,"name":"user_id","required":false,"type":"string"},{"id":13,"name":"ua","required":false,"type":"string"},{"id":14,"name":"visits_id","required":false,"type":"string"},{"id":15,"name":"first_visit_time","required":false,"type":"string"},{"id":16,"name":"first_visit_load_id","required":false,"type":"string"},{"id":17,"name":"first_visit_ref_domain","required":false,"type":"string"},{"id":18,"name"
:"first_visit_subdomain_time","required":false,"type":"string"},{"id":19,"name":"first_visit_subdomain_load_id","required":false,"type":"string"},{"id":20,"name":"first_visit_subdomain_refer","required":false,"type":"string"},{"id":21,"name":"visits_ref","required":false,"type":"string"},{"id":22,"name":"visits_keyword","required":false,"type":"string"},{"id":23,"name":"visits_area_id","required":false,"type":"string"},{"id":24,"name":"bt_visits_ref","required":false,"type":"string"},{"id":25,"name":"last_visit_subdomain_refer","required":false,"type":"string"},{"id":26,"name":"site_ref","required":false,"type":"string"},{"id":27,"name":"scene_type","required":false,"type":"string"},{"id":28,"name":"show_id","required":false,"type":"string"},{"id":29,"name":"ref","required":false,"type":"string"},{"id":30,"name":"refer_domain","required":false,"type":"string"},{"id":31,"name":"key_word","required":false,"type":"string"},{"id":32,"name":"area_id","required":false,"type":"string"},{"i
d":33,"name":"current_url","required":false,"type":"string"},{"id":34,"name":"current_domain","required":false,"type":"string"},{"id":35,"name":"site_id","required":false,"type":"string"},{"id":36,"name":"category_id","required":false,"type":"string"},{"id":37,"name":"sub_category_id","required":false,"type":"string"},{"id":38,"name":"object_id","required":false,"type":"string"},{"id":39,"name":"series_id","required":false,"type":"string"},{"id":40,"name":"spec_id","required":false,"type":"string"},{"id":41,"name":"dealer_id","required":false,"type":"string"},{"id":42,"name":"level_id","required":false,"type":"string"},{"id":43,"name":"doc_title","required":false,"type":"string"},{"id":44,"name":"page_custom_vars","required":false,"type":"string"},{"id":45,"name":"platform","required":false,"type":"string"},{"id":46,"name":"screen","required":false,"type":"string"},{"id":47,"name":"color_depth","required":false,"type":"string"},{"id":48,"name":"browser_charset","required":false,"typ
e":"string"},{"id":49,"name":"browser_language","required":false,"type":"string"},{"id":50,"name":"browser_cookie_enable","required":false,"type":"string"},{"id":51,"name":"extend","required":false,"type":"string"},{"id":52,"name":"create_time","required":false,"type":"string"},{"id":53,"name":"id","required":false,"type":"string"},{"id":54,"name":"custom1","required":false,"type":"string"},{"id":55,"name":"custom2","required":false,"type":"string"},{"id":56,"name":"customer3","required":false,"type":"string"},{"id":57,"name":"customer4","required":false,"type":"string"},{"id":58,"name":"longitude","required":false,"type":"string"},{"id":59,"name":"latitude","required":false,"type":"string"},{"id":60,"name":"browse_type","required":false,"type":"string"},{"id":61,"name":"ac_userid","required":false,"type":"string"},{"id":62,"name":"ac_app_key","required":false,"type":"string"},{"id":63,"name":"ac_app_platform","required":false,"type":"string"},{"id":64,"name":"ac_app_ver","required"
:false,"type":"string"},{"id":65,"name":"ac_app_deviceid","required":false,"type":"string"},{"id":66,"name":"cur_visit_ref_domain","required":false,"type":"string"},{"id":67,"name":"cookie_content","required":false,"type":"string"},{"id":68,"name":"pre_visits_ref","required":false,"type":"string"},{"id":69,"name":"lt_visits_ref","required":false,"type":"string"},{"id":70,"name":"abtest","required":false,"type":"string"},{"id":71,"name":"wx_openid","required":false,"type":"string"},{"id":72,"name":"wx_source","required":false,"type":"string"},{"id":73,"name":"log_src","required":false,"type":"string"},{"id":74,"name":"dt","required":false,"type":"string"},{"id":75,"name":"hour","required":false,"type":"string"}]}2021-02-02
15:57:37,999 INFO [main] org.apache.iceberg.mr.hive.HiveIcebergSerDe:
selectedColumns 1, [wx_source]2021-02-02 15:57:38,000 INFO [main]
org.apache.iceberg.mr.hive.HiveIcebergSerDe: Using projectedSchema
{"type":"struct","fields":[{"id":72,"name":"wx_source","requir
ed":false,"type":"string"}]}2021-02-02 15:57:38,005 INFO [main]
org.apache.hadoop.hive.ql.exec.mr.ExecMapper: <MAP>Id =0 <Children>null
<TS>Id =3 <Children> <FIL>Id =18 <Children>
<SEL>Id =5 <Children> <RS>Id =7
<Children> <\Children> <Parent>Id = 5
null<\Parent> <\RS> <\Children>
<Parent>Id = 18 null<\Parent> <\SEL> <\Children>
<Parent>Id = 3 null<\Parent> <\FIL> <\Children> <Parent>Id = 0
null<\Parent> <\TS> <\Children> <Parent><\Parent><\MAP>2021-02-02
15:57:38,007 INFO [main] org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing operator TS[3]2021-02-02 15:57:38,008 INFO [main]
org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing operator
FIL[18]2021-02-02 15:57:43,031 INFO [main]
org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing operator SEL[5]2021-
02-02 15:57:43,031 INFO [main] org.apache.hadoop.hive.ql.exec.SelectOperator:
SELECT
struct<version:string,load_id:string,pre_load_id:string,sign:string,visit_time:string,ip:string,province_id:int,city_id:int,county_id:string,appkey:string,session_id:string,user_id:int,ua:string,visits_id:string,first_visit_time:string,first_visit_load_id:string,first_visit_ref_domain:string,first_visit_subdomain_time:string,first_visit_subdomain_load_id:string,first_visit_subdomain_refer:string,visits_ref:string,visits_keyword:string,visits_area_id:string,bt_visits_ref:string,last_visit_subdomain_refer:string,site_ref:string,scene_type:string,show_id:string,ref:string,refer_domain:string,key_word:string,area_id:string,current_url:string,current_domain:string,site_id:int,category_id:int,sub_category_id:int,object_id:int,series_id:int,spec_id:int,dealer_id:int,level_id:int,doc_title:string,page_custom_vars:string,platform:int,screen:string,color_depth:string,browser_charset:string,browser_language:st
ring,browser_cookie_enable:string,extend:string,create_time:string,id:string,custom1:int,custom2:int,customer3:string,customer4:string,longitude:float,latitude:float,browse_type:string,ac_userid:string,ac_app_key:string,ac_app_platform:string,ac_app_ver:string,ac_app_deviceid:string,cur_visit_ref_domain:string,cookie_content:string,pre_visits_ref:string,lt_visits_ref:string,abtest:string,wx_openid:string,wx_source:string,log_src:string,dt:string,hour:string>2021-02-02
15:57:43,033 INFO [main] org.apache.hadoop.hive.ql.exec.ReduceSinkOperator:
Initializing operator RS[7]2021-02-02 15:57:43,036 INFO [main]
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator: Using tag = 12021-02-02
15:57:43,050 INFO [main] org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing operator TS[0]2021-02-02 15:57:43,050 INFO [main]
org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing operator
FIL[17]2021-02-02 15:57:43,051 INFO [main]
org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing
operator SEL[2]2021-02-02 15:57:43,051 INFO [main]
org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT
struct<wx_source:string>2021-02-02 15:57:43,052 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.RuntimeException: Error in configuring object at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)Caused by:
java.lang.refl
ect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) ...
9 moreCaused by: java.lang.RuntimeException: Error in configuring object at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) ... 14
moreCaused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.
DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) ...
17 moreCaused by: java.lang.RuntimeException: Map operator initialization
failed at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:136) ...
22 moreCaused by: java.lang.RuntimeException: cannot find field log_src from
[org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector$IcebergRecordStructField@8838736]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:442)
at
org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldRef(IcebergRecordObjectInspector.java:78)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initi
alize(ExprNodeGenericFuncEvaluator.java:139) at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:980) at
org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:1006)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:77)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:355) at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504) at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365) at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504) at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365) at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:498)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:115)
... 22 more
2021-02-02 15:57:43,053 INFO [main] org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
