ycjunhua commented on issue #3704:
URL: https://github.com/apache/hudi/issues/3704#issuecomment-925466818
10:34:05,441 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- hoodie_stream_write (3/4) (ff5491c88ffc17a816e22c29b6da47be) switched
from DEPLOYING to RUNNING.
10:34:05,441 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- hoodie_stream_write (1/4) (888768308836b8f4831890fa2934fe2c) switched
from DEPLOYING to RUNNING.
10:34:05,443 INFO org.apache.flink.runtime.taskmanager.Task
- hoodie_stream_write (4/4)#0 (5e2085d96585101e586a7ea47bc56a98) switched
from CREATED to DEPLOYING.
10:34:05,444 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task hoodie_stream_write (4/4)#0
(5e2085d96585101e586a7ea47bc56a98) [DEPLOYING].
10:34:05,444 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
bc89ec13c8cfe5f30f33b9b92894ad6d.
10:34:05,444 WARN org.apache.flink.metrics.MetricGroup
- The operator name Source: Values(tuples=[[{ 1, _UTF-16LE'1',
_UTF-16LE'中国', _UTF-16LE'0', 1, _UTF-16LE'中国', _UTF-16LE'sap_area',
_UTF-16LE'', _UTF-16LE'ENABLE', _UTF-16LE'zhangsan', 2021-09-18 16:48:00,
_UTF-16LE'ayj', 2021-09-18 16:48:00, 0 }]]) exceeded the 80 characters length
limit and was truncated.
10:34:05,444 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
764034c3267a443562b693e46aad20e5.
10:34:05,446 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: hoodie_stream_write (4/4)#0
(5e2085d96585101e586a7ea47bc56a98) [DEPLOYING].
10:34:05,446 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
45b2be58c17c46712a1f99ffb71124c0.
10:34:05,447 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
3b5cadd7193ba9bd4dc2cf7ccc9667b2.
10:34:05,447 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
4a41e5491be9026da0456dd23f0709e4.
10:34:05,447 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
69e4facaa72255fa4011ad886be6e830.
10:34:05,447 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
f738d448f48642d75b3bf69475df9c0a.
10:34:05,447 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
21a45995954d4f80f8e9559bd6c050c2.
10:34:05,448 INFO
org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Activate slot
21a45995954d4f80f8e9559bd6c050c2.
10:34:05,449 INFO org.apache.flink.streaming.runtime.tasks.StreamTask
- Using job/cluster config to configure application-defined state backend:
MemoryStateBackend (data in heap memory / checkpoints to JobManager)
(checkpoints: 'null', savepoints: 'null', asynchronous: UNDEFINED,
maxStateSize: 5242880)
10:34:05,449 INFO org.apache.flink.streaming.runtime.tasks.StreamTask
- Using application-defined state backend: MemoryStateBackend (data in heap
memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null',
asynchronous: TRUE, maxStateSize: 5242880)
10:34:05,449 INFO org.apache.flink.runtime.taskmanager.Task
- hoodie_stream_write (4/4)#0 (5e2085d96585101e586a7ea47bc56a98) switched
from DEPLOYING to RUNNING.
10:34:05,450 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- hoodie_stream_write (4/4) (5e2085d96585101e586a7ea47bc56a98) switched
from DEPLOYING to RUNNING.
10:34:05,450 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Received task Sink: clean_commits (1/1)#0
(e46f7227496a98cb8360e4afa2fcdca1), deploy into slot with allocation id
21a45995954d4f80f8e9559bd6c050c2.
10:34:05,451 INFO org.apache.flink.runtime.taskmanager.Task
- Sink: clean_commits (1/1)#0 (e46f7227496a98cb8360e4afa2fcdca1) switched
from CREATED to DEPLOYING.
10:34:05,451 INFO org.apache.flink.runtime.taskmanager.Task
- Loading JAR files for task Sink: clean_commits (1/1)#0
(e46f7227496a98cb8360e4afa2fcdca1) [DEPLOYING].
10:34:05,452 INFO org.apache.flink.runtime.taskmanager.Task
- Registering task at network: Sink: clean_commits (1/1)#0
(e46f7227496a98cb8360e4afa2fcdca1) [DEPLOYING].
10:34:05,454 INFO org.apache.flink.streaming.runtime.tasks.StreamTask
- Using job/cluster config to configure application-defined state backend:
MemoryStateBackend (data in heap memory / checkpoints to JobManager)
(checkpoints: 'null', savepoints: 'null', asynchronous: UNDEFINED,
maxStateSize: 5242880)
10:34:05,454 INFO org.apache.flink.streaming.runtime.tasks.StreamTask
- Using application-defined state backend: MemoryStateBackend (data in heap
memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null',
asynchronous: TRUE, maxStateSize: 5242880)
10:34:05,454 INFO org.apache.flink.runtime.taskmanager.Task
- Sink: clean_commits (1/1)#0 (e46f7227496a98cb8360e4afa2fcdca1) switched
from DEPLOYING to RUNNING.
10:34:05,458 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Sink: clean_commits (1/1) (e46f7227496a98cb8360e4afa2fcdca1) switched
from DEPLOYING to RUNNING.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,518 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,518 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,518 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,517 INFO
org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder - Finished to
build heap keyed state-backend.
10:34:05,528 WARN org.apache.flink.runtime.taskmanager.TaskManagerLocation
- No hostname could be resolved for the IP address 127.0.0.1, using IP
address as host name. Local input split assignment (such as for HDFS files) may
be impacted.
10:34:05,534 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,541 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,548 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,547 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,550 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,550 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,547 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,547 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,552 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,552 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,552 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,552 INFO org.apache.flink.runtime.state.heap.HeapKeyedStateBackend
- Initializing heap keyed state backend with stream factory.
10:34:05,613 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,614 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,614 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,614 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,634 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,636 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,637 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,638 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,640 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,641 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,645 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,647 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,662 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,662 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,663 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,662 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,665 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,668 INFO org.apache.hudi.sink.StreamWriteFunction
- Send bootstrap write metadata event to coordinator, task[1].
10:34:05,668 INFO org.apache.hudi.sink.StreamWriteFunction
- Send bootstrap write metadata event to coordinator, task[3].
10:34:05,668 INFO org.apache.hudi.sink.StreamWriteFunction
- Send bootstrap write metadata event to coordinator, task[2].
10:34:05,674 INFO org.apache.hudi.sink.StreamWriteFunction
- Send bootstrap write metadata event to coordinator, task[0].
10:34:05,676 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Executor executes action [handle write metadata event for instant ]
success!
10:34:05,676 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Executor executes action [handle write metadata event for instant ]
success!
10:34:05,677 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Executor executes action [handle write metadata event for instant ]
success!
10:34:05,678 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,678 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,679 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,679 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading Active commit timeline for
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,681 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,690 INFO org.apache.hudi.common.table.view.FileSystemViewManager
- Creating View Manager with storage type :MEMORY
10:34:05,690 INFO org.apache.hudi.common.table.view.FileSystemViewManager
- Creating in-memory based Table View
10:34:05,691 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,694 INFO org.apache.hudi.sink.partitioner.profile.WriteProfile
- Refresh average bytes per record => 1024
10:34:05,694 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,696 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,699 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Executor executes action [handle write metadata event for instant ]
success!
10:34:05,704 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,707 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,708 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,708 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading Active commit timeline for
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,709 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,717 INFO org.apache.flink.runtime.taskmanager.Task
- Source: Values(tuples=[[{ 1, _UTF-16LE'1', _UTF-16LE'中国', _UTF-16LE'0',
1, _UTF-16LE'中国', _UTF-16LE'sap_area', _UTF-16LE'', _UTF-16LE'ENABLE',
_UTF-16LE'zhangsan', 2021-09-18 16:48:00, _UTF-16LE'ayj', 2021-09-18 16:48:00,
0 }]]) (1/1)#0 (6a4bc3af8b2ff4a559b4ea525d82672f) switched from RUNNING to
FINISHED.
10:34:05,717 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Source: Values(tuples=[[{ 1, _UTF-16LE'1',
_UTF-16LE'中国', _UTF-16LE'0', 1, _UTF-16LE'中国', _UTF-16LE'sap_area',
_UTF-16LE'', _UTF-16LE'ENABLE', _UTF-16LE'zhangsan', 2021-09-18 16:48:00,
_UTF-16LE'ayj', 2021-09-18 16:48:00, 0 }]]) (1/1)#0
(6a4bc3af8b2ff4a559b4ea525d82672f).
10:34:05,720 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (3/8)#0 (f55cea56f6c92ae6431d74d60038df8d) switched from
RUNNING to FINISHED.
10:34:05,720 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (3/8)#0
(f55cea56f6c92ae6431d74d60038df8d).
10:34:05,717 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,723 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (4/8)#0 (7d03b476ed15f492d5f2b3b6fd6d70a5) switched from
RUNNING to FINISHED.
10:34:05,730 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (4/8)#0
(7d03b476ed15f492d5f2b3b6fd6d70a5).
10:34:05,722 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (8/8)#0 (4cbc4b54611bc044378ef998f3684352) switched from
RUNNING to FINISHED.
10:34:05,722 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Source: Values(tuples=[[{ 1, _UTF-16LE'1', _UTF-16LE'中国',
_UTF-16LE'0', 1, _UTF-16LE'中国', _UTF-16LE'sap_area', _UTF-16LE'',
_UTF-16LE'ENABLE', _UTF-16LE'zhangsan', 2021-09-18 16:48:00, _UTF-16LE'ayj',
2021-09-18 16:48:00, 0 }]]) (1/1)#0 6a4bc3af8b2ff4a559b4ea525d82672f.
10:34:05,734 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (5/8)#0 (7c2e16589f48980e9d1a98b2e7657812) switched from
RUNNING to FINISHED.
10:34:05,734 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (5/8)#0
(7c2e16589f48980e9d1a98b2e7657812).
10:34:05,734 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (7/8)#0 (54a81f66851c0dd2941a3f803d219748) switched from
RUNNING to FINISHED.
10:34:05,734 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (7/8)#0
(54a81f66851c0dd2941a3f803d219748).
10:34:05,735 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (2/8)#0 (2f7ff7feae62ead6c95de8adcbae446f) switched from
RUNNING to FINISHED.
10:34:05,735 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (2/8)#0
(2f7ff7feae62ead6c95de8adcbae446f).
10:34:05,735 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (8/8)#0
(4cbc4b54611bc044378ef998f3684352).
10:34:05,737 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,738 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,738 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading Active commit timeline for
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,739 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,741 INFO org.apache.hudi.common.table.view.FileSystemViewManager
- Creating View Manager with storage type :REMOTE_FIRST
10:34:05,741 INFO org.apache.hudi.common.table.view.FileSystemViewManager
- Creating remote first table view
10:34:05,742 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (3/8)#0 f55cea56f6c92ae6431d74d60038df8d.
10:34:05,743 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (4/8)#0 7d03b476ed15f492d5f2b3b6fd6d70a5.
10:34:05,743 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (5/8)#0 7c2e16589f48980e9d1a98b2e7657812.
10:34:05,743 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (7/8)#0 54a81f66851c0dd2941a3f803d219748.
10:34:05,743 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (2/8)#0 2f7ff7feae62ead6c95de8adcbae446f.
10:34:05,743 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (8/8)#0 4cbc4b54611bc044378ef998f3684352.
10:34:05,744 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,744 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Source: Values(tuples=[[{ 1, _UTF-16LE'1', _UTF-16LE'中国', _UTF-16LE'0',
1, _UTF-16LE'中国', _UTF-16LE'sap_area', _UTF-16LE'', _UTF-16LE'ENABLE',
_UTF-16LE'zhangsan', 2021-09-18 16:48:00, _UTF-16LE'ayj', 2021-09-18 16:48:00,
0 }]]) (1/1) (6a4bc3af8b2ff4a559b4ea525d82672f) switched from RUNNING to
FINISHED.
10:34:05,746 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,747 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,747 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading Active commit timeline for
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,747 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (3/8) (f55cea56f6c92ae6431d74d60038df8d) switched from
RUNNING to FINISHED.
10:34:05,748 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (4/8) (7d03b476ed15f492d5f2b3b6fd6d70a5) switched from
RUNNING to FINISHED.
10:34:05,748 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Optional.empty
10:34:05,748 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (5/8) (7c2e16589f48980e9d1a98b2e7657812) switched from
RUNNING to FINISHED.
10:34:05,749 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (7/8) (54a81f66851c0dd2941a3f803d219748) switched from
RUNNING to FINISHED.
10:34:05,749 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (2/8) (2f7ff7feae62ead6c95de8adcbae446f) switched from
RUNNING to FINISHED.
10:34:05,749 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (8/8) (4cbc4b54611bc044378ef998f3684352) switched from
RUNNING to FINISHED.
10:34:05,750 INFO org.apache.hudi.client.AbstractHoodieWriteClient
- Generate a new instant time: 20210923103405 action: commit
10:34:05,751 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Creating a new
instant [==>20210923103405__commit__REQUESTED]
10:34:05,753 INFO org.apache.flink.runtime.taskmanager.Task
- Filter -> Map (6/8)#0 (1167445b24e5d29a6b9ed39e64ae8b7b) switched from
RUNNING to FINISHED.
10:34:05,753 INFO org.apache.flink.runtime.taskmanager.Task
- Freeing task resources for Filter -> Map (6/8)#0
(1167445b24e5d29a6b9ed39e64ae8b7b).
10:34:05,754 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor
- Un-registering task and sending final execution state FINISHED to
JobManager for task Filter -> Map (6/8)#0 1167445b24e5d29a6b9ed39e64ae8b7b.
10:34:05,754 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph
- Filter -> Map (6/8) (1167445b24e5d29a6b9ed39e64ae8b7b) switched from
RUNNING to FINISHED.
10:34:05,755 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Loading HoodieTableMetaClient from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,764 INFO org.apache.hudi.common.table.HoodieTableConfig
- Loading table properties from
file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/hoodie.properties
10:34:05,765 INFO org.apache.hudi.common.table.HoodieTableMetaClient
- Finished Loading Table of type COPY_ON_WRITE(version=1,
baseFileFormat=PARQUET) from
file:///d:/yuou/dev/ods/mmt-new/router_configure/configure_area
10:34:05,767 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Option{val=[==>20210923103405__commit__REQUESTED]}
10:34:05,768 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Checking for
file exists
?file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/20210923103405.commit.requested
10:34:05,770 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Create new file
for toInstant
?file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area/.hoodie/20210923103405.inflight
10:34:05,770 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Create instant [20210923103405] for table [configure_area] with type
[COPY_ON_WRITE]
10:34:05,770 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator
- Executor executes action [initialize instant ] success!
10:34:05,945 INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
upto : Option{val=[==>20210923103405__commit__INFLIGHT]}
Exception in thread "main" org.apache.hudi.exception.HoodieException: Get
table avro schema error
at
org.apache.hudi.table.HoodieTableSource.getInputFormat(HoodieTableSource.java:322)
at
org.apache.hudi.table.HoodieTableSource.getInputFormat(HoodieTableSource.java:302)
at
org.apache.hudi.table.HoodieTableSource$1.produceDataStream(HoodieTableSource.java:190)
at
org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalTableSourceScan.createSourceTransformation(CommonPhysicalTableSourceScan.scala:88)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlanInternal(StreamExecTableSourceScan.scala:91)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlanInternal(StreamExecTableSourceScan.scala:44)
at
org.apache.flink.table.planner.plan.nodes.exec.ExecNode$class.translateToPlan(ExecNode.scala:59)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecTableSourceScan.translateToPlan(StreamExecTableSourceScan.scala:44)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToTransformation(StreamExecLegacySink.scala:158)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlanInternal(StreamExecLegacySink.scala:82)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlanInternal(StreamExecLegacySink.scala:48)
at
org.apache.flink.table.planner.plan.nodes.exec.ExecNode$class.translateToPlan(ExecNode.scala:59)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecLegacySink.translateToPlan(StreamExecLegacySink.scala:48)
at
org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:66)
at
org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:65)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
org.apache.flink.table.planner.delegation.StreamPlanner.translateToPlan(StreamPlanner.scala:65)
at
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:167)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1329)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:707)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeOperation(TableEnvironmentImpl.java:1107)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:666)
at
com.yuou.flinkhudi.util.flink.dwd.generate.ContryAreaCodeData$.generateContryAreaCodeData(ContryAreaCodeData.scala:93)
at
com.yuou.flinkhudi.util.flink.dwd.generate.ContryAreaCodeData$.main(ContryAreaCodeData.scala:101)
at
com.yuou.flinkhudi.util.flink.dwd.generate.ContryAreaCodeData.main(ContryAreaCodeData.scala)
Caused by: org.apache.hudi.exception.InvalidTableException: Invalid Hoodie
Table. file:/d:/yuou/dev/ods/mmt-new/router_configure/configure_area
at
org.apache.hudi.common.table.TableSchemaResolver.lambda$getTableParquetSchemaFromDataFile$0(TableSchemaResolver.java:88)
at org.apache.hudi.common.util.Option.orElseThrow(Option.java:123)
at
org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchemaFromDataFile(TableSchemaResolver.java:88)
at
org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchemaFromDataFile(TableSchemaResolver.java:153)
at
org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchema(TableSchemaResolver.java:187)
at
org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchema(TableSchemaResolver.java:163)
at
org.apache.hudi.table.HoodieTableSource.getInputFormat(HoodieTableSource.java:320)
... 31 more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
