Leoyzen opened a new issue, #7845:
URL: https://github.com/apache/hudi/issues/7845
**Describe the problem you faced**
『Unsupported Operation Exception』 raises while compacting.
**To Reproduce**
Steps to reproduce the behavior:
1. compaction is running on MOR table
**Expected behavior**
Compaction successfully.
**Environment Description**
* Hudi version : 0.13.0-rc1
* Spark version : N/A
* Hive version : 3.1.2
* Hadoop version : 3.1.3
* Storage (HDFS/S3/GCS..) : OSS
* Running on Docker? (yes/no) : YES, HA VVP
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```LOG
2023-02-04 00:05:19,130 [pool-26-thread-1] ERROR
org.apache.hudi.sink.compact.CompactOperator [] - Executor
executes action [Execute compaction for instant 20230201172137801 from task 1]
error
org.apache.hudi.exception.HoodieException: Unsupported Operation Exception
at
org.apache.hudi.common.util.collection.BitCaskDiskMap.values(BitCaskDiskMap.java:303)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.common.util.collection.ExternalSpillableMap.values(ExternalSpillableMap.java:278)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at java.util.Collections$UnmodifiableMap.values(Collections.java:1487)
~[?:1.8.0_102]
at
org.apache.hudi.io.HoodieMergeHandle.writeIncomingRecords(HoodieMergeHandle.java:465)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.io.HoodieMergeHandle.close(HoodieMergeHandle.java:477)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.action.commit.HoodieMergeHelper.runMerge(HoodieMergeHelper.java:168)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.HoodieFlinkCopyOnWriteTable.handleUpdateInternal(HoodieFlinkCopyOnWriteTable.java:374)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.HoodieFlinkCopyOnWriteTable.handleUpdate(HoodieFlinkCopyOnWriteTable.java:365)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.action.compact.CompactionExecutionHelper.writeFileAndGetWriteStats(CompactionExecutionHelper.java:64)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.action.compact.HoodieCompactor.compact(HoodieCompactor.java:231)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.table.action.compact.HoodieCompactor.compact(HoodieCompactor.java:144)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.sink.compact.CompactOperator.doCompaction(CompactOperator.java:136)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.sink.compact.CompactOperator.lambda$processElement$0(CompactOperator.java:119)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130)
~[flink-connector-hudi-flip-27-1.5-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147)
[?:1.8.0_102]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
[?:1.8.0_102]
at java.lang.Thread.run(Thread.java:834) [?:1.8.0_102]
```
**Possible Causes**
From the pr
https://github.com/apache/hudi/commit/f8028a400ebe7a64d7745fae90e769306a8a07a7，
the `getRecords` call from `HoodieMergedLogRecordScanner.java` changed to
return a `UnmodifiableMap`.
```JAVA
public Map<String, HoodieRecord> getRecords() {
return Collections.unmodifiableMap(records);
}
```
But in `HoodieMergeHandle`, there is a `instanceof ExternalSpillableMap`
which don't meet the condition.
https://github.com/apache/hudi/blob/fc031b9d3b04933aae00eab3c869446fba46b19a/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java#L392
https://github.com/apache/hudi/blob/fc031b9d3b04933aae00eab3c869446fba46b19a/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieMergeHandle.java#L407
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
