[
https://issues.apache.org/jira/browse/HUDI-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo closed HUDI-3945.
---------------------------
Resolution: Fixed
> After the async compaction operation is complete, the task should exit.
> -----------------------------------------------------------------------
>
> Key: HUDI-3945
> URL: https://issues.apache.org/jira/browse/HUDI-3945
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: YangXuan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.11.1
>
>
> This problem occurs if you perform the following operations:
> 1、Create a mor table and perform the upsert operation on it for five times.
> 2、Gets the timestamp of the last upsert execution, assuming it is
> 20220318191221.
> 3、Do compaction schedule by spark-submit and the timestamp is 20220318191221
> plus 5.
> spark-submit --conf
> "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties"
> --jars /opt/client/Hudi/hudi/lib/hudi-client-common*.jar --class
> org.apache.hudi.utilities.HoodieCompactor
> /opt/client/Hudi/hudi/lib/hudi-utilities*.jar --base-path
> /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100
> --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time
> 20220318191226 --schedule --strategy
> org.apache.hudi.table.action.compact.strategy.UnBoundedCompactionStrategy
> 4、Run compaction by spark-submit and you will see that the task of running
> compaction is executed successfully, but the spark task does not exit.
> spark-submit --conf
> "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties"
> --num-executors 4 --jars
> /opt/client/Hudi/hudi/lib/hudi-client-common-{_}.jar --class
> org.apache.hudi.utilities.HoodieCompactor
> /opt/client/Hudi/hudi/lib/hudi-utilities_{_}.jar --base-path
> /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100
> --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time
> 20220318191226
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
