[
https://issues.apache.org/jira/browse/HUDI-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708576#comment-17708576
]
Rahil Chertara commented on HUDI-3945:
--------------------------------------
Hi, I currently have my doubts on this pr and left some comments there. When I
try the steps above in the JIRA without the pr changes im not seeing any issues
and the spark task is shutting down gracefully. Can you please give more
context on how you saw this issue exactly?
> After the async compaction operation is complete, the task should exit.
> -----------------------------------------------------------------------
>
> Key: HUDI-3945
> URL: https://issues.apache.org/jira/browse/HUDI-3945
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: YangXuan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.11.1
>
>
> This problem occurs if you perform the following operations:
> 1、Create a mor table and perform the upsert operation on it for five times.
> 2、Gets the timestamp of the last upsert execution, assuming it is
> 20220318191221.
> 3、Do compaction schedule by spark-submit and the timestamp is 20220318191221
> plus 5.
> spark-submit --conf
> "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties"
> --jars /opt/client/Hudi/hudi/lib/hudi-client-common*.jar --class
> org.apache.hudi.utilities.HoodieCompactor
> /opt/client/Hudi/hudi/lib/hudi-utilities*.jar --base-path
> /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100
> --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time
> 20220318191226 --schedule --strategy
> org.apache.hudi.table.action.compact.strategy.UnBoundedCompactionStrategy
> 4、Run compaction by spark-submit and you will see that the task of running
> compaction is executed successfully, but the spark task does not exit.
> spark-submit --conf
> "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties"
> --num-executors 4 --jars
> /opt/client/Hudi/hudi/lib/hudi-client-common-{_}.jar --class
> org.apache.hudi.utilities.HoodieCompactor
> /opt/client/Hudi/hudi/lib/hudi-utilities_{_}.jar --base-path
> /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100
> --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time
> 20220318191226
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
