[
https://issues.apache.org/jira/browse/HUDI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-6670:
----------------------------
Description:
Metadata table validator throws the following exception when there is completed
rollback and no completed commits in the data table and there is no completed
instants in the MDT. In this case, both data table and MDT are empty, but the
{code:java}
23/08/08 22:48:13 WARN HoodieMetadataTableValidator: Metadata table is not
available to read for now,
org.apache.hudi.exception.HoodieValidationException: There is no completed
instant for metadata table.
at
org.apache.hudi.utilities.HoodieMetadataTableValidator.checkMetadataTableIsAvailable(HoodieMetadataTableValidator.java:500)
at
org.apache.hudi.utilities.HoodieMetadataTableValidator.doMetadataTableValidation(HoodieMetadataTableValidator.java:405)
at
org.apache.hudi.utilities.HoodieMetadataTableValidator.doHoodieMetadataTableValidationOnce(HoodieMetadataTableValidator.java:377)
at
org.apache.hudi.utilities.HoodieMetadataTableValidator.run(HoodieMetadataTableValidator.java:362)
at
org.apache.hudi.utilities.HoodieMetadataTableValidator.main(HoodieMetadataTableValidator.java:342)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code}
Data table timeline
{code:java}
Aug 6 22:57 20230807055713708.deltacommit.requested
Aug 6 22:57 20230807055743816.rollback
Aug 6 22:57 20230807055743816.rollback.inflight
Aug 6 22:57 20230807055743816.rollback.requested {code}
MDT timeline
{code:java}
Aug 6 22:56 00000000000000010.deltacommit.inflight
Aug 6 22:56 00000000000000010.deltacommit.requested
Aug 6 22:57 20230807055748276.restore.inflight
Aug 6 22:57 20230807055748276.restore.requested
Aug 6 22:57 20230807055749482.rollback.inflight
Aug 6 22:57 20230807055749482.rollback.requested {code}
> Fix timeline check in metadata table validator
> ----------------------------------------------
>
> Key: HUDI-6670
> URL: https://issues.apache.org/jira/browse/HUDI-6670
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Priority: Major
>
> Metadata table validator throws the following exception when there is
> completed rollback and no completed commits in the data table and there is no
> completed instants in the MDT. In this case, both data table and MDT are
> empty, but the
> {code:java}
> 23/08/08 22:48:13 WARN HoodieMetadataTableValidator: Metadata table is not
> available to read for now,
> org.apache.hudi.exception.HoodieValidationException: There is no completed
> instant for metadata table.
> at
> org.apache.hudi.utilities.HoodieMetadataTableValidator.checkMetadataTableIsAvailable(HoodieMetadataTableValidator.java:500)
> at
> org.apache.hudi.utilities.HoodieMetadataTableValidator.doMetadataTableValidation(HoodieMetadataTableValidator.java:405)
> at
> org.apache.hudi.utilities.HoodieMetadataTableValidator.doHoodieMetadataTableValidationOnce(HoodieMetadataTableValidator.java:377)
> at
> org.apache.hudi.utilities.HoodieMetadataTableValidator.run(HoodieMetadataTableValidator.java:362)
> at
> org.apache.hudi.utilities.HoodieMetadataTableValidator.main(HoodieMetadataTableValidator.java:342)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code}
> Data table timeline
> {code:java}
> Aug 6 22:57 20230807055713708.deltacommit.requested
> Aug 6 22:57 20230807055743816.rollback
> Aug 6 22:57 20230807055743816.rollback.inflight
> Aug 6 22:57 20230807055743816.rollback.requested {code}
> MDT timeline
> {code:java}
> Aug 6 22:56 00000000000000010.deltacommit.inflight
> Aug 6 22:56 00000000000000010.deltacommit.requested
> Aug 6 22:57 20230807055748276.restore.inflight
> Aug 6 22:57 20230807055748276.restore.requested
> Aug 6 22:57 20230807055749482.rollback.inflight
> Aug 6 22:57 20230807055749482.rollback.requested {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
