[
https://issues.apache.org/jira/browse/HUDI-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17398431#comment-17398431
]
Prashant Wason commented on HUDI-2303:
--------------------------------------
Confirmed: When the dataset is being compacted, the latest base file is being
missed. Hence, the test fails as the last committed data is missing. Checked
that there is no sync failure and table is consistent. The very next lookup
from the table returns all data. So this is very specific to the inline
compaction workflow.
I suspect it is to do with the TimelineServer or using an older
HoodieBackedTableMetadata instance which reads the older data.
> TestMereIntoLogOnlyTable with metadata enabled surfaces likely bug
> ------------------------------------------------------------------
>
> Key: HUDI-2303
> URL: https://issues.apache.org/jira/browse/HUDI-2303
> Project: Apache Hudi
> Issue Type: Bug
> Components: Common Core
> Reporter: Udit Mehrotra
> Assignee: Prashant Wason
> Priority: Major
>
> While enabling Metadata as part of
> [https://github.com/apache/hudi/pull/3411/] one of the test that fails is
> *TestMereIntoLogOnlyTable*.
> Upon looking a bit, what I found is after the final *Merge* command there is
> an inline compaction that is triggered. The parquet file formed as part of
> the compaction misses out on the data from the latest log file right before
> compaction.
> I think it might be because of metadata returning an incorrect list for
> compaction, missing out on the latest log file.
> cc [~pwason] [~vinoth]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
