[
https://issues.apache.org/jira/browse/HUDI-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17922239#comment-17922239
]
Davis Zhang commented on HUDI-8936:
-----------------------------------
Synced with Lin, [https://github.com/apache/hudi/pull/12735] is because
override payload does not handle delete at all. He has a PR adding the support.
He will confirm with his change the test will pass. Ethan will review his PR
targeting 101.
[https://github.com/apache/hudi/pull/12736] is because SQL does not trigger
auto upgrade. As a result, we are using backward compatible reader/writer
working on version 6 MDT on which is known to be an unsupported case. It has
the symptom that records become invisible.
To move forward, Ethan will disable MDT in version 6 test dimension and the
auto upgrade issue is tracked https://issues.apache.org/jira/browse/HUDI-8937
> MERGE INTO delete does not work in some cases of COMMIT_TIME_ORDERING
> ---------------------------------------------------------------------
>
> Key: HUDI-8936
> URL: https://issues.apache.org/jira/browse/HUDI-8936
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Y Ethan Guo
> Assignee: Davis Zhang
> Priority: Blocker
> Fix For: 1.0.1
>
> Attachments: Screenshot 2025-01-29 at 12.31.01.png
>
> Original Estimate: 6h
> Remaining Estimate: 6h
>
> After fixing HUDI-8850 on existing tests,
> [https://github.com/apache/hudi/pull/12725] introduces more test cases on
> COMMIT_TIME_ORDERING, and some of them failed because MERGE INTO delete does
> not take effect.
> Test dimensions:
> All failures from "Test merge operations .." are due to the validation
> failure below
> !Screenshot 2025-01-29 at 12.31.01.png|width=1411,height=311!
> Sample error message:
> {code:java}
> Expected Array([3,C,30.0,100], [4,D2,45.0,101], [5,E2,55.0,99],
> [6,F2,65.0,100]), but got Array([0,X,20.0,100], [1,A,10.0,100],
> [2,B,20.0,100], [3,C,30.0,100], [4,D2,45.0,101], [5,E2,55.0,99],
> [6,F2,65.0,100])
> ScalaTestFailureLocation:
> org.apache.spark.sql.hudi.common.HoodieSparkSqlTestBase at
> (HoodieSparkSqlTestBase.scala:130)
> org.scalatest.exceptions.TestFailedException: Expected Array([3,C,30.0,100],
> [4,D2,45.0,101], [5,E2,55.0,99], [6,F2,65.0,100]), but got
> Array([0,X,20.0,100], [1,A,10.0,100], [2,B,20.0,100], [3,C,30.0,100],
> [4,D2,45.0,101], [5,E2,55.0,99], [6,F2,65.0,100])
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
> at
> org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1562)
> at org.scalatest.Assertions.assertResult(Assertions.scala:867)
> at org.scalatest.Assertions.assertResult$(Assertions.scala:863)
> at org.scalatest.funsuite.AnyFunSuite.assertResult(AnyFunSuite.scala:1562)
> at
> org.apache.spark.sql.hudi.common.HoodieSparkSqlTestBase.checkAnswer(HoodieSparkSqlTestBase.scala:130)
> at
> org.apache.spark.sql.hudi.dml.TestMergeModeCommitTimeOrdering.$anonfun$new$9(TestMergeModeCommitTimeOrdering.scala:309)
> at
> org.apache.spark.sql.hudi.dml.TestMergeModeCommitTimeOrdering.$anonfun$new$9$adapted(TestMergeModeCommitTimeOrdering.scala:239)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
