[
https://issues.apache.org/jira/browse/HUDI-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17922448#comment-17922448
]
Lin Liu commented on HUDI-8938:
-------------------------------
For MOR table version 6 test case, what I observed is:
After executing the first two queries, we have to commits, and one base file
and one log file:
{code:java}
➜ spark-b1e36e21-0244-4b10-8878-0e52a20ba44f ls -al
total 896
drwxr-xr-x@ 9 linliu staff 288 Jan 30 08:50 .
drwx------@ 3300 linliu staff 105600 Jan 30 08:50 ..
-rw-r--r--@ 1 linliu staff 16 Jan 30 08:50
..f833210b-2fbb-4dae-bc7b-691187c6eea6-0_20250130165022189.log.1_1-0-1.crc
-rw-r--r--@ 1 linliu staff 12 Jan 30 08:50
..hoodie_partition_metadata.crc
-rw-r--r--@ 1 linliu staff 3408 Jan 30 08:50
.f833210b-2fbb-4dae-bc7b-691187c6eea6-0_0-13-28_20250130165022189.parquet.crc
-rw-r--r--@ 1 linliu staff 1003 Jan 30 08:50
.f833210b-2fbb-4dae-bc7b-691187c6eea6-0_20250130165022189.log.1_1-0-1
drwxr-xr-x@ 19 linliu staff 608 Jan 30 08:50 .hoodie
-rw-r--r--@ 1 linliu staff 96 Jan 30 08:50 .hoodie_partition_metadata
-rw-r--r--@ 1 linliu staff 435008 Jan 30 08:50
f833210b-2fbb-4dae-bc7b-691187c6eea6-0_0-13-28_20250130165022189.parquet {code}
However, after the MIT query, we can see that, the first log file is gone.
{code:java}
➜ spark-b1e36e21-0244-4b10-8878-0e52a20ba44f ls -al
total 896
drwxr-xr-x@ 9 linliu staff 288 Jan 30 08:56 .
drwx------@ 3300 linliu staff 105600 Jan 30 08:57 ..
-rw-r--r--@ 1 linliu staff 16 Jan 30 08:56
..f833210b-2fbb-4dae-bc7b-691187c6eea6-0_20250130165022189.log.2_1-0-1.crc
-rw-r--r--@ 1 linliu staff 12 Jan 30 08:50
..hoodie_partition_metadata.crc
-rw-r--r--@ 1 linliu staff 3408 Jan 30 08:50
.f833210b-2fbb-4dae-bc7b-691187c6eea6-0_0-13-28_20250130165022189.parquet.crc
-rw-r--r--@ 1 linliu staff 889 Jan 30 08:56
.f833210b-2fbb-4dae-bc7b-691187c6eea6-0_20250130165022189.log.2_1-0-1
drwxr-xr-x@ 25 linliu staff 800 Jan 30 08:56 .hoodie
-rw-r--r--@ 1 linliu staff 96 Jan 30 08:50 .hoodie_partition_metadata
-rw-r--r--@ 1 linliu staff 435008 Jan 30 08:50
f833210b-2fbb-4dae-bc7b-691187c6eea6-0_0-13-28_20250130165022189.parquet {code}
What is the expect behavior for MOR table version 6 here?
> INSERT INTO generates unexpected result on table version 6
> ----------------------------------------------------------
>
> Key: HUDI-8938
> URL: https://issues.apache.org/jira/browse/HUDI-8938
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Y Ethan Guo
> Assignee: Lin Liu
> Priority: Blocker
> Fix For: 1.0.1
>
>
> After fixing HUDI-8850 on existing tests,
> [https://github.com/apache/hudi/pull/12725] introduces more test cases on
> COMMIT_TIME_ORDERING, and one of them failed because after INSERT INTO the
> query result is wrong
> Test dimensions:
> TestMergeModeCommitTimeOrdering
> "mor,6,true,true"
> TestMergeModeEventTimeOrdering
> "mor,6,true", "mor,6,false"
>
> Error:
> {code:java}
> Expected Array([1,A,10.0,100], [1,A_equal,60.0,100], [2,B,20.0,100],
> [2,B_equal,70.0,100]), but got Array([1,A_equal,60.0,100],
> [2,B_equal,70.0,100])
> ScalaTestFailureLocation:
> org.apache.spark.sql.hudi.common.HoodieSparkSqlTestBase at
> (HoodieSparkSqlTestBase.scala:130)
> org.scalatest.exceptions.TestFailedException: Expected Array([1,A,10.0,100],
> [1,A_equal,60.0,100], [2,B,20.0,100], [2,B_equal,70.0,100]), but got
> Array([1,A_equal,60.0,100], [2,B_equal,70.0,100])
> at
> org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
> at
> org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
> at
> org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1562)
> at org.scalatest.Assertions.assertResult(Assertions.scala:867)
> at org.scalatest.Assertions.assertResult$(Assertions.scala:863)
> at org.scalatest.funsuite.AnyFunSuite.assertResult(AnyFunSuite.scala:1562)
> at
> org.apache.spark.sql.hudi.common.HoodieSparkSqlTestBase.checkAnswer(HoodieSparkSqlTestBase.scala:130)
> at
> org.apache.spark.sql.hudi.dml.TestMergeModeCommitTimeOrdering.$anonfun$new$5(TestMergeModeCommitTimeOrdering.scala:144)
> at
> org.apache.spark.sql.hudi.dml.TestMergeModeCommitTimeOrdering.$anonfun$new$5$adapted(TestMergeModeCommitTimeOrdering.scala:93)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
