voonhous opened a new issue, #20659:
URL: https://github.com/apache/hudi/issues/20659

   ### Bug Description
   
   **What happened:**
   
   On a MOR snapshot (`_rt`) read, the Trino connector returns different CHAR 
values than the read-optimized table or the base file does for the same row. It 
returns no error, only wrong values and missing rows:
   
   - Leading spaces are dropped: `' ab'` reads back as `'ab'`.
   - Tabs and other control characters at either end are dropped: `'\tx\t'` 
reads back as `'x'`.
   - A filter like `WHERE code = CAST(' ab' AS CHAR(5))` silently skips the row 
on the snapshot read but matches it on the read-optimized read. Joins and GROUP 
BY on the column are affected the same way.
   
   Both updated rows and rows that were never updated are affected, as long as 
their file group has a log file, because the merge converts every base row in 
the group as well.
   
   Cause: `HudiAvroSerializer.writeSlice` converts CHAR with `String.trim()`, 
which strips all leading and trailing characters <= U+0020. Trino's Parquet 
reader only removes trailing U+0020 padding 
(`Chars.byteCountWithoutTrailingSpace`). The code came in with the connector 
migration in #18837 and has not changed since.
   
   **What you expected:**
   
   The snapshot read returns the same CHAR value as the read-optimized read: 
truncated to the declared length and with only trailing spaces removed.
   
   **Steps to reproduce:**
   1. Create a MOR table where the metastore declares a column `code char(5)` 
over a Hudi string field.
   2. Insert `k3` with `code = ' ab'` and `k1` with any value, then upsert `k1` 
with `code = '\tx\t'` so the file group gets a log file.
   3. Compare `SELECT code FROM t WHERE key = 'k3'` with the same query on 
`t_rt`: `' ab  '` vs `'ab   '`.
   
   A functional test that reproduces this is below. It fails on master and 
passes with the fix in #20638.
   
   <details><summary>Repro test (TestHudiMorMergeModeSemantics, on 
CommitTimeOrderingHudiTablesInitializer)</summary>
   
   Fixture change: add `new Column("code", charHiveType(5), ...)` with Avro 
type STRING; base rows `k1 = "k1"`, `k2 = "k2"`, `k3 = " ab"`; the k1 log 
update writes `"\tx\t"`.
   
   ```java
   @Test
   public void testCharMatchesBaseReaderOnUntouchedRow()
   {
       String query = "SELECT code FROM %s WHERE key = 'k3'";
       Object readOptimized = 
computeScalar(query.formatted(CommitTimeOrderingHudiTablesInitializer.TABLE_NAME));
       assertThat(readOptimized).isEqualTo(" ab  ");
       
assertThat(computeScalar(query.formatted(CommitTimeOrderingHudiTablesInitializer.RT_TABLE_NAME))).isEqualTo(readOptimized);
   }
   
   @Test
   public void testCharFilterMatchesLeadingSpaceOnSnapshotRead()
   {
       assertQuery(
               "SELECT key FROM " + 
CommitTimeOrderingHudiTablesInitializer.RT_TABLE_NAME + " WHERE code = CAST(' 
ab' AS CHAR(5))",
               "VALUES 'k3'");
   }
   
   @Test
   public void testCharKeepsTabsOnUpdatedRow()
   {
       assertThat(computeScalar("SELECT code FROM " + 
CommitTimeOrderingHudiTablesInitializer.RT_TABLE_NAME + " WHERE key = 'k1'"))
               .isEqualTo("\tx\t  ");
   }
   ```
   
   </details>
   
   ### Environment
   
   **Hudi version:** master (1.3.0-SNAPSHOT, at a65a573a)
   **Query engine:** Trino, in-repo `hudi-trino` connector
   **Relevant configs:** MOR table, snapshot (`_rt`) query, metastore column 
type `char(n)`
   
   ### Logs and Stack Trace
   
   No error is raised. The test output on master is below.
   
   ```
   testCharMatchesBaseReaderOnUntouchedRow   expected: " ab  "      but was: 
"ab   "
   testCharKeepsTabsOnUpdatedRow             expected: "\tx\t  "    but was: "x 
   "
   testCharFilterMatchesLeadingSpaceOnSnapshotRead   expected 1 row (k3), got 0
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to