DanielLeens commented on issue #11499: URL: https://github.com/apache/seatunnel/issues/11499#issuecomment-5022489702
I traced the Spark conversion path, and this is a real Parquet-specific bug rather than a Hive configuration issue. `ParquetReadStrategy.read(path, tableId, output)` currently constructs `new FileSourceSplit(path)`, which drops the supplied table id. The CSV, JSON, Text, XML, and Excel readers use the table-aware constructor. Spark `InternalRowConverter` reserves an internal field for `SeaTunnelRow.getTableId()`, so the missing id reaches the generated unsafe row as null and triggers the `UTF8String` failure shown in the stack trace. The current `dev` branch still contains this path, and I did not find an open PR or recent merged fix for it. The existing `bug` and `help wanted` labels are therefore appropriate. A focused fix should preserve `tableId` when Parquet creates the split, add a Spark translation regression that verifies the internal table-id field is non-null, and audit the other Parquet split construction paths so the same metadata is not lost in split-size or row-group mode. Contributions are very welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
