fatmanverse commented on issue #11499: URL: https://github.com/apache/seatunnel/issues/11499#issuecomment-5032903960
Hi @liunaijie @DanielLeens, I've opened #11516 to fix this. I traced the same root cause you both identified: `ParquetReadStrategy.read(path, tableId, output)` builds the split via `new FileSourceSplit(path)`, which drops the tableId (sets it to null), while Json/Text/Csv/Xml/Excel use the table-aware constructor and Orc calls `setTableId` directly. On Spark this null reaches `InternalRowConverter#parcel` and triggers the `UTF8String` NPE in the generated writer. @liunaijie you mentioned you already had a fix — I didn't find an open PR, so I went ahead and opened one to keep this moving. If you already have work in progress, I'm glad to defer to your PR or collaborate, just let me know. Addressing @DanielLeens's suggestions, the PR: 1. Preserves the tableId in `ParquetReadStrategy` (the root-cause fix). 2. Adds a defensive guard in the Spark `InternalRowConverter` so a null tableId is normalized to `""` (`SeaTunnelRow` already defaults tableId to `""`), plus a regression test in `MultiTableManagerTest` that reproduces the exact NPE without the guard. 3. Audited the other Parquet split-construction paths — `ParquetFileSplitStrategy` (row-group mode) and `AccordingToSplitSizeSplitStrategy` (split-size mode) already pass tableId through the 4-arg constructor, so there's no further leak. Review and feedback welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
