liunaijie commented on issue #11499: URL: https://github.com/apache/seatunnel/issues/11499#issuecomment-5018297244
The root cause is: spark translation need a `tableId` field https://github.com/apache/seatunnel/blob/dev/seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/serialization/InternalRowConverter.java#L181 But when read parquet file, it pass `null`. https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/ParquetReadStrategy.java#L97 In my fix, I updated the ParquetReadStrategy, also pass `tableId` to `FileSourceSplit`. Then this issue solved. I created this issue so that everyone can check for more similar cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
