Reo-LEI commented on issue #3551: URL: https://github.com/apache/iceberg/issues/3551#issuecomment-1005360552
OK, I think I figure it out this problem. In #2898 of [EqualityFieldKeySelector](https://github.com/apache/iceberg/blob/14af007c7e6fa206cc44dbfde7546feb33718277/flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/sink/EqualityFieldKeySelector.java#L73) return the `RowDataWrapper` as key directly. When we use RocksDB as flink state backend and distribute record through network, flink `StreamTaskNetworkOutput` will set key context by `StreamOperatorStateHandler` and RocksDB state backend will serialize the key by kryo. But kryo does not directly support serialized lamda functions, and finally we will got this error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
