aokolnychyi edited a comment on issue #569: Use broadcast variables in IcebergSource URL: https://github.com/apache/incubator-iceberg/pull/569#issuecomment-562096640 @rdblue, I like the idea and it can work if we make `Supplier` serializable. There is one more point we should be careful about. Iceberg's `ReadTask` can reference `ManifestReadTask` that holds a reference to `FileIO`. We don't broadcast that and it should work assuming that Spark will continue to use Java serialization for tasks.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
