openinx commented on issue #1643: URL: https://github.com/apache/iceberg/issues/1643#issuecomment-714888205
I think we have reached an agreement that it's useful to reuse objects in the row data iterators, the current problem is : how to reuse it ? Now we `RichInputFormat#nextRecord(T reuse)` have provided the `reuse` instance for reusing, but our parquet, orc, avro have an internal object reuse mechanism. Unless we change the `Iteratable` interface to accept the `reuse` from `RichInputFormat`, we couldn't bridge them into a single `reuse` instance. So another way I can image is copying it from the internal parquet, orc, avro iterator's `reuse` to the `RichInputFormat`'s `reuse` . But as you said, for a generic type `T`, it does not expose a `Cloneable` method to accomplish the copying. In that way, we may need a copy function similar to `Cloneable` when constructing the `FlinkInputFormat` ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
