rdsr edited a comment on pull request #1267: URL: https://github.com/apache/iceberg/pull/1267#issuecomment-665870006
> I run MR in local mode whereas this seems to be running in distributed mode with YARN. I'll have to dig deeper but my guess is that `table` is null because in that case the calls to `getSplits` and then `getRecordReader` happens in two different processes. > > You mind trying this [branch](https://github.com/guilload/iceberg/tree/guilload--carry-table-schema-and-projection-within-iceberg-split)? I've added a `schema` and a `projection` attribute to `IcebergSplit`, this way we can remove the "forwarding" logic in the input format. @rdsr, what do you think of this approach? One downside is the increase in size of the serialized splits. Yea, I think that might be the reason ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
