szehon-ho edited a comment on issue #3263: URL: https://github.com/apache/iceberg/issues/3263#issuecomment-939201068
I was thinking to explicitly set it to -1 when constructing the DataFile in listAvroPartitions, but I am worried its setting wrong metadata (though it is explicitly set to -1 for Metrics..) Doesn't seem like Avro file has any metadata to quickly get recordCount, I wonder if making a Spark job to read the rows is overkill? Any thoughts @RussellSpitzer and @aokolnychyi ? Or is the add_file for avro designed to just set -1 for record count? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
