jackye1995 commented on issue #2068: URL: https://github.com/apache/iceberg/issues/2068#issuecomment-759646229
> We currently use the SparkTableUtil's listPartition function. It has custom code for parquet, orc, and avro and uses the directory structure to determine which files belong to which partitions. After that it's just a getFile, table.append.appendFile. Internally we also provide an option for specifying a specific partition to import. Thanks for the explanation, this makes sense to me now. In this case, I think it is also valuable to expose this option to specify a specific partition in this procedure. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
