rdblue commented on a change in pull request #374: Migrate spark table to iceberg table URL: https://github.com/apache/incubator-iceberg/pull/374#discussion_r318200212
########## File path: spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala ########## @@ -297,5 +302,81 @@ object SparkTableUtil { ) } } + + private def buildManifest(table: Table, + sparkDataFiles: Seq[SparkDataFile], + partitionSpec: PartitionSpec): ManifestFile = { + val outputFile = table.io + .newOutputFile(FileFormat.AVRO.addExtension("/tmp/" + UUID.randomUUID.toString)) Review comment: This can't be local because it is written on executors and read from the driver. If those are on different machines, the driver won't be able to find the location. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org