kevinjqliu commented on code in PR #2043: URL: https://github.com/apache/iceberg-rust/pull/2043#discussion_r2701899753
########## crates/integration_tests/testdata/spark/Dockerfile: ########## Review Comment: copied over from https://github.com/apache/iceberg-python/blob/ba65619113ebfd56515975b2e31a3f3df14bed55/dev/spark/Dockerfile ########## crates/integration_tests/testdata/spark/provision.py: ########## @@ -18,17 +18,8 @@ from pyspark.sql import SparkSession from pyspark.sql.functions import current_date, date_add, expr -# The configuration is important, otherwise we get many small -# parquet files with a single row. When a positional delete -# hits the Parquet file with one row, the parquet file gets -# dropped instead of having a merge-on-read delete file. Review Comment: `.coalesce(1)` below has the same effect -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
