rdblue commented on a change in pull request #119: Split files when planning
scan tasks
URL: https://github.com/apache/incubator-iceberg/pull/119#discussion_r263938049
##########
File path:
spark/src/test/java/com/netflix/iceberg/spark/source/TestFilteredScan.java
##########
@@ -429,7 +429,7 @@ private File buildPartitionedTable(String desc,
PartitionSpec spec, String udf,
Table byId = TABLES.create(SCHEMA, spec, location.toString());
// do not combine splits because the tests expect a split per partition
- byId.updateProperties().set("read.split.target-size", "1").commit();
+ byId.updateProperties().set("read.split.target-size", "2048").commit();
Review comment:
Can you update the comment to note that a lower value will also cause
failures because the files are split? Something like "use a target size that
will not combine or split files ..."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
