kbendick commented on a change in pull request #2081:
URL: https://github.com/apache/iceberg/pull/2081#discussion_r556385303
##########
File path:
spark3/src/test/java/org/apache/iceberg/spark/source/TestFilteredScan.java
##########
@@ -529,7 +529,7 @@ private Table buildPartitionedTable(String desc,
PartitionSpec spec, String udf,
return Lists.newArrayList(
record(schema, 0L, parse("2017-12-22T09:20:44.294658+00:00"),
"junction"),
record(schema, 1L, parse("2017-12-22T07:15:34.582910+00:00"),
"alligator"),
- record(schema, 2L, parse("2017-12-22T06:02:09.243857+00:00"),
"forrest"),
+ record(schema, 2L, parse("2017-12-22T06:02:09.243857+00:00"), ""),
Review comment:
Also, for reference, I did make several separate tests and they all
passed. But it just seems cleaner to me to introduce more common edge cases
into the existing tests cases. I'm surprised that the tests that run on random
data hasn't caught this before. But the tests in `TestFilteredScan` all go over
the code paths that deal with this. And after we merge this, I can add in two
small / inexpensive tests in the `NOT_STARTS_WITH` pr that will more explicitly
cover this edge case - by partitioning a DF using a filter of `data like '%'`
and `data not like '%'`, which previously threw an exception due to the
truncation length until this change is added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
