HeartSaVioR commented on a change in pull request #31435:
URL: https://github.com/apache/spark/pull/31435#discussion_r568609672
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/DataSourceScanExecRedactionSuite.scala
##########
@@ -137,9 +137,24 @@ class DataSourceScanExecRedactionSuite extends
DataSourceScanRedactionTest {
assert(location.isDefined)
// The location metadata should at least contain one path
assert(location.get.contains(paths.head))
- // If the temp path length is larger than 100, the metadata length
should not exceed
- // twice of the length; otherwise, the metadata length should be
controlled within 200.
- assert(location.get.length < Math.max(paths.head.length, 100) * 2)
+
+ // The location metadata should have bracket wrapping paths
+ assert(location.get.indexOf('[') > -1)
+ assert(location.get.indexOf(']') > -1)
+
+ // extract paths in location metadata (removing classname, brackets,
separators)
+ val pathsInLocation = location.get.substring(
+ location.get.indexOf('[') + 1, location.get.indexOf(']')).split(",
").toSeq
+
+ // If the temp path length is less than (stop appending threshold - 1),
say, 100 - 1 = 99,
+ // location should include more than one paths. Otherwise location
should include only one
+ // path.
+ // (Note we apply subtraction with 1 to count start bracket '['.)
+ if (paths.head.length < 99) {
+ assert(pathsInLocation.size >= 2)
Review comment:
Yeah makes sense, but let's leave it as it is, as I think I'd propose
the another form which would make clear how many elements they are instead of
only showing available paths and don't say there're more.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
