martin-g commented on code in PR #18857:
URL: https://github.com/apache/datafusion/pull/18857#discussion_r2549663396
##########
datafusion/sqllogictest/src/util.rs:
##########
@@ -82,13 +82,36 @@ pub fn df_value_validator(
actual: &[Vec<String>],
expected: &[String],
) -> bool {
+ // Support ignore marker <slt:ignore> to skip volatile parts of output.
+ const IGNORE_MARKER: &str = "<slt:ignore>";
+ let contains_ignore_marker = expected.iter().any(|line|
line.contains(IGNORE_MARKER));
+
let normalized_expected =
expected.iter().map(normalizer).collect::<Vec<_>>();
let normalized_actual = actual
.iter()
.map(|strs| strs.iter().join(" "))
.map(|str| str.trim_end().to_string())
.collect_vec();
+ // If ignore marker present, perform fragment-based matching on the full
snapshot.
+ if contains_ignore_marker {
+ let expected_snapshot = normalized_expected.join("\n");
+ let actual_snapshot = normalized_actual.join("\n");
+ let fragments: Vec<&str> =
expected_snapshot.split(IGNORE_MARKER).collect();
+ let mut pos = 0;
+ for frag in fragments {
+ if frag.is_empty() {
+ continue;
+ }
+ if let Some(idx) = actual_snapshot[pos..].find(frag) {
+ pos += idx + frag.len();
+ } else {
+ return false;
Review Comment:
This early return would make it hard to debug. It would be nice to log a
warning with the reason, as done below (line 116+).
##########
datafusion/sqllogictest/README.md:
##########
@@ -142,6 +142,17 @@ select substr('Andrew Lamb', 1, 6), '|'
Andrew |
```
+## Cookbook: Ignoring volatile output
+
+Sometimes parts of a result change every run (timestamps, counters, etc). To
keep the rest of the snapshot checked in, replace those fragments with the
`<slt:ignore>` marker inside the expected block. During validation the marker
acts like a wildcard, so only the surrounding text must match.
Review Comment:
```suggestion
Sometimes parts of a result change every run (timestamps, counters, etc.).
To keep the rest of the snapshot checked in, replace those fragments with the
`<slt:ignore>` marker inside the expected block. During validation the marker
acts like a wildcard, so only the surrounding text must match.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
