Wei Liu created SPARK-40495:
-------------------------------
Summary: Add additional tests to StreamingSessionWindowSuite
Key: SPARK-40495
URL: https://issues.apache.org/jira/browse/SPARK-40495
Project: Spark
Issue Type: Test
Components: Tests
Affects Versions: 3.3.0
Reporter: Wei Liu
Add complex tests to {{{}StreamingSessionWindowSuite{}}}. Concretely, I created
two helper functions,
* one is called {{{}sessionWindowQueryNestedKey{}}}, which would convert
{{sessionId}} from the single word key used in {{sessionWindowQuery}} to a
nested column key. For example: {{{}"hello" -> (("hello", "hello"),
"hello"){}}}.
* The other is called {{{}sessionWindowQueryMultiColKey{}}}. It would convert
{{sessionId}} from the single word key used in {{sessionWindowQuery}} to two
columns. For example: "hello" -> col1: ("hello", "hello"), col2: "hello"
With the two new helper functions, I added more tests for the tests for
{{complete mode}} and {{cap gap duration}} ({{{}append{}}} and {{async state}}
was not included, because the first two is enough for testing the change I
added). The logic of the tests are not changed at all, just the key.
For the aggregation test ({{{}session window - with more aggregation
functions{}}}), I added some more functions as well as a UDAF to test, and I
tried {{first()}} and {{last()}} function on a nested triple, created using the
same method as the above.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
