[
https://issues.apache.org/jira/browse/SPARK-40495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-40495.
----------------------------------
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 37936
[https://github.com/apache/spark/pull/37936]
> Add additional tests to StreamingSessionWindowSuite
> ---------------------------------------------------
>
> Key: SPARK-40495
> URL: https://issues.apache.org/jira/browse/SPARK-40495
> Project: Spark
> Issue Type: Test
> Components: Tests
> Affects Versions: 3.3.0
> Reporter: Wei Liu
> Assignee: Wei Liu
> Priority: Minor
> Fix For: 3.4.0
>
>
> Add complex tests to {{{}StreamingSessionWindowSuite{}}}. Concretely, I
> created two helper functions,
> * one is called {{{}sessionWindowQueryNestedKey{}}}, which would convert
> {{sessionId}} from the single word key used in {{sessionWindowQuery}} to a
> nested column key. For example: {{{}"hello" -> (("hello", "hello"),
> "hello"){}}}.
> * The other is called {{{}sessionWindowQueryMultiColKey{}}}. It would
> convert {{sessionId}} from the single word key used in {{sessionWindowQuery}}
> to two columns. For example: "hello" -> col1: ("hello", "hello"), col2:
> "hello"
> With the two new helper functions, I added more tests for the tests for
> {{complete mode}} and {{cap gap duration}} ({{{}append{}}} and {{async
> state}} was not included, because the first two is enough for testing the
> change I added). The logic of the tests are not changed at all, just the key.
> For the aggregation test ({{{}session window - with more aggregation
> functions{}}}), I added some more functions as well as a UDAF to test, and I
> tried {{first()}} and {{last()}} function on a nested triple, created using
> the same method as the above.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
