yadavay-amzn commented on PR #56951: URL: https://github.com/apache/spark/pull/56951#issuecomment-4888018849
Following the review discussion on #56944 and #56984 (thanks @uros-b), this PR now carries the consolidated window-struct-over-nanos watermark handling: the CheckAnalysis window-struct guard accepts a nanosecond end field, and the statefulOperators eviction struct-branch extracts endFieldType via watermarkLiteral. That handling depends on the watermark infrastructure from #56944 (SPARK-57830), so this PR is stacked on it - the diff shows #56944's commits until it merges, after which I will rebase onto master. Added e2e tests: window() over a nanosecond event-time column + withWatermark, asserting nanosecond window bounds and state eviction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
