Github user tdas commented on the issue:
https://github.com/apache/spark/pull/21220
@brkyvz Answers to your questions.
1. We have already fixed those emptyDF optimizations. The optimization only
quick in `df.isStreaming = false`, and emptyDFs generated by sources should be
have isStreaming = true. That's for v1 sources. For v2 sources, the engine
already takes care of that by making sure that
`StreamingDataSourceV2Relation.isStreaming` is true where
`StreamingDataSourceV2Relation` is the logical plan leaf inserted into the
micro-batch logical plan irrespective of empty or not.
2. I will do both type of timeouts in flatMapGroupsWithState in later PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
