Kousuke Saruta created SPARK-19913: -------------------------------------- Summary: Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach Key: SPARK-19913 URL: https://issues.apache.org/jira/browse/SPARK-19913 Project: Spark Issue Type: Improvement Components: Structured Streaming Affects Versions: 2.2.0 Reporter: Kousuke Saruta Priority: Minor
When batches are executed with memory, console or foreach format, `assertNotPartitioned` will check whether output is not partitioned and throw AnalysisException in case it is. But I wonder it's better to log warning rather than throw the exception because partitioning does not affect output for those formats but also does not bring any negative impacts. Also, this assertion is not applied when the format is `console`. I think in this case too, we should assert that . By fixing them, we can easily switch the format to memory or console for debug purposes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org