[
https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069528#comment-16069528
]
Amit Baghel commented on SPARK-21249:
-------------------------------------
[~hyukjin.kwon] Sorry for this jira post as I couldn't decide if the type
should be bug or question. I have updated the jira description with all cases I
tried for File Sink with mapGroupsWithState.I will post this to mailing list.
Thanks.
> Is it possible to use File Sink with mapGroupsWithState in Structured
> Streaming?
> --------------------------------------------------------------------------------
>
> Key: SPARK-21249
> URL: https://issues.apache.org/jira/browse/SPARK-21249
> Project: Spark
> Issue Type: Question
> Components: Structured Streaming
> Affects Versions: 2.2.0
> Reporter: Amit Baghel
> Priority: Minor
>
> I am working with 2.2.0-SNAPSHOT and Structured Streaming. Is it possible to
> use File Sink with mapGroupsWithState? I verified below cases and getting
> exception in all.
> * Aggregation without watermark and Append mode
> {code}
> Exception in thread "main" org.apache.spark.sql.AnalysisException: Append
> output mode not supported when there are streaming aggregations on streaming
> DataFrames/DataSets without watermark;;
> {code}
> * Aggregation with watermark and Append mode
> {code}
> Exception in thread "main" org.apache.spark.sql.AnalysisException:
> mapGroupsWithState is not supported with aggregation on a streaming
> DataFrame/Dataset;;
> {code}
> * No Aggregation and Append mode
> {code}
> Exception in thread "main" org.apache.spark.sql.AnalysisException:
> mapGroupsWithState is not supported with Append output mode on a streaming
> DataFrame/Dataset;;
> {code}
> * No Aggregation and Update mode
> {code}
> Exception in thread "main" org.apache.spark.sql.AnalysisException: Data
> source parquet does not support Update output mode;
> {code}
> * No Aggregation and Complete mode
> {code}
> Exception in thread "main" org.apache.spark.sql.AnalysisException: Data
> source parquet does not support Complete output mode;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
