[ https://issues.apache.org/jira/browse/SPARK-27977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876478#comment-16876478 ]
Alessandro D'Armiento commented on SPARK-27977: ----------------------------------------------- {{MicroBatchWriter}} implementation has been replaced with {{MicroBatchWrite. }} {{Is the issue still present?}} > MicroBatchWriter should use StreamWriter for human-friendly textual > representation (toString) > --------------------------------------------------------------------------------------------- > > Key: SPARK-27977 > URL: https://issues.apache.org/jira/browse/SPARK-27977 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Affects Versions: 3.0.0 > Reporter: Jacek Laskowski > Priority: Minor > > The following is a extended explain for a streaming query: > {code} > == Parsed Logical Plan == > WriteToDataSourceV2 > org.apache.spark.sql.execution.streaming.sources.MicroBatchWriter@4737caef > +- Project [value#39 AS value#0] > +- Streaming RelationV2 socket[value#39] (Options: > [host=localhost,port=8888]) > == Analyzed Logical Plan == > WriteToDataSourceV2 > org.apache.spark.sql.execution.streaming.sources.MicroBatchWriter@4737caef > +- Project [value#39 AS value#0] > +- Streaming RelationV2 socket[value#39] (Options: > [host=localhost,port=8888]) > == Optimized Logical Plan == > WriteToDataSourceV2 > org.apache.spark.sql.execution.streaming.sources.MicroBatchWriter@4737caef > +- Streaming RelationV2 socket[value#39] (Options: [host=localhost,port=8888]) > == Physical Plan == > WriteToDataSourceV2 > org.apache.spark.sql.execution.streaming.sources.MicroBatchWriter@4737caef > +- *(1) Project [value#39] > +- *(1) ScanV2 socket[value#39] (Options: [host=localhost,port=8888]) > {code} > As you may have noticed, {{WriteToDataSourceV2}} is followed by the internal > representation of {{MicroBatchWriter}} that is a mere adapter for > {{StreamWriter}}, e.g. {{ConsoleWriter}}. > It'd be more debugging-friendly if the plans included whatever > {{StreamWriter.toString}} would (which in case of {{ConsoleWriter}} would be > {{ConsoleWriter[numRows=..., truncate=...]}} which gives more context). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org