This is an automated email from the ASF dual-hosted git repository.

HyukjinKwon pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


    from 1d3149fc1b90 [MINOR][PYTHON] Reject empty memory strings with 
ValueError
     add a11fd253b5a3 [SPARK-60734][SS] Prune the columns that the scans of 
Data Source V2 streaming sources read

No new revisions were added by this update.

Summary of changes:
 .../src/main/resources/error/error-conditions.json |   12 +
 .../sql/connector/catalog/TableCapability.java     |   22 +-
 .../read/SupportsPushDownRequiredColumns.java      |    6 +
 .../spark/sql/errors/QueryExecutionErrors.scala    |   23 +
 .../datasources/v2/DataSourceV2Relation.scala      |   26 +-
 .../org/apache/spark/sql/internal/SQLConf.scala    |   17 +
 .../sql/connector/catalog/InMemoryBaseTable.scala  |   99 +-
 .../datasources/v2/DataSourceV2Strategy.scala      |   41 +-
 .../datasources/v2/columnar/ColumnarTable.scala    |   12 +-
 .../streaming/checkpointing/OffsetSeq.scala        |   16 +
 .../streaming/continuous/ContinuousExecution.scala |   28 +-
 .../streaming/runtime/MicroBatchExecution.scala    |   25 +-
 .../runtime/StreamingV2ScanPushDown.scala          |  498 ++++++++
 .../v2/columnar/ColumnarDataSourceSuite.scala      |   25 +
 .../execution/streaming/OffsetSeqLogSuite.scala    |   26 +
 .../StreamingV2ScanPushDownRealTimeSuite.scala     |  159 +++
 .../streaming/StreamingV2ScanPushDownSuite.scala   | 1325 ++++++++++++++++++++
 17 files changed, 2314 insertions(+), 46 deletions(-)
 create mode 100644 
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/StreamingV2ScanPushDown.scala
 create mode 100644 
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingV2ScanPushDownRealTimeSuite.scala
 create mode 100644 
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingV2ScanPushDownSuite.scala


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to