Gengliang Wang created SPARK-61908:
--------------------------------------
Summary: Support continuous processing for native data sources
Key: SPARK-61908
URL: https://issues.apache.org/jira/browse/SPARK-61908
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 5.0.0
Reporter: Gengliang Wang
A streaming query with {{Trigger.Continuous}} that reads from or writes to a
native data source fails when it starts: version 1 of the interface
({{spark_datasource.h}}) only has micro-batch reads and the commits of the
epochs of micro-batches.
Proposal: add continuous reads and writes to the interface within version 1,
for the data sources that declare a capability for them, and expose them in the
Rust SDK (SPARK-61282).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]