Gengliang Wang created SPARK-61908:
--------------------------------------

             Summary: Support continuous processing for native data sources
                 Key: SPARK-61908
                 URL: https://issues.apache.org/jira/browse/SPARK-61908
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 5.0.0
            Reporter: Gengliang Wang


A streaming query with {{Trigger.Continuous}} that reads from or writes to a 
native data source fails when it starts: version 1 of the interface 
({{spark_datasource.h}}) only has micro-batch reads and the commits of the 
epochs of micro-batches.

Proposal: add continuous reads and writes to the interface within version 1, 
for the data sources that declare a capability for them, and expose them in the 
Rust SDK (SPARK-61282).




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to