Eric Marnadi created SPARK-60270:
------------------------------------
Summary: Preserve streaming source names for table reads in Spark
Connect
Key: SPARK-60270
URL: https://issues.apache.org/jira/browse/SPARK-60270
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 5.0.0
Reporter: Eric Marnadi
In Apache Spark, Spark Connect Python and Scala
readStream.name("orders_source").table("db.orders") do not transmit the source
name because Read.NamedTable has no source_name field. Named load() already
transmits DataSource.source_name, and Classic table() already propagates names
through NamedStreamingRelation. Add an optional NamedTable source_name field,
propagate it through the Connect clients and server planner, and add regression
coverage. This issue concerns only code in the public apache/spark repository.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
