cloud-fan commented on code in PR #54108:
URL: https://github.com/apache/spark/pull/54108#discussion_r2757288732
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/memory.scala:
##########
@@ -43,36 +43,42 @@ import
org.apache.spark.sql.internal.connector.SimpleTableProvider
import org.apache.spark.sql.types.StructType
import org.apache.spark.sql.util.CaseInsensitiveStringMap
-object MemoryStream extends LowPriorityMemoryStreamImplicits {
+object MemoryStream {
protected val currentBlockId = new AtomicInteger(0)
protected val memoryStreamId = new AtomicInteger(0)
- def apply[A : Encoder](implicit sparkSession: SparkSession): MemoryStream[A]
=
- new MemoryStream[A](memoryStreamId.getAndIncrement(), sparkSession)
-
- def apply[A : Encoder](numPartitions: Int)(implicit sparkSession:
SparkSession): MemoryStream[A] =
- new MemoryStream[A](memoryStreamId.getAndIncrement(), sparkSession,
Some(numPartitions))
-}
-
-/**
- * Provides lower-priority implicits for MemoryStream to prevent ambiguity
when both
- * SparkSession and SQLContext are in scope. The implicits in the companion
object,
- * which use SparkSession, take higher precedence.
- */
-trait LowPriorityMemoryStreamImplicits {
- this: MemoryStream.type =>
-
- // Deprecated: Used when an implicit SQLContext is in scope
- @deprecated("Use MemoryStream.apply with an implicit SparkSession instead of
SQLContext", "4.1.0")
- def apply[A: Encoder]()(implicit sqlContext: SQLContext): MemoryStream[A] =
Review Comment:
This is the problem. This is not backward compatible, as the previous API is
`def apply[A: Encoder](implicit sqlContext: SQLContext): MemoryStream[A]` (no
parentheses).
There is no way to keep both implicits. So the proposal here is to only keep
implicit `SQLContext`, and require to pass `SparkSession` implicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
