zhengruifeng commented on code in PR #46012:
URL: https://github.com/apache/spark/pull/46012#discussion_r1562547428
##########
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SessionHolder.scala:
##########
@@ -381,6 +405,53 @@ case class SessionHolder(userId: String, sessionId:
String, session: SparkSessio
*/
private[connect] val pythonAccumulator: Option[PythonAccumulator] =
Try(session.sparkContext.collectionAccumulator[Array[Byte]]).toOption
+
+ /**
+ * Transform a relation into a logical plan, using the plan cache if enabled.
+ * The plan cache is enable only if
`spark.connect.session.planCache.maxSize` is greater than zero
+ * AND `spark.connect.session.planCache.enabled` is true.
+ * @param rel The relation to transform.
+ * @param cachePlan Whether to cache the result logical plan.
+ * @param transform Function to transform the relation into a logical plan.
+ * @return The logical plan.
+ */
+ private[connect] def usePlanCache(rel: proto.Relation, cachePlan: Boolean)(
Review Comment:
I think we may want to exclude some kinds of plans:
1, LOCAL_RELATION, its size might be large;
2, CACHED_REMOTE_RELATION, it is already cached in `dataFrameCache` in the
`SessionHolder`;
3, CATALOG, it may be a CREATE_TABLE command, should not be skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
