[
https://issues.apache.org/jira/browse/SPARK-49409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-49409:
-----------------------------------
Labels: pull-request-available (was: )
> CONNECT_SESSION_PLAN_CACHE_SIZE is too small for certain programming patterns
> -----------------------------------------------------------------------------
>
> Key: SPARK-49409
> URL: https://issues.apache.org/jira/browse/SPARK-49409
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 4.0.0
> Reporter: Changgyoo Park
> Priority: Major
> Labels: pull-request-available
>
> Example:
>
> ```
> df_1 = df_a.filter(col('X').isNotNull())
> df_2 = df_b.filter(col('SAFE_SU_Conv').isNotNull())
> ....
> df_x = ...
> for _ in range(0, 5):
> df_x = df_x.select(...)
> ...
> df_3 = df_1.join(df_2, ...)
> ```
> => df_x completely invalidates all the cached entries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
