[
https://issues.apache.org/jira/browse/SPARK-60807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60807:
-----------------------------------
Labels: pull-request-available (was: )
> Restore offloaded ML model summaries during fetch
> -------------------------------------------------
>
> Key: SPARK-60807
> URL: https://issues.apache.org/jira/browse/SPARK-60807
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 5.0.0
> Reporter: Ruifeng Zheng
> Priority: Major
> Labels: pull-request-available
>
> Fetching a Spark Connect ML training-summary attribute can fail with
> CONNECT_ML.MODEL_SUMMARY_LOST after model offloading.
> The Python client recovers by sending CreateSummary and then retrying FETCH
> once. The in-memory ML cache uses soft values, so an entry can be evicted
> again between those RPCs. The retry then reloads a model without its summary.
> Relation FETCH has a related gap: it restores the summary and then performs
> another cache lookup through AttributeHelper.
> A deterministic reproduction is to evict a fitted model before each command
> FETCH, or configure the server ML cache with a zero in-memory size limit.
> Summary access should succeed while the model remains available on disk.
> Restore the summary and evaluate the attribute on the same strongly
> referenced model instance within one FETCH request. The command should carry
> the summary dataset, as relation FETCH already does, while retaining
> compatibility with older clients and servers.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
