zhengruifeng opened a new pull request, #50013:
URL: https://github.com/apache/spark/pull/50013
### What changes were proposed in this pull request?
Introduce model size estimation to control ml cache
### Why are the changes needed?
We should control the memory usage of cached ml models
1, When `CONNECT_SESSION_ML_CACHE_SIZE>0`,
before training a new model, estimate its size, fail the training if the
estimated size > `CONNECT_SESSION_ML_CACHE_SIZE`
before caching a new model, fail if `model.estimatedSize >
CONNECT_SESSION_ML_CACHE_SIZE`;
2, When `CONNECT_SESSION_ML_CACHE_TOTAL_ITEM_SIZE>0`,
set the `maximumWeight` of the cache, and apply the model size as the
`weighter`, the guava cache will evict old model (because it hasn't been used
recently or very often) when necessary.
3, Add config `CONNECT_SESSION_ML_CACHE_SIZE` to control the number of
cached items, it only take effect when
`CONNECT_SESSION_ML_CACHE_TOTAL_ITEM_SIZE<=0`, because guava cache doesn't
support set `maximumWeight` and `maximumSize` together.
4, Add config `CONNECT_SESSION_ML_CACHE_TIMEOUT`, to evict cached item when
a duration has elapsed after the creation or last access.
### Does this PR introduce _any_ user-facing change?
Only for ML connect
### How was this patch tested?
added tests
### Was this patch authored or co-authored using generative AI tooling?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
