[
https://issues.apache.org/jira/browse/SAMZA-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165627#comment-14165627
]
Chris Riccomini commented on SAMZA-424:
---------------------------------------
bq. Yeah with respect to logging it is still a bit dubious, right?
My personal opinion is that logging for caches is something that we should just
forget about. It's an optimization, but the complexity that it introduces is
not worth the trade-off.
bq. I agree that having a mechanism to share objects would be very useful but
this isn't specific to caching right?
This is precisely why I wanted to back off on the isShared discussion above,
and worry about the wiring/object sharing as part of SAMZA-40.
bq. I see this more as a generalization of init() but for the container not the
task with a mechanism to somehow stash objects somewhere where you could
retrieve them in the task or in the task init.
The way I see wiring happening is that there are somethings that are 1 per job,
some that are 1 per container, and some that are 1 per task. The "shared cache"
is an object that should be wired and shared at the container level. This
hierarchy seems pretty elegant to me, and I feel that there's likely a way that
we can make the wiring reflect this in a programmatic way. I don't have a
concrete API proposal, though. In any case, I think it should be punted on for
now, and dealt with in SAMZA-40.
So, given that I think we should not do logged caches, and that shared caches
should be addressed as part of SAMZA-40, I can't find a really compelling
reason to bake this into the framework either. We can survive with app-level
caching for now. I think this is now the 3rd time that I've changed my mind. :/
Sorry, hehe.
As an aside: a lot of the friction we're hitting here seems to stem from the
fact that we have other broken windows in the framework that we need to fix
before this feature becomes viable (SAMZA-40, SAMZA-348, SAMZA-346). I'd
recommend focusing on those, and re-visiting this afterwards.
> Add a Cache state API to the Samza container
> --------------------------------------------
>
> Key: SAMZA-424
> URL: https://issues.apache.org/jira/browse/SAMZA-424
> Project: Samza
> Issue Type: New Feature
> Components: container
> Reporter: Chinmay Soman
> Assignee: Chinmay Soman
> Attachments: SAMZA-424-Cache-API_0.pdf, SAMZA-424-Cache-API_1.md,
> SAMZA-424-Cache-API_2.md, SAMZA-424-Cache-API_2.pdf, samza-424-cache-api_1.pdf
>
>
> There are cases when the user code needs access to a 'cache' which can be
> used to store custom data. This cache is different from the KeyValue store in
> the following ways:
> * At the very least Needs to support LRU (Least Recently Used) and TTL (Time
> To Live) eviction strategies
> * May not support all() and range() operations (since this wreaks havoc with
> the eviction operation)
> * Needs to exist at a per task or a per container level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
