[
https://issues.apache.org/jira/browse/KAFKA-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746102#comment-17746102
]
Guozhang Wang commented on KAFKA-15215:
---------------------------------------
[~ableegoldman] Thanks for bringing this up! I agree that it would be very
beneficial to customized stores with a configurable store type, what we need to
figure out though is how to ask users to provide the customized store impls.
Some quick options on top of my head:
1. Ask users to "register" a set of customized stores, one for each store type
(i.e. kv, time-ranged, versioned, etc) with a naming convention like a prefix
so that if the dsl.store configured as "XYZ", the runtime would try to find
such impl classes named as "XYZ.keyvalue.." in class loading.
2. Provide a "StoreImplSelector" API to extend (or replace) the config-based
"default.dsl.store". Its default implementation could still follow the config
value, to return the corresponding in-mem / rocksDB impl classes given the
store type; users can override this API to select new impl classes of their own
(again, potentially still rely on the config with new config values; and if the
corresponding config name is not expected, then error out).
Personally I feel just among these two, the second option seems a clear winner
--- of course there might be other smarter options as we brainstorm more ---
since it a) would not enforce users to provide one impl classes for each
possible store type, b) also as a quality bar to restrict us trying to add more
store types casually in the future as it will impact all the API instantiations.
> The default.dsl.store config is not compatible with custom state stores
> -----------------------------------------------------------------------
>
> Key: KAFKA-15215
> URL: https://issues.apache.org/jira/browse/KAFKA-15215
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Assignee: Almog Gavra
> Priority: Major
> Labels: needs-kip
>
> Sort of a bug, sort of a new/missing feature. When we added the long-awaited
> default.dsl.store config, it was decided to scope the initial KIP to just the
> two out-of-the-box state stores types offered by Streams, rocksdb and
> in-memory. The reason being that this would address a large number of the
> relevant use cases, and could always be followed up with another KIP for
> custom state stores if/when the demand arose.
> Of course, since rocksdb is the default anyways, the only beneficiaries of
> this KIP right now are the people who specifically want only in-memory stores
> – yet custom state stores users are probably by far the ones with the
> greatest need for an easier way to configure the store type across an entire
> application. And unfortunately, because the config currently relies on enum
> definitions for the known OOTB store types, there's not really any way to
> extend this feature as it is to work with custom implementations.
> I think this is a great feature, which is why I hope to see it extended to
> the broader user base. Most likely we'll want to introduce a new config for
> this, though whether it replaces the old default.dsl.store config or
> complements it will have to be decided during the KIP discussion
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
