[
https://issues.apache.org/jira/browse/SAMZA-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250599#comment-15250599
]
Jake Maes commented on SAMZA-938:
---------------------------------
Taking a step back, it seems what SAMZA-813 was really trying to accomplish is
to optimize performance for using RocksDB as a queue.
Unfortunately, the patch exposed new methods in all the interfaces and store
implementations that are only actually functional in the RocksDB implementation
(and really only needed for a store that has the same tombstoning/compaction
semantics as RocksDB). I personally don't think this is the way to go, even for
a major release where backward incompatible changes are permissible. Even if we
split the new methods into a separate interface, all the extension classes
(LoggedStore, CachedStore) would need to have a bunch of "instanceof" checks to
avoid calling the method on instances that only implement the old interfaces.
Interestingly, there were some suggestions for the queue use case posted last
year:
https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB
>From those suggestions, it seems we could implement a subclass of
>RocksDBKeyValueStore which employs those optimizations without changing the KV
>store interfaces. Namely, it could persist the start/end SCNs of the queue and
>the iterator would only iterate over that range. We also might want to enforce
>that only the most recently accessed key is deleted (to enforce queue
>semantics). This way, users can interact with the store just like any other
>KVStore, primarily using the existing put(), delete(), and iterate() methods.
>I think this is much cleaner.
I think it will be a fair amount of work to implement and test this queue
implementation though, so we could either bite the bullet now or back out the
SAMZA-813 patch and revisit the suggested solution after the release.
@yipan, thoughts? Do you agree on the suggestion for a queue-oriented store? Do
you think we should prioritize it now, or save it for after the release?
> KV-store API changes should be backward compatible to 0.10
> ----------------------------------------------------------
>
> Key: SAMZA-938
> URL: https://issues.apache.org/jira/browse/SAMZA-938
> Project: Samza
> Issue Type: Bug
> Affects Versions: 0.10.1
> Reporter: Yi Pan (Data Infrastructure)
> Assignee: Jake Maes
> Fix For: 0.10.1
>
>
> SAMZA-813 introduced new methods in KeyValueStore and KeyValueIterator
> interface classes, which in JDK7, breaks the backward compatibility for
> customized implementation of KeyValueStore and KeyValueIterator.
> As 0.10.1 is a bug-fix release, we should make sure that the backward
> compatibility is retained. Hence, it would be better to have the extended
> interface classes to implement the seekable functions, instead of changing
> the existing classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
