[
https://issues.apache.org/jira/browse/SAMZA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837893#comment-13837893
]
Chris Riccomini commented on SAMZA-94:
--------------------------------------
The first question is: should we protect against this?
If so, how should we protect ourselves. Some potential solutions:
1. Test the serde before we use it (fromBytes(toBytes(null)) == null &&
toBytes(null) == null).
2. Check if a message is null before trying to (de)serialize it, and if it is,
skip the toBytes/fromBytes call, and just use null.
The thing about this behavior is that it's really kafka-specific. Kafka's log
compacted streams model deletes as messages with a null value. For example, if
you send k=a, v=1, and then send k=a, v=null, the second message is a signal to
delete all messages with k=a from the stream.
Need to think about this a bit.
> State management kv-store delete() does not remove key
> ------------------------------------------------------
>
> Key: SAMZA-94
> URL: https://issues.apache.org/jira/browse/SAMZA-94
> Project: Samza
> Issue Type: Bug
> Components: kv
> Affects Versions: 0.6.0
> Reporter: Alan Li
> Assignee: Chris Riccomini
> Fix For: 0.7.0
>
>
> Sample code snippet:
> {noformat}
> _realtimeStateStore.put("foo", new HashMap());
>
> KeyValueIterator<String, Map<String, State>> realtimeIterator =
> _realtimeStateStore.all();
> while (realtimeIterator.hasNext())
> {
> Entry<String, Map<String, State>> entry = realtimeIterator.next();
> System.out.println("read from leveldb: " + entry.getKey() + "=>" +
> entry.getValue());
> }
> realtimeIterator.close();
>
> _realtimeStateStore.delete("foo");
>
> KeyValueIterator<String, Map<String, State>> iter =
> _realtimeStateStore.all();
> while (iter.hasNext())
> {
> Entry<String, Map<String, State>> entry = iter.next();
> System.out.println("read from leveldb: " + entry.getKey() + "=>" +
> entry.getValue());
> }
> iter.close();
> {noformat}
> Output:
> read from leveldb: foo=>{}
> read from leveldb: foo=>null
> I would expect that the kv would remove both the key & value on delete so
> that subsequent iterators will not return keys that have been deleted.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
