Ethan Rose created HDDS-3976:
--------------------------------
Summary: KeyValueBlockIterator#nextBlock skips valid blocks
Key: HDDS-3976
URL: https://issues.apache.org/jira/browse/HDDS-3976
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Ethan Rose
Assignee: Ethan Rose
HDDS-3854 fixed a bug in KeyValueBlockIterator#hasNext, but introduced another
one in KeyValueBlockIterator#nextBlock, which depends on the behavior of that
method. When the first key encountered does not pass the filter, the internal
nextBlock field is never intialized. Then a call to nextBlock() results in call
to hasNext() which returns true, which recursively calls nextBlock() etc until
the end of the set is reached and an exception is thrown. This skips all valid
keys that may occur past the first invalid key.
This bug was identified while working on HDDS-3869, which adds a strong typing
layer before objects are serialized into RocksDB for datanode. Due to RocksDB
internals, this changes the database layout so that all prefixed keys are
returned at the beginning of the key set, instead of in the end. Since the
original layout returned all prefixed keys at the end of the key set, this bug
was not evident in any of the original unit tests, since the behavior described
above could not occur.
Additionally, the current implementation modifies the internal iterator's state
on hasNext() calls, so multiple consecutive hasNext() calls will actually skip
over elements. While this is not necessarily a bug, it is atypical iterator
behavior and should be resolved to prevent user error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
