GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/3174
[FLINK-5576] extend deserialization functions of KvStateRequestSerializer
to detect unconsumed bytes
`KvStateRequestSerializer#deserializeValue()` deserializes a given byte
array. This is used by clients and unit tests and it is fair to assume that
these byte arrays represent a complete value since we do not offer a method to
continue reading from the middle of the array anyway. Therefore, we can treat
unconsumed bytes as errors, e.g. from a wrong serializer being used, and throw
an `IOException` with an appropriate failure message.
Similarly, this adds a better failure message to exceptions in
`KvStateRequestSerializer#deserializeList()` by wrapping the original
`IOException` into a new one with an appropriate error message just as in #3172.
The new unit tests require #3171 to be accepted first on which this PR is
also based.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-5576
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3174.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3174
----
commit e29d5ef3099d5f5ffc817f3e5cdbb79c523b34c7
Author: Nico Kruber <[email protected]>
Date: 2017-01-18T17:52:57Z
[FLINK-5561] fix DataInputDeserializer#available() 1 smaller than correct
commit e589058e92d1f2a57d242987327777baa1fe64d4
Author: Nico Kruber <[email protected]>
Date: 2017-01-19T16:06:00Z
[FLINK-5576] let KvStateRequestSerializer#deserializeValue() detect
unconsumed bytes
KvStateRequestSerializer#deserializeValue deserializes a given byte array.
This is used by clients and unit tests and it is fair to assume that these byte
arrays represent a complete value since we do not offer a method to continue
reading from the middle of the array anyway. Therefore, we can treat unconsumed
bytes as errors, e.g. from a wrong serializer being used, and throw a
IOException with an appropriate failure message.
commit 12d2108333b23e01b7b978a08e868a5f888506f9
Author: Nico Kruber <[email protected]>
Date: 2017-01-19T16:10:54Z
[FLINK-5576] add a better failure message to exceptions in
KvStateRequestSerializer#deserializeList()
As in FLINK-5559, wrap the original IOException into a new one with an
appropriate error message to better diagnose it.
commit 9770e7f3269adfa5b80dd2125bcdbf8cebc4dab8
Author: Nico Kruber <[email protected]>
Date: 2017-01-19T16:12:02Z
[FLINK-5576] add more unit tests for
KvStateRequestSerializer#deserializeList()
These tests ensure that some special cases not properly tested before are
handled correctly in future.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
