Steve Niemitz created BEAM-13081:
------------------------------------
Summary: Portable representation of "packed bitset indicating null
fields" in beam Row format is not compatible with jvm representations
Key: BEAM-13081
URL: https://issues.apache.org/jira/browse/BEAM-13081
Project: Beam
Issue Type: Bug
Components: cross-language
Reporter: Steve Niemitz
Assignee: Steve Niemitz
The JVM RowCoder strips trailing 0s from the null-value bitmap, while both
python and go expect all bits to be present in the encoded bitmap. This causes
index-out-of-range errors when trying to decode a row encoded on the JVM in
other languages in some circumstances.
For example, given a Row with 10 nullable fields, if the first 8 are null and
the last two are set, the row will fail to decode in python, because the
nullable bitmap will only have 1 byte, but the python coder expects 2.
As discussed in the thread, the best solution here is probably to change the
python (and go) coders to accept truncated nullable bitmaps.
More discussion here:
[https://lists.apache.org/thread.html/r2f148e29902bda8bb0ff7106fffb8a5494295450827ad7fd17289383%40%3Cdev.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
