[
https://issues.apache.org/jira/browse/PHOENIX-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299341#comment-16299341
]
James Taylor commented on PHOENIX-4382:
---------------------------------------
The fix for the new serialization is fine, but I'd still like to understand
completely when this issue occurs (and when we can work around it based on the
type information we have).
Is this correct (for the old scheme)?
- we (incorrectly) store {separatorByte, # of nulls} for fixed length types
- for a SMALLINT (i.e. 2 byte short), we can't distinguish between this
separator and null
For IMMUTABLE_STORAGE_SCHEME we know the table schema, so when I mention "fixed
length types", I mean the type as defined in the table schema. Do we use the
type information to help us disambiguate and if not, would it help if we did?
> Immutable table SINGLE_CELL_ARRAY_WITH_OFFSETS values starting with separator
> byte return null in query results
> ---------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-4382
> URL: https://issues.apache.org/jira/browse/PHOENIX-4382
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.14.0
> Reporter: Vincent Poon
> Assignee: Vincent Poon
> Attachments: PHOENIX-4382.v1.master.patch,
> PHOENIX-4382.v2.master.patch, UpsertBigValuesIT.java
>
>
> For immutable tables, upsert of some values like Short.MAX_VALUE results in a
> null value in query resultsets. Mutable tables are not affected. I tried
> with BigInt and got the same problem.
> For Short, the breaking point seems to be 32512.
> This is happening because of the way we serialize nulls. For nulls, we write
> out [separatorByte, #_of_nulls]. However, some data values, like
> Short.MAX_VALUE, start with separatorByte, we can't distinguish between a
> null and these values. Currently the code assumes it's a null when it sees a
> leading separatorByte, hence the incorrect query results.
> See attached test - testShort() , testBigInt()
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
