[
https://issues.apache.org/jira/browse/IMPALA-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-10839 started by Daniel Becker.
----------------------------------------------
> NULL values are displayed on a wrong level for nested structs (ORC)
> -------------------------------------------------------------------
>
> Key: IMPALA-10839
> URL: https://issues.apache.org/jira/browse/IMPALA-10839
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Gabor Kaszab
> Assignee: Daniel Becker
> Priority: Major
> Labels: ORC, complextype, correctness, nested_types, scanner
>
> When querying a non-toplevel nested struct then the NULL values are displayed
> in an incorrect level. E.g.:
> {code:java}
> select id, outer_struct.inner_struct3 from
> functional_orc_def.complextypes_nested_structs where id >= 4;
> {code}
> {code:java}
> +----+----------------------------+
> | id | outer_struct.inner_struct3 |
> +----+----------------------------+
> | 4 | {"s":{"i":null,"s":null}} |
> | 5 | {"s":null} |
> +----+----------------------------+
> {code}
> However, here in the first row the expected would be that 's' is null and not
> its members and in the second line the result should be 'NULL'.
> For reference see what is returned when querying 'outer_struct' instead of
> 'outer_struct.inner_struct3':
> {code:java}
> +----+-------------------------------------------------------------------------------------------------------------------------------+
> | 4 |
> {"str":"","inner_struct1":{"str":"somestr2","de":12345.12},"inner_struct2":{"i":1,"str":"string"},"inner_struct3":{"s":null}}
> |
> | 5 |
> {"str":null,"inner_struct1":null,"inner_struct2":null,"inner_struct3":null}
> |
> +----+-------------------------------------------------------------------------------------------------------------------------------+
> {code}
> Note, this issues is with ORC format.
> After some digging I found that these incorrect null values are already
> present in the ORC scanner where OrcStructReader reads the rows in
> ReadValue() and ReadValueBatch() functions.
> As a first step it would be nice to verify that the external ORC reader we
> use for reading the actual values from the files gives correct results.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
