[
https://issues.apache.org/jira/browse/IMPALA-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951405#comment-16951405
]
ASF subversion and git services commented on IMPALA-8498:
---------------------------------------------------------
Commit f65c2a754fe3011a17aa3bde093c117e57ffedcb in impala's branch
refs/heads/master from norbert.luksa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f65c2a7 ]
IMPALA-8498: Write column index for floating types when NaN is not present
IMPALA-7307 disabled column index writing for floating point columns
until PARQUET-1222 is resolved. However, the problematic values are
only the NaNs. Therefore we can write column index if NaNs are not
present in data.
Testing:
* Added tests which should fail if a column index is
present while having NaN values in the column.
Change-Id: Ic9d367500243c8ca142a16ebfeef6c841f013434
Reviewed-on: http://gerrit.cloudera.org:8080/14264
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Write column index for floating types when NaN is not present
> -------------------------------------------------------------
>
> Key: IMPALA-8498
> URL: https://issues.apache.org/jira/browse/IMPALA-8498
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Norbert Luksa
> Priority: Major
> Labels: ramp-up
>
> IMPALA-7304 disabled column index writing for floating point columns until
> PARQUET-1222 is resolved.
> PARQUET-1222 is responsible for defining a total order for floating values,
> but the problematic values are only the NaNs. Therefore we can write the
> column index if NaNs are not present in the data. Parquet-MR also does this,
> following the principles in
> [https://github.com/apache/parquet-format/blob/75eb7a7b84e6e62bfb09668b6d8d40b12597456e/src/main/thrift/parquet.thrift#L827-L834]
>
> Impala should follow this behavior, and also when storing zeroes, it should
> store -0.0 as minimum and +0.0 as maximum.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
