[
https://issues.apache.org/jira/browse/IMPALA-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17265775#comment-17265775
]
ASF subversion and git services commented on IMPALA-10426:
----------------------------------------------------------
Commit 696dafed662cf7ef06b2a84bb4622eab4863f717 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=696dafe ]
IMPALA-10426: Fix crash when inserting invalid timestamps
Insertion of invalid timestamps causes Impala to crash when it uses
the INT64 Parquet timestamp types.
This patch fixes the error by checking for null values in
Int64TimestampColumnWriterBase::ConvertValue().
Testing:
* added e2e tests
Change-Id: I74fb754580663c99e1d8c3b73f8d62ea3305ac93
Reviewed-on: http://gerrit.cloudera.org:8080/16951
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Impala crashes when it tries to write invalid timestamp value with INT64
> Parquet timestamp type
> -----------------------------------------------------------------------------------------------
>
> Key: IMPALA-10426
> URL: https://issues.apache.org/jira/browse/IMPALA-10426
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Blocker
> Labels: crash, newbie, ramp-up
>
> *Reproduction*
> {noformat}
> set parquet_timestamp_type=INT64_MICROS;
> create table ts_test (ts timestamp) stored as parquet;
> insert into ts_test values ('21-01-07 10:53:47.224885000');
> {noformat}
> *Stack trace*
> {noformat}
> #0 0x00007ff56e752438 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x00007ff56e75403a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x00007ff571486789 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #3 0x00007ff57164b2c0 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #4 0x00007ff57149062f in JVM_handle_linux_signal () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #5 0x00007ff571483488 in ?? () from
> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
> #6 <signal handler called>
> #7 0x0000000001daaf36 in boost::date_time::date<boost::gregorian::date,
> boost::gregorian::gregorian_calendar,
> boost::gregorian::date_duration>::is_not_a_date (this=0x8)
> at
> /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:117
> #8 0x0000000001daaec0 in boost::date_time::date<boost::gregorian::date,
> boost::gregorian::gregorian_calendar,
> boost::gregorian::date_duration>::is_special (this=0x8)
> at
> /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:112
> #9 0x0000000001daa36a in impala::TimestampValue::HasDate (this=0x0) at
> /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:182
> #10 0x0000000001daa3a6 in impala::TimestampValue::HasDateAndTime (this=0x0)
> at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:184
> #11 0x0000000002bbf180 in impala::TimestampValue::FloorUtcToUnixTimeMicros
> (this=0x0, unix_time_micros=0x11b711c0) at
> /home/boroknagyz/Impala/be/src/runtime/timestamp-value.inline.h:137
> #12 0x0000000002eae9d3 in
> impala::HdfsParquetTableWriter::Int64MicroTimestampColumnWriter::ConvertTimestamp
> (this=0x11b70f80, ts=..., result=0x11b711c0)
> at
> /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:642
> #13 0x0000000002eae870 in
> impala::HdfsParquetTableWriter::Int64TimestampColumnWriterBase::ConvertValue
> (this=0x11b70f80, value=0x0)
> at
> /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:601
> #14 0x0000000002eaeacd in
> impala::HdfsParquetTableWriter::BaseColumnWriter::AppendRow (this=0x11b70f80,
> row=0xcdc4000) at
> /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:664
> #15 0x0000000002ea6d00 in impala::HdfsParquetTableWriter::AppendRows
> (this=0xe1f2e00, batch=0x10518700, row_group_indices=...,
> new_file=0x7ff4bf5a301f)
> ...{noformat}
> At frame #14 the timestamp value in the row is NULL, so ConvertValue() is
> invoked with 'value' being NULL which is not handled well.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
