Csaba Ringhofer created IMPALA-8180:
---------------------------------------
Summary: Change Kudu timestamp writer to round towards minus
infinity
Key: IMPALA-8180
URL: https://issues.apache.org/jira/browse/IMPALA-8180
Project: IMPALA
Issue Type: New Feature
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Csaba Ringhofer
Fix For: Impala 4.0
Kudu timestamps are microseconds since Unix epoch stored as int64, so Impala
has to round its nanosecond timestamps before writing them to Kudu tables.
Currently this is done by rounding to the nearest microsecond. Meanwhile Hive
uses rounding towards minus infinity when reducing the precision of timestamps,
which is a better way in my opinion, because it cannot move a timestamp into a
different day, and should be also a bit faster.
Changing the rounding method is breaking change, so I would only do this in the
next major release.
Example:
create table tkudu (id int primary key, t timestamp) stored as kudu;
insert into tkudu values
(1,"1970-01-01 00:00:00.1111111"), -- all sub-second parts are 7 digit
(2,"1970-01-01 23:59:59.9999999"),
(3,"1969-12-31 23:59:59.9999999");
select * from tkudu;
This currently returns:
1,1970-01-01 00:00:00.111111000
2,1970-01-02 00:00:00
3,1970-01-01 00:00:00
1 was rounded down to microsec precision, while 2 and 3 were rounded up and
also stepped to another day.
If the table was written using rounding toward minus infinity, then the query
would return this:
1,1970-01-01 00:00:00.111111000
2,1970-01-01 23:59:59.999999000
3,1969-12-31 23:59:59.999999000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
