bersprockets opened a new pull request #35430:
URL: https://github.com/apache/spark/pull/35430
### What changes were proposed in this pull request?
Add TimestampNTZType to UnsafeRow's list of mutable fields.
### Why are the changes needed?
Assume this data:
```
create or replace temp view v1 as
select * from values
(1, timestamp_ntz'2012-01-01 00:00:00', 10000),
(2, timestamp_ntz'2012-01-01 00:00:00', 20000),
(1, timestamp_ntz'2012-01-01 00:00:00', 5000),
(1, timestamp_ntz'2013-01-01 00:00:00', 48000),
(2, timestamp_ntz'2013-01-01 00:00:00', 30000)
as data(a, b, c);
```
The following query produces incorrect results:
```
select *
from v1
pivot (
sum(c)
for a in (1, 2)
);
```
The timestamp_ntz values are corrupted:
```
2012-01-01 19:05:19.476736 15000 20000
2013-01-01 19:05:19.476736 48000 30000
```
Because `UnsafeRow.isFixedLength` returns `false` for data type
`TimestampNTZType`, `GenerateUnsafeRowJoiner` generates code for the
`TIMESTAMP_NTZ` field as though it was a variable length field (it adds an
offset to the Long value, thus corrupting the timestamp value).
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
New unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
