SEPURI-SAI-KRISHNA opened a new pull request, #12826:
URL: https://github.com/apache/seatunnel/pull/12826
### Purpose of this pull request
Closes #12825. With `parquet_avro_write_timestamp_as_int96 = true`, the file
sink wrote INT96 timestamps 543 years in the future under a Thai-locale JVM,
and the file on disk was permanently wrong with no error.
`ParquetWriteStrategy` built the INT96 value through
`Calendar.getInstance(TimeZone.getTimeZone("UTC"))`. That call consults the
default **locale** as well as the time zone, and the locale selects the
calendar system: under a Thai default locale it returns a `BuddhistCalendar`,
whose `get(Calendar.YEAR)` is the Buddhist year. That value was then handed to
`LocalDate.of(...)` as a Gregorian year.
### The change
One argument plus one import:
```java
Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"),
Locale.ROOT);
```
A comment records why `Locale.ROOT` is load-bearing rather than cosmetic, so
it is not removed later as noise. Nothing else changes: the time zone stays
UTC, the julian-day and time-of-day arithmetic is untouched, and no option,
schema or file layout is affected. Correctly written files are unaffected,
because `Locale.ROOT` gives the same `GregorianCalendar` that an `en_US` JVM
already produced.
### Does this PR introduce any user-facing change?
No new option or behaviour to configure. Jobs running on a
Gregorian-calendar locale are bit-for-bit unchanged. Jobs running on a Thai or
Japanese-imperial locale stop writing corrupted timestamps. No `docs/en` or
`docs/zh` change is needed, and this is a defect fix rather than an
incompatibility, so `incompatible-changes.md` is not touched.
### How was this patch tested?
A new test in the existing `ParquetWriteStrategyTest`, alongside the two
INT96 tests already there rather than in a new class. It writes
`2024-03-15T13:45:30` through `ParquetWriteStrategy` and reads it back through
`ParquetReadStrategy` under `en_US` and then under `th_TH`, restoring the
previous default locale in a `finally`, and asserts both round trips are exact.
Mutation check, since a test that never fails proves nothing. Removing
`Locale.ROOT` again:
```
Tests run: 4, Failures: 1
ParquetWriteStrategyTest.testParquetWriteInt96IsUnaffectedByANonGregorianDefaultLocale
expected: <2024-03-15T13:45:30> but was: <2567-03-15T13:45:30>
```
**Exactly one of the four fails, and it is the new one.** The two existing
INT96 tests pass against the defect, which is why it survived: they assert the
parquet schema types and write `LocalDateTime.now()`, never comparing a
round-tripped value.
Module run, `connector-file-base`: **337 tests, 0 failures, 14 errors**. The
same **14 errors occur on unmodified `dev` with 336 tests**, so this change
adds one test and no new error. They are environmental on this machine: two
`HadoopLoginFactoryTest` cases cannot get a Kerberos realm, and the rest are
`ParquetReadStrategyTest` cases failing on the Snappy native library
(`FAILED_TO_LOAD_NATIVE`) and the `NoClassDefFound` that cascades from it.
`spotless:check` passes.
No E2E test is added or updated: `int96` appears nowhere under
`seatunnel-e2e`, so there is no existing IT for this flag to extend, and the
unit test sits where the other INT96 coverage already lives.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according [New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
