SEPURI-SAI-KRISHNA opened a new issue, #12975:
URL: https://github.com/apache/seatunnel/issues/12975

   ### Search before asking
   
   - [x] I searched the open and closed issues and pull requests and found no 
report of this. Queries included `FORMATDATETIME locale`, `FROM_UNIXTIME 
locale`, `DateTimeFormatter locale`, `ofPattern Locale`, `Locale.getDefault`, 
`locale dependent`, `month name language`, `Turkish locale`, `i18n date format` 
and `non-deterministic date format`, on both the issue search and the separate 
pull request search. The locale reports that exist (#11949, #12617, #12581, 
#12496) are all about case conversion of identifiers, a different mechanism. 
#12825 is the only date-and-locale report and it was a Parquet INT96 
calendar-system bug, fixed in #12826.
   
   ### What happened
   
   The Zeta SQL transform builds the formatter for `FORMATDATETIME` and 
`FROM_UNIXTIME` with `DateTimeFormatter.ofPattern(format)`, which captures 
`Locale.getDefault(Locale.Category.FORMAT)` at construction. Any pattern letter 
that renders text rather than digits (`MMM`, `MMMM`, `EEE`, `EEEE`, `a`, `G`, 
`QQQ`) therefore produces a different string depending on the default locale of 
whichever worker JVM runs the transform.
   
   - `DateTimeFunction.java:616` (`formatdatetime`)
   - `DateTimeFunction.java:799` (`fromUnixTime`)
   
   The same job, over the same rows, writes different values to the sink on 
different workers, and the values are not reproducible across environments.
   
   **The clearest sign that this is unintended is inside the same class.** 
`DAYNAME` and `MONTHNAME` deliberately pin English: `getMonthsAndWeeks` at 
`DateTimeFunction.java:327-336` uses 
`DateFormatSymbols.getInstance(Locale.ENGLISH)`. So one weekday value, in one 
row, in one transform, comes out of `DAYNAME` in English and out of 
`FORMATDATETIME(dt, 'EEEE')` in the worker's language.
   
   I measured this by compiling `seatunnel-transforms-v2` from `dev` and 
calling the real `DateTimeFunction` methods, not a reimplementation. Input 
`2026-10-23T14:05:09`:
   
   | default locale | `DAYNAME(dt)` | `MONTHNAME(dt)` | 
`FORMATDATETIME(dt,'EEEE')` | `FORMATDATETIME(dt,'dd MMM yyyy')` | 
`FROM_UNIXTIME(ts,'MMM dd','UTC')` |
   |---|---|---|---|---|---|
   | `en-US` | Friday | October | Friday | 23 Oct 2026 | Oct 23 |
   | `tr-TR` | Friday | October | Cuma | 23 Eki 2026 | Eki 23 |
   | `ja-JP` | Friday | October | 金曜日 | 23 10月 2026 | 10月 23 |
   | `de-DE` | Friday | October | Freitag | 23 Okt. 2026 | Okt. 23 |
   | `ar-EG` | Friday | October | الجمعة | 23 أكتوبر 2026 | أكتوبر 23 |
   
   **What is not affected, checked rather than assumed.** Numeric-only patterns 
are stable: `yyyy-MM-dd HH:mm:ss` through the same function gives one distinct 
result across all five locales. All 13 patterns of `ZetaDateTimeFormat` are 
numeric-only and produced zero variation across seven locales including 
`th-TH-u-ca-buddhist` and `ar-EG-u-nu-arab`, so that enum is not part of this 
report. Every existing use of these two functions in the tests, the e2e configs 
and the docs is numeric-only, which is why no test catches this today.
   
   ### SeaTunnel Version
   
   `dev` at `14a27ac26a0a71b56722f86cdf8ec2779094ceaf`
   
   ### SeaTunnel Config
   
   ```conf
   env {
     job.mode = "BATCH"
   }
   
   source {
     FakeSource {
       plugin_output = "fake"
       schema = {
         fields {
           id = "int"
           dt = "timestamp"
         }
       }
       rows = [
         {fields = [1, "2026-10-23T14:05:09"], kind = INSERT}
       ]
     }
   }
   
   transform {
     Sql {
       plugin_input = "fake"
       plugin_output = "out"
       query = "select id, DAYNAME(dt) as a, FORMATDATETIME(dt, 'EEEE') as b, 
FORMATDATETIME(dt, 'dd MMM yyyy') as c from fake"
     }
   }
   
   sink {
     Console {
       plugin_input = "out"
     }
   }
   ```
   
   ### Running Command
   
   ```shell
   # the pattern text is the only thing that changes between these two runs
   ./bin/seatunnel.sh --config ./config/sql_locale.conf -e local
   JAVA_TOOL_OPTIONS="-Duser.language=tr -Duser.country=TR" ./bin/seatunnel.sh 
--config ./config/sql_locale.conf -e local
   ```
   
   ### Error Exception
   
   No exception is raised. That is what makes this hard to notice: the job 
succeeds and writes different data.
   
   ```
   # default locale en-US
   a = Friday,  b = Friday,  c = 23 Oct 2026
   
   # default locale tr-TR, same job, same row
   a = Friday,  b = Cuma,    c = 23 Eki 2026
   #            ^^^^^^^^^^ DAYNAME stays English because 
DateTimeFunction.java:331
   #                       pins Locale.ENGLISH, while FORMATDATETIME follows 
the JVM
   ```
   
   ### Zeta or Flink or Spark Version
   
   Not engine specific. The defect is in `seatunnel-transforms-v2`, which every 
engine shares. I ran the functions directly rather than a cluster, so I am not 
claiming a cluster reproduction here.
   
   ### Java or Scala Version
   
   `openjdk 17.0.20.1 2026-08-18`, build `17.0.20.1+1-1-24.04-Ubuntu`
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to