Adalennis opened a new issue, #9032:
URL: https://github.com/apache/hop/issues/9032
### Apache Hop version?
2.20.0-SNAPSHOT
### Java version?
Java 21
### Operating system
macOS
### What happened?
With an encoding whose Java charset writes a byte order mark (`UTF-16`,
`x-UTF-16LE-BOM`),
Text File Output writes a BOM before every value, separator and line break,
not once at
the start of the file. Readers skip the first one and keep the others as
`U+FEFF`
characters inside the data: a value `V001` is read back as `V001\uFEFF`.
Steps to reproduce: a Data Grid with 3 rows into a Text File Output (no
header,
separator `;`, Encoding `x-UTF-16LE-BOM`). With one String field the file
has 6 BOMs
instead of 1; with two fields it has 12, one before every value, separator
and
line break:
fffe 5600 3000 3000 3100 fffe 3b00 fffe 4d00 ... BOM V001 BOM ; BOM
Maria ...
`UTF-16` gives the same result (big-endian BOMs); `UTF-16LE` gives none, so
there is no
way to write a UTF-16 file with a single leading BOM, which some receiving
systems expect.
Cause: `TextFileOutput` encodes each value on its own with
`String.getBytes(encoding)`
(lines 577, 592 and 643 on main), and the separator, enclosure and newline
once each with
`getBytes(encoding)` in `init` (lines 1100–1106), which are then written
again and again.
Java's UTF-16 charsets add a BOM to every `getBytes` result.
Expected: one BOM at the start of the file, and the values encoded without
one.
### Issue Priority
Priority: 3
### Issue Component
Component: Hop Gui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
