Ben Sherman created GROOVY-12539:
------------------------------------
Summary: FileReaderSource does not strip a UTF-8 BOM: checks 0xFF
instead of 0xBF
Key: GROOVY-12539
URL: https://issues.apache.org/jira/browse/GROOVY-12539
Project: Groovy
Issue Type: Bug
Components: Compiler
Affects Versions: 4.0.33, 5.0.0
Reporter: Ben Sherman
`FileReaderSource.getReader()` is meant to skip a UTF-8 byte order mark when
the source encoding is UTF-8, but it compares the third byte with `0xFF`
instead of `0xBF`:
```java
int i = in.read();
hasBOM &= i == 0xEF;
i = in.read();
hasBOM &= i == 0xBB;
i = in.read();
hasBOM &= i == 0xFF; // should be 0xBF
```
The UTF-8 BOM is `EF BB BF`. The check never matches, the stream is reset, and
the reader returns U+FEFF as the first character. The parser then fails with
`Unexpected character: '' @ line 1, column 1`, where the reported character is
invisible.
Reproduction
```console
$ printf '\xef\xbb\xbfprintln "hello"\n' > bom.groovy
```
```java
import java.io.File;
import org.codehaus.groovy.control.CompilerConfiguration;
import org.codehaus.groovy.control.io.FileReaderSource;
public class Repro {
public static void main(String[] args) throws Exception {
var config = new CompilerConfiguration();
config.setSourceEncoding("UTF-8");
var reader = new FileReaderSource(new File(args[0]),
config).getReader();
System.out.printf("first char: U+%04X%n", reader.read());
}
}
```
```console
$ java -cp groovy-4.0.33.jar Repro.java bom.groovy
first char: U+FEFF
```
Expected: `first char: U+0070` (`p`).
Fix
Change `0xFF` to `0xBF`.
Notes
- A compiler built from `new SourceUnit(File, ...)` reads the file through
`FileReaderSource`, so a file with a BOM fails to compile.
- `groovy bom.groovy` fails with the same error, but through
`GroovyCodeSource`, which reads the file with `ResourceGroovyMethods.getText`
and keeps the BOM. Fixing `FileReaderSource` alone does not fix that path.
GROOVY-7661 (open) asks for an option to consume a BOM in the I/O methods.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
