Vyacheslav Koptilin created IGNITE-15892:
--------------------------------------------
Summary: Calcite tests can be used with a limited list of charsets
Key: IGNITE-15892
URL: https://issues.apache.org/jira/browse/IGNITE-15892
Project: Ignite
Issue Type: Bug
Reporter: Vyacheslav Koptilin
Assignee: Vyacheslav Koptilin
Fix For: 3.0.0-alpha4
The attempt to run calcite tests on a platform with the default charset
WINDOWS-1252, for example, results in the following exception:
{noformat}
java.nio.charset.UnsupportedCharsetException: WINDOWS-1252 at
org.apache.calcite.sql.SqlUtil.getCharset(SqlUtil.java:1008)
at org.apache.calcite.util.NlsString.<init>(NlsString.java:124)
at org.apache.calcite.util.NlsString.<init>(NlsString.java:116)
at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:964)
at org.apache.calcite.rex.RexBuilder.<init>(RexBuilder.java:132)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgnitePlanner.<init>(IgnitePlanner.java:126)
at
org.apache.ignite.internal.processors.query.calcite.prepare.PlanningContext.planner(PlanningContext.java:163)
at
org.apache.ignite.internal.processors.query.calcite.planner.AbstractPlannerTest.physicalPlan(AbstractPlannerTest.java:232)
at
org.apache.ignite.internal.processors.query.calcite.planner.TableDmlPlannerTest.insertCachesTableScan(TableDmlPlannerTest.java:57)
{noformat}
It seems the list of supported charsets can be found here:
{code:java|title=org.apache.calcite.sql.SqlUtil}
/**
* Translates a character set name from a SQL-level name into a Java-level
* name.
*
* @param name SQL-level name
* @return Java-level name, or null if SQL-level name is unknown
*/
public static @Nullable String translateCharacterSetName(String name) {
switch (name) {
case "BIG5":
return "Big5";
case "LATIN1":
return "ISO-8859-1";
case "UTF8":
return "UTF-8";
case "UTF16":
case "UTF-16":
return ConversionUtil.NATIVE_UTF16_CHARSET_NAME;
case "GB2312":
case "GBK":
case "UTF-16BE":
case "UTF-16LE":
case "ISO-8859-1":
case "UTF-8":
return name;
default:
return null;
}
}
{code}
On the other hand, IGNITE-15545 proposes getting the default charset from the
JVM that may contradict the list of supported charsets:
{code:java|title=IgniteTypeFactory}
@Override
public Charset getDefaultCharset() {
// Use JVM default charset rather then Calcite default charset
(ISO-8859-1).
return Charset.defaultCharset();
}
{code}
The obvious and dirty way to work around this issue is to specify the default
charset explicitly
{code:java|title=calcite/pom.xml}
<!--
Need to override file.encoding property as calcite tests
support a limited list of charsets.
-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Djava.util.logging.config.file=../../config/java.util.logging.properties
-Dfile.encoding=UTF-8</argLine>
</configuration>
</plugin>
{code}
Another option is switching to UTF-8 or UTF-16 in case of the default JVM
charset is not properly supported:
{code:java|title=IgniteTypeFactory}
public Charset getDefaultCharset() {
// Use JVM default charset rather then Calcite default charset
(ISO-8859-1).
Charset jvmDefault = Charset.defaultCharset();
if
(SqlUtil.translateCharacterSetName(jvmDefault.name().toUpperCase(Locale.ROOT))
== null)
jvmDefault = StandardCharsets.UTF_8;
return jvmDefault;
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
