[jira] [Commented] (AVRO-1493) Avoid the "Turkish Locale Problem"
[
https://issues.apache.org/jira/browse/AVRO-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048863#comment-15048863
]
Kevin Schultz commented on AVRO-1493:
-
We are seeing the same error using Avro on Android. I dug around a bit and at
least for the fingerprint generation problem my guess is that the problem is
caused by a few places where the Locale is not explicitly set.
org.apache.avro.generic.GenericData.java:
builder.append(hex.toUpperCase());
org.apache.avro.Schema.java: order
Field.Order.valueOf(orderNode.getTextValue().toUpperCase());
org.apache.avro.Schema.java: private Type() {
this.name = this.name().toLowerCase(); }
org.apache.avro.Schema.java: private Order() {
this.name = this.name().toLowerCase(); }
> Avoid the "Turkish Locale Problem"
> --
>
> Key: AVRO-1493
> URL: https://issues.apache.org/jira/browse/AVRO-1493
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.6
> Environment: Hadoop trunk build error on mac-os with turkish locale.
>Reporter: Serkan Taş
> Fix For: 1.7.8
>
>
> Locale dependent String.toUpperCase(), String.toLowerCase() causes unexpected
> behavior if the the locale is Turkish
> Not sure about String.equalsIgnoreCase(..).
> Here is the error :
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> [ERROR]
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
> I f i check the code i discovered the reason for error :
> public static final org.apache.avro.Schema SCHEMA$ = new
> org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[{\"name\":\"intField\",\"type\":\"Ýnt\"}]}");
> For the code generated from schema, locale dependent capitalization of letter
> "i" turns in to "Ý" should be the same for "I" to "ı".
> Same bug exist in OPENEJB-1071, OAK-260, IBATIS-218.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1493) Avoid the "Turkish Locale Problem"
[
https://issues.apache.org/jira/browse/AVRO-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049384#comment-15049384
]
Kevin Schultz commented on AVRO-1493:
-
I have a test case that illustrates the issue & a patch to fix it, I just need
to find my way through the Avro contribution process (and remember how to use
SVN)
https://github.com/krschultz/avro/commit/39d6b9db492b0f5f4d0ed1a32f0cb5c7be0fa11f
> Avoid the "Turkish Locale Problem"
> --
>
> Key: AVRO-1493
> URL: https://issues.apache.org/jira/browse/AVRO-1493
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.6
> Environment: Hadoop trunk build error on mac-os with turkish locale.
>Reporter: Serkan Taş
> Fix For: 1.7.8
>
>
> Locale dependent String.toUpperCase(), String.toLowerCase() causes unexpected
> behavior if the the locale is Turkish
> Not sure about String.equalsIgnoreCase(..).
> Here is the error :
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> [ERROR]
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
> I f i check the code i discovered the reason for error :
> public static final org.apache.avro.Schema SCHEMA$ = new
> org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[{\"name\":\"intField\",\"type\":\"Ýnt\"}]}");
> For the code generated from schema, locale dependent capitalization of letter
> "i" turns in to "Ý" should be the same for "I" to "ı".
> Same bug exist in OPENEJB-1071, OAK-260, IBATIS-218.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
