[ https://issues.apache.org/jira/browse/FLINK-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311043#comment-15311043 ]
ASF GitHub Bot commented on FLINK-3921: --------------------------------------- GitHub user rekhajoshm opened a pull request: https://github.com/apache/flink/pull/2060 [FLINK-3921] StringParser encoding Corrected StringParser encoding You can merge this pull request into a Git repository by running: $ git pull https://github.com/rekhajoshm/flink FLINK-3921 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2060.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2060 ---- commit 1e1ce9efefccf5a5585be802af4200e9d1ae7a98 Author: Rekha Joshi <rekhajo...@gmail.com> Date: 2016-06-01T20:03:50Z Merge pull request #1 from apache/master Apache Flink master pull commit 675b6a44e76ae71901bc6d4eaea1d09b6f789ff6 Author: Joshi <rekhajo...@gmail.com> Date: 2016-06-01T20:26:47Z [FLINK-3921] StringParser encoding ---- > StringParser not specifying encoding to use > ------------------------------------------- > > Key: FLINK-3921 > URL: https://issues.apache.org/jira/browse/FLINK-3921 > Project: Flink > Issue Type: Improvement > Components: Core > Affects Versions: 1.0.3 > Reporter: Tatu Saloranta > Assignee: Rekha Joshi > Priority: Trivial > > Class `flink.types.parser.StringParser` has javadocs indicating that contents > are expected to be Ascii, similar to `StringValueParser`. That makes sense, > but when constructing actual instance, no encoding is specified; on line 66 > f.ex: > this.result = new String(bytes, startPos+1, i - startPos - 2); > which leads to using whatever default platform encoding is. If contents > really are always Ascii (would not count on that as parser is used from CSV > reader), not a big deal, but it can lead to the usual Latin-1-VS-UTF-8 issues. > So I think that encoding should be explicitly specified, whatever is to be > used: javadocs claim ascii, so could be "us-ascii", but could well be UTF-8 > or even ISO-8859-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)