[
https://issues.apache.org/jira/browse/BEAM-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Kedin updated BEAM-3581:
------------------------------
Description:
Beam SQL overrides default charset that Calcite uses and sets it to UTF16. It
is done via system properties.
Problem is that we do this only when it hasn't been set yet. So if system
property has been set to ISO-8859-1 (Calcite's default), then test runs will
fail when trying to encode characters not supported in that encoding.
Solution:
- because it's a system property, we don't want to force override it;
- for the same reason we cannot set it for a specific query execution;
- we can expose a static method on BeamSql to override these properties if
explicitly requested;
- affected tests will explicitly override it;
- otherwise behavior will stay unchanged and we will respect defaults and user
settings;
was:
Beam SQL overrides default charset that Calcite uses and sets it to UTF16. It
is done via system properties.
Problem is that we do this only when it hasn't been set yet. So if system
property has been set to ISO-8859-1 (Calcite's default), then test runs will
fail when trying to encode characters not supported in that encoding.
Solution:
- because it's a system property, we don't want to force override it;
- for the same reason we cannot set it for a specific query execution;
- we can expose a static method on BeamSql to override these properties if
explicitly requested;
- tests will explicitly override it;
- otherwise behavior will stay unchanged and we will respect defaults and user
settings;
> [SQL] Support for Non-ASCII chars is flaky
> ------------------------------------------
>
> Key: BEAM-3581
> URL: https://issues.apache.org/jira/browse/BEAM-3581
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Reporter: Anton Kedin
> Assignee: Anton Kedin
> Priority: Major
>
> Beam SQL overrides default charset that Calcite uses and sets it to UTF16. It
> is done via system properties.
> Problem is that we do this only when it hasn't been set yet. So if system
> property has been set to ISO-8859-1 (Calcite's default), then test runs will
> fail when trying to encode characters not supported in that encoding.
> Solution:
> - because it's a system property, we don't want to force override it;
> - for the same reason we cannot set it for a specific query execution;
> - we can expose a static method on BeamSql to override these properties if
> explicitly requested;
> - affected tests will explicitly override it;
> - otherwise behavior will stay unchanged and we will respect defaults and
> user settings;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
