[
https://issues.apache.org/jira/browse/KYLIN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
André Luís da Rosa de Lima updated KYLIN-4757:
----------------------------------------------
Description:
Using Postgres as data source, columns of datatype numeric without a explicitly
set precision and scale become "decimal(256)" in kylin. 256 comes from the
value of "kylin.source.hive.default-varchar-precision" (despite using jdbc as
datasource). This causes an error when building a cube.
The problem can be fixed in
"org.apache.kylin.source.jdbc.extensible.JdbcExplorer" class
([https://github.com/apache/kylin/tree/
master/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111|https://github.com/apache/kylin/blob/master/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111]
[)|https://github.com/apache/kylin/blob/kylin-4.0.0-alpha/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)]
The csize variable defined in line 96 receives the value 131089 from
columns.getInt("COLUMN_SIZE");
I fixed it by modifying the following part in the source file starting after
line 111:
{code:java}
int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize
: -1;
precision = Math.min(precision,
KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision());
int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
{code}
to this:
{code:java}
int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize
: -1;
int maxPrecision = precision;
if (kylinType.equals("char")) {
maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultCharPrecision();
} else if (kylinType.equals("varchar")) {
maxPrecision =
KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision();
} else if ((kylinType.equals("decimal") || kylinType.equals("numeric"))) {
maxPrecision =
KylinConfig.getInstanceFromEnv().getDefaultDecimalPrecision();
}
precision = Math.min(precision, maxPrecision);
int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
{code}
was:
Using Postgres as data source, columns of datatype numeric without a explicitly
set precision and scale become "decimal(256)" in kylin. 256 comes from the
value of "kylin.source.hive.default-varchar-precision" (despite using jdbc as
datasource). This causes an error when building a cube.
The problem can be fixed in
"org.apache.kylin.source.jdbc.extensible.JdbcExplorer" class
([https://github.com/apache/kylin/tree/|https://github.com/apache/kylin/blob/kylin-4.0.0-alpha/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)]
[master/|https://github.com/apache/kylin/blob/kylin-4.0.0-alpha/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)]
[source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)|https://github.com/apache/kylin/blob/kylin-4.0.0-alpha/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)]
The csize variable defined in line 96 receives the value 131089 from
columns.getInt("COLUMN_SIZE");
I fixed it by modifying the following part in the source file starting after
line 111:
{code:java}
int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize
: -1;
precision = Math.min(precision,
KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision());
int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
{code}
to this:
{code:java}
int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize
: -1;
int maxPrecision = precision;
if (kylinType.equals("char")) {
maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultCharPrecision();
} else if (kylinType.equals("varchar")) {
maxPrecision =
KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision();
} else if ((kylinType.equals("decimal") || kylinType.equals("numeric"))) {
maxPrecision =
KylinConfig.getInstanceFromEnv().getDefaultDecimalPrecision();
}
precision = Math.min(precision, maxPrecision);
int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
{code}
> Impossible precision for decimal datatype in kylin if the source column is
> numeric in postgres
> ----------------------------------------------------------------------------------------------
>
> Key: KYLIN-4757
> URL: https://issues.apache.org/jira/browse/KYLIN-4757
> Project: Kylin
> Issue Type: Bug
> Components: RDBMS Source
> Affects Versions: v3.1.0
> Reporter: André Luís da Rosa de Lima
> Priority: Major
> Labels: postgresql, source-jdbc
>
> Using Postgres as data source, columns of datatype numeric without a
> explicitly set precision and scale become "decimal(256)" in kylin. 256 comes
> from the value of "kylin.source.hive.default-varchar-precision" (despite
> using jdbc as datasource). This causes an error when building a cube.
> The problem can be fixed in
> "org.apache.kylin.source.jdbc.extensible.JdbcExplorer" class
> ([https://github.com/apache/kylin/tree/
> master/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111|https://github.com/apache/kylin/blob/master/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111]
>
> [)|https://github.com/apache/kylin/blob/kylin-4.0.0-alpha/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111)]
> The csize variable defined in line 96 receives the value 131089 from
> columns.getInt("COLUMN_SIZE");
> I fixed it by modifying the following part in the source file starting after
> line 111:
> {code:java}
> int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ?
> csize : -1;
> precision = Math.min(precision,
> KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision());
>
> int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits :
> -1;
> {code}
> to this:
>
> {code:java}
> int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ?
> csize : -1;
> int maxPrecision = precision;
> if (kylinType.equals("char")) {
> maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultCharPrecision();
> } else if (kylinType.equals("varchar")) {
> maxPrecision =
> KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision();
> } else if ((kylinType.equals("decimal") || kylinType.equals("numeric"))) {
> maxPrecision =
> KylinConfig.getInstanceFromEnv().getDefaultDecimalPrecision();
> }
> precision = Math.min(precision, maxPrecision);
> int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits :
> -1;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
