[jira] [Commented] (FLINK-23305) Support semi/anti interval joins

2021-10-21 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432420#comment-17432420
 ] 

xuyangzhong commented on FLINK-23305:
-

I'll try to fix these issue. (FLINK-23305 / FLINK-23319 / FLINK-23320)

> Support semi/anti interval joins
> 
>
> Key: FLINK-23305
> URL: https://issues.apache.org/jira/browse/FLINK-23305
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Planner
>Reporter: Timo Walther
>Priority: Major
>
> I don't see a reason why we shouldn't support interval joins also for 
> semi/anti joins like:
> {code}
> SELECT *
> FROM OT
> WHERE EXISTS (
> SELECT *
> FROM TT
> WHERE TT.tx = OT.tx AND
> TT.isEnd = TRUE AND
> TT.rowtime BETWEEN OT.rowtime AND OT.rowtime + INTERVAL '1' HOUR)
> {code}
> The resulting plan contains a join operation anyway but without detecting the 
> interval.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-19722) Pushdown Watermark to SourceProvider (new Source)

2021-10-13 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428567#comment-17428567
 ] 

xuyangzhong commented on FLINK-19722:
-

I'll try to fix this issue.

> Pushdown Watermark to SourceProvider (new Source)
> -
>
> Key: FLINK-19722
> URL: https://issues.apache.org/jira/browse/FLINK-19722
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Planner
>Reporter: Jingsong Lee
>Priority: Major
>  Labels: auto-deprioritized-critical
> Fix For: 1.15.0
>
>
> See {{StreamExecutionEnvironment.fromSource(Source, WatermarkStrategy)}}
> The new source can get watermark strategy to handle split watermark.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-24492) incorrect implicit type conversion between numeric and (var)char

2021-10-09 Thread xuyangzhong (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuyangzhong updated FLINK-24492:

Description: 
The result of the sql "select 1 = '1'" is false. This is caused by the CodeGen. 
CodeGen  incorrectly transform this "=" to "BinaryStringData.equals (int 1)". 
And "<>" has the same wrong result.

In my opinion, "=" should have the same behavior with ">" and "<", which have 
the correct results. So before calcite solves this bug or flink supports this 
kind of implicit type conversion, we'd better temporarily forbidding this 
implicit type conversion in "=" and "<>".

  was:
The result of the sql "select 1 = '1'" is false. This is caused by the CodeGen. 
CodeGen  incorrectly transform this "=" to "BinaryStringData.equals (int 1)". 
And "<>" has the same wrong result.

In my opinion, "=" should have the same behavior with ">" and "<", which have 
the correct results. So before calcite solves this bug or flink support this 
kind of implicit type conversion, we'd better temporarily forbidding this 
implicit type conversion in "=" and "<>".


> incorrect implicit type conversion between numeric and (var)char
> 
>
> Key: FLINK-24492
> URL: https://issues.apache.org/jira/browse/FLINK-24492
> Project: Flink
>  Issue Type: Bug
>Reporter: xuyangzhong
>Priority: Minor
>
> The result of the sql "select 1 = '1'" is false. This is caused by the 
> CodeGen. CodeGen  incorrectly transform this "=" to "BinaryStringData.equals 
> (int 1)". And "<>" has the same wrong result.
> In my opinion, "=" should have the same behavior with ">" and "<", which have 
> the correct results. So before calcite solves this bug or flink supports this 
> kind of implicit type conversion, we'd better temporarily forbidding this 
> implicit type conversion in "=" and "<>".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-24492) incorrect implicit type conversion between numeric and (var)char

2021-10-09 Thread xuyangzhong (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuyangzhong updated FLINK-24492:

Description: 
The result of the sql "select 1 = '1'" is false. This is caused by the CodeGen. 
CodeGen  incorrectly transform this "=" to "BinaryStringData.equals (int 1)". 
And "<>" has the same wrong result.

In my opinion, "=" should have the same behavior with ">" and "<", which have 
the correct results. So before calcite solves this bug or flink support this 
kind of implicit type conversion, we'd better temporarily forbidding this 
implicit type conversion in "=" and "<>".

  was:
The result of the sql "select 1 = '1'" is false. This is caused by the CodeGen. 
CodeGen  incorrectly transform this "=" to "BinaryStringData.equals (int 1)". 
And "<>" has the same wrong result.

In my opinion, "=" should have the same behavior with ">" and "<", which have 
the correct results. So before calcite solves this bug, we'd better temporarily 
forbidding this implicit type conversion in "=" and "<>".


> incorrect implicit type conversion between numeric and (var)char
> 
>
> Key: FLINK-24492
> URL: https://issues.apache.org/jira/browse/FLINK-24492
> Project: Flink
>  Issue Type: Bug
>Reporter: xuyangzhong
>Priority: Minor
>
> The result of the sql "select 1 = '1'" is false. This is caused by the 
> CodeGen. CodeGen  incorrectly transform this "=" to "BinaryStringData.equals 
> (int 1)". And "<>" has the same wrong result.
> In my opinion, "=" should have the same behavior with ">" and "<", which have 
> the correct results. So before calcite solves this bug or flink support this 
> kind of implicit type conversion, we'd better temporarily forbidding this 
> implicit type conversion in "=" and "<>".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-24492) incorrect implicit type conversion between numeric and (var)char

2021-10-09 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426562#comment-17426562
 ] 

xuyangzhong commented on FLINK-24492:
-

link https://issues.apache.org/jira/browse/FLINK-18234

> incorrect implicit type conversion between numeric and (var)char
> 
>
> Key: FLINK-24492
> URL: https://issues.apache.org/jira/browse/FLINK-24492
> Project: Flink
>  Issue Type: Bug
>Reporter: xuyangzhong
>Priority: Minor
>
> The result of the sql "select 1 = '1'" is false. This is caused by the 
> CodeGen. CodeGen  incorrectly transform this "=" to "BinaryStringData.equals 
> (int 1)". And "<>" has the same wrong result.
> In my opinion, "=" should have the same behavior with ">" and "<", which have 
> the correct results. So before calcite solves this bug, we'd better 
> temporarily forbidding this implicit type conversion in "=" and "<>".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-18234) Implicit type conversion in join condition

2021-10-09 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-18234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426561#comment-17426561
 ] 

xuyangzhong commented on FLINK-18234:
-

link https://issues.apache.org/jira/browse/FLINK-24492

> Implicit type conversion in join condition
> --
>
> Key: FLINK-18234
> URL: https://issues.apache.org/jira/browse/FLINK-18234
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Affects Versions: 1.10.1
>Reporter: YufeiLiu
>Priority: Minor
>  Labels: auto-deprioritized-major
>
> Execute sql "SELECT a1, b1 FROM A JOIN B ON a2 = b4", a2(BIGINT) b4(VARCHAR) 
> will throw exception 
> {code}
> org.apache.flink.table.api.TableException: VARCHAR(2147483647) and INTEGER 
> does not have common type now
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule$$anonfun$onMatch$1.apply(JoinConditionTypeCoerceRule.scala:76)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule$$anonfun$onMatch$1.apply(JoinConditionTypeCoerceRule.scala:65)
>   at scala.collection.Iterator$class.foreach(Iterator.scala:891)
>   at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
>   at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>   at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule.onMatch(JoinConditionTypeCoerceRule.scala:65)
>   at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:328)
>   at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:562)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:427)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:264)
>   at 
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:223)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:210)
>   at 
> org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:69)
> {code}
> Should we do some implicit type coercion in this case? It works on old 
> version, and also can use in WHERE condition like "WHERE a4 = 3"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-24492) incorrect implicit type conversion between numeric and (var)char

2021-10-09 Thread xuyangzhong (Jira) 
xuyangzhong created FLINK-24492:
---

 Summary: incorrect implicit type conversion between numeric and 
(var)char
 Key: FLINK-24492
 URL: https://issues.apache.org/jira/browse/FLINK-24492
 Project: Flink
  Issue Type: Bug
Reporter: xuyangzhong


The result of the sql "select 1 = '1'" is false. This is caused by the CodeGen. 
CodeGen  incorrectly transform this "=" to "BinaryStringData.equals (int 1)". 
And "<>" has the same wrong result.

In my opinion, "=" should have the same behavior with ">" and "<", which have 
the correct results. So before calcite solves this bug, we'd better temporarily 
forbidding this implicit type conversion in "=" and "<>".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-18234) Implicit type conversion in join condition

2021-10-08 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-18234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426438#comment-17426438
 ] 

xuyangzhong commented on FLINK-18234:
-

I'll try to fix this bug

> Implicit type conversion in join condition
> --
>
> Key: FLINK-18234
> URL: https://issues.apache.org/jira/browse/FLINK-18234
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Affects Versions: 1.10.1
>Reporter: YufeiLiu
>Priority: Minor
>  Labels: auto-deprioritized-major
>
> Execute sql "SELECT a1, b1 FROM A JOIN B ON a2 = b4", a2(BIGINT) b4(VARCHAR) 
> will throw exception 
> {code}
> org.apache.flink.table.api.TableException: VARCHAR(2147483647) and INTEGER 
> does not have common type now
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule$$anonfun$onMatch$1.apply(JoinConditionTypeCoerceRule.scala:76)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule$$anonfun$onMatch$1.apply(JoinConditionTypeCoerceRule.scala:65)
>   at scala.collection.Iterator$class.foreach(Iterator.scala:891)
>   at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
>   at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>   at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.JoinConditionTypeCoerceRule.onMatch(JoinConditionTypeCoerceRule.scala:65)
>   at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:328)
>   at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:562)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:427)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:264)
>   at 
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:223)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:210)
>   at 
> org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:69)
> {code}
> Should we do some implicit type coercion in this case? It works on old 
> version, and also can use in WHERE condition like "WHERE a4 = 3"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-23271) RuntimeException: while resolving method 'booleanValue' in class class java.math.BigDecimal

2021-10-07 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-23271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425946#comment-17425946
 ] 

xuyangzhong commented on FLINK-23271:
-

Hi , [~matriv].

The result of the discussion in calcite is the following:

1. Casting decimal (and float, real, double) into boolean will be forbidden.

2. Casting integer types(contains tinyint, smallint, int, bigint) into boolean 
is allowed.

So I think we better follow the behavior of calcite because calcite maybe 
represents the most db's behavior.

What's your opinion?

> RuntimeException: while resolving method 'booleanValue' in class class 
> java.math.BigDecimal
> ---
>
> Key: FLINK-23271
> URL: https://issues.apache.org/jira/browse/FLINK-23271
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: xiaojin.wy
>Priority: Major
>
> *Sql :*
> CREATE TABLE database3_t0(
> c0 DECIMAL , c1 SMALLINT
> ) WITH (
>  'connector' = 'filesystem',
>  'path' = 'hdfs:///tmp/database3_t0.csv',
>  'format' = 'csv' 
> );
> INSERT OVERWRITE database8_t0(c0, c1) VALUES(2113554022, cast(-22975 as 
> SMALLINT)), (1570419395, cast(-26858 as SMALLINT)), (-1569861129, cast(-20143 
> as SMALLINT));
> SELECT database8_t0.c0 AS ref0 FROM database8_t0 WHERE CAST 
> (0.10915913549909961 AS BOOLEAN;
> *After excuting the sql, you will find the error:*
> java.lang.RuntimeException: while resolving method 'booleanValue' in class 
> class java.math.BigDecimal
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:424)
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:435)
>   at 
> org.apache.calcite.linq4j.tree.Expressions.unbox(Expressions.java:1453)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:398)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:326)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCast(RexToLixTranslator.java:538)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$CastImplementor.implementSafe(RexImpTable.java:2450)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:2894)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:2859)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1084)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexCall.accept(RexCall.java:174)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:970)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:237)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:231)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateList(RexToLixTranslator.java:818)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateProjects(RexToLixTranslator.java:198)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:90)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:66)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2101)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:326)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:287)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:262)
>   at 
> org.apache.flink.table.planner.plan.utils.FlinkRexUtil$.simplify(FlinkRexUtil.scala:224)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.simplify(SimplifyFilterConditionRule.scala:63)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.onMatch(SimplifyFilterConditionRule.scala:46)
>   at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
>   at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
>   at 
>

[jira] [Commented] (FLINK-23271) RuntimeException: while resolving method 'booleanValue' in class class java.math.BigDecimal

2021-09-30 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-23271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422638#comment-17422638
 ] 

xuyangzhong commented on FLINK-23271:
-

In order to fix this issue, I am currently paying attention to the issue of 
calcite regarding the legality of cast. It seems that the sql standard 
considers and calcite will consider that cast from decimal (even double, float, 
real) to boolean is invalid(you can see the issue here: [CALCITE-4777]). So 
perhaps FLINK should follow the rule just like calcite or sql standard, or 
FLINK can take this as an extension.

> RuntimeException: while resolving method 'booleanValue' in class class 
> java.math.BigDecimal
> ---
>
> Key: FLINK-23271
> URL: https://issues.apache.org/jira/browse/FLINK-23271
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: xiaojin.wy
>Priority: Major
>
> *Sql :*
> CREATE TABLE database3_t0(
> c0 DECIMAL , c1 SMALLINT
> ) WITH (
>  'connector' = 'filesystem',
>  'path' = 'hdfs:///tmp/database3_t0.csv',
>  'format' = 'csv' 
> );
> INSERT OVERWRITE database8_t0(c0, c1) VALUES(2113554022, cast(-22975 as 
> SMALLINT)), (1570419395, cast(-26858 as SMALLINT)), (-1569861129, cast(-20143 
> as SMALLINT));
> SELECT database8_t0.c0 AS ref0 FROM database8_t0 WHERE CAST 
> (0.10915913549909961 AS BOOLEAN;
> *After excuting the sql, you will find the error:*
> java.lang.RuntimeException: while resolving method 'booleanValue' in class 
> class java.math.BigDecimal
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:424)
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:435)
>   at 
> org.apache.calcite.linq4j.tree.Expressions.unbox(Expressions.java:1453)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:398)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:326)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCast(RexToLixTranslator.java:538)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$CastImplementor.implementSafe(RexImpTable.java:2450)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:2894)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:2859)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1084)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexCall.accept(RexCall.java:174)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:970)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:237)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:231)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateList(RexToLixTranslator.java:818)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateProjects(RexToLixTranslator.java:198)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:90)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:66)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2101)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:326)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:287)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:262)
>   at 
> org.apache.flink.table.planner.plan.utils.FlinkRexUtil$.simplify(FlinkRexUtil.scala:224)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.simplify(SimplifyFilterConditionRule.scala:63)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.onMatch(SimplifyFilterConditionRule.scala:46)
>   at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
>   at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
>   at 
>

[jira] [Created] (FLINK-24318) Casting a number to boolean has different results between 'select' fields and 'where' condition

2021-09-17 Thread xuyangzhong (Jira) 
xuyangzhong created FLINK-24318:
---

 Summary: Casting a number to boolean has different results between 
'select' fields and 'where' condition
 Key: FLINK-24318
 URL: https://issues.apache.org/jira/browse/FLINK-24318
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Planner
Reporter: xuyangzhong
 Fix For: 1.15.0


The same cast in the following two sql:
{code:java}
// SQL 1
SELECT cast(0.1 as boolean)

// SQL 2
SELECT * from test2 where cast(0.1 as boolean)
{code}
has different results.

The cast result in SQL 1 is true and the cast in SQL 2 is false.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (FLINK-24291) Decimal precision is lost when deserializing in test cases

2021-09-16 Thread xuyangzhong (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuyangzhong updated FLINK-24291:

Summary: Decimal precision is lost when deserializing in test cases  (was: 
Decimal precision is lost when deserializing)

> Decimal precision is lost when deserializing in test cases
> --
>
> Key: FLINK-24291
> URL: https://issues.apache.org/jira/browse/FLINK-24291
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Reporter: xuyangzhong
>Priority: Minor
>
> When added the test case following into FileSystemItCaseBase:
> {code:java}
> // create table
> tableEnv.executeSql(
>   s"""
>  |create table test2 (
>  |  c0 decimal(10,0), c1 int
>  |) with (
>  |  'connector' = 'filesystem',
>  |  'path' = '/Users/zhongxuyang/test/test',
>  |  'format' = 'testcsv'
>  |)
>""".stripMargin
> )
> //test file content is:
> //2113554011,1
> //2113554022,2
> {code}
> and
> {code:java}
> // select sql
> @Test
> def myTest2(): Unit={
>   check(
> "SELECT c0 FROM test2",
> Seq(
>   row(2113554011),
>   row(2113554022)
> ))
> }
> {code}
> i got an exception :
> {code}
> java.lang.RuntimeException: Failed to fetch next 
> resultjava.lang.RuntimeException: Failed to fetch next result
>  at 
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
>  at 
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
>  at 
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
>  at java.util.Iterator.forEachRemaining(Iterator.java:115) at 
> org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:109) 
> at 
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:300)
>  at 
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:140)
>  at 
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:106)
>  at 
> org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.check(BatchFileSystemITCaseBase.scala:46)
>  at 
> org.apache.flink.table.planner.runtime.FileSystemITCaseBase$class.myTest2(FileSystemITCaseBase.scala:128)
>  at 
> org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.myTest2(BatchFileSystemITCaseBase.scala:33)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
>  at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
> at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at 
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
>  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
>  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
>  at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at 
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at 
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at 
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at 
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at 
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
> org.junit.rules.RunRules.evaluate(RunRules.java:20) at 
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
> org.junit.runners.ParentRunner.run(ParentRunner.java:413) at 
> org.junit.runner.JUnitCore.run(JUnitCore.java:137) at 
>

[jira] [Commented] (FLINK-23271) RuntimeException: while resolving method 'booleanValue' in class class java.math.BigDecimal

2021-09-16 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-23271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416419#comment-17416419
 ] 

xuyangzhong commented on FLINK-23271:
-

link: https://issues.apache.org/jira/browse/CALCITE-4777

> RuntimeException: while resolving method 'booleanValue' in class class 
> java.math.BigDecimal
> ---
>
> Key: FLINK-23271
> URL: https://issues.apache.org/jira/browse/FLINK-23271
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: xiaojin.wy
>Priority: Major
>
> *Sql :*
> CREATE TABLE database3_t0(
> c0 DECIMAL , c1 SMALLINT
> ) WITH (
>  'connector' = 'filesystem',
>  'path' = 'hdfs:///tmp/database3_t0.csv',
>  'format' = 'csv' 
> );
> INSERT OVERWRITE database8_t0(c0, c1) VALUES(2113554022, cast(-22975 as 
> SMALLINT)), (1570419395, cast(-26858 as SMALLINT)), (-1569861129, cast(-20143 
> as SMALLINT));
> SELECT database8_t0.c0 AS ref0 FROM database8_t0 WHERE CAST 
> (0.10915913549909961 AS BOOLEAN;
> *After excuting the sql, you will find the error:*
> java.lang.RuntimeException: while resolving method 'booleanValue' in class 
> class java.math.BigDecimal
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:424)
>   at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:435)
>   at 
> org.apache.calcite.linq4j.tree.Expressions.unbox(Expressions.java:1453)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:398)
>   at 
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:326)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCast(RexToLixTranslator.java:538)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$CastImplementor.implementSafe(RexImpTable.java:2450)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:2894)
>   at 
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:2859)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1084)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexCall.accept(RexCall.java:174)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:970)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
>   at org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:237)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:231)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateList(RexToLixTranslator.java:818)
>   at 
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateProjects(RexToLixTranslator.java:198)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:90)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:66)
>   at 
> org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2101)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:326)
>   at 
> org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:287)
>   at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:262)
>   at 
> org.apache.flink.table.planner.plan.utils.FlinkRexUtil$.simplify(FlinkRexUtil.scala:224)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.simplify(SimplifyFilterConditionRule.scala:63)
>   at 
> org.apache.flink.table.planner.plan.rules.logical.SimplifyFilterConditionRule.onMatch(SimplifyFilterConditionRule.scala:46)
>   at 
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
>   at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:243)
>   at 
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
>   at 
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
>   at 
>

[jira] [Updated] (FLINK-24291) Decimal precision is lost when deserializing

2021-09-15 Thread xuyangzhong (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuyangzhong updated FLINK-24291:

Description: 
When added the test case following into FileSystemItCaseBase:
{code:java}
// create table
tableEnv.executeSql(
  s"""
 |create table test2 (
 |  c0 decimal(10,0), c1 int
 |) with (
 |  'connector' = 'filesystem',
 |  'path' = '/Users/zhongxuyang/test/test',
 |  'format' = 'testcsv'
 |)
   """.stripMargin
)

//test file content is:
//2113554011,1
//2113554022,2
{code}
and
{code:java}
// select sql
@Test
def myTest2(): Unit={
  check(
"SELECT c0 FROM test2",
Seq(
  row(2113554011),
  row(2113554022)
))
}
{code}
i got an exception :

 

 
{code:java}
// code placeholder
java.lang.RuntimeException: Failed to fetch next result
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
 at 
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
 at java.util.Iterator.forEachRemaining(Iterator.java:115) at 
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:109) at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:300)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:140)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:106)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.check(BatchFileSystemITCaseBase.scala:46)
 at 
org.apache.flink.table.planner.runtime.FileSystemITCaseBase$class.myTest2(FileSystemITCaseBase.scala:128)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.myTest2(BatchFileSystemITCaseBase.scala:33)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) at 
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at 
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.RunRules.evaluate(RunRules.java:20) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:413) at 
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
 at 
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
 at 
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
 at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

Caused by: java.io.IOException: Failed to fetch job execution result at 
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:184)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:121)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:106)
 ... 43 more

[jira] [Updated] (FLINK-24291) Decimal precision is lost when deserializing

2021-09-15 Thread xuyangzhong (Jira) 


 [ 
https://issues.apache.org/jira/browse/FLINK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xuyangzhong updated FLINK-24291:

Description: 
When added the test case following into FileSystemItCaseBase:
{code:java}
// create table
tableEnv.executeSql(
  s"""
 |create table test2 (
 |  c0 decimal(10,0), c1 int
 |) with (
 |  'connector' = 'filesystem',
 |  'path' = '/Users/zhongxuyang/test/test',
 |  'format' = 'testcsv'
 |)
   """.stripMargin
)

//test file content is:
//2113554011,1
//2113554022,2
{code}
and
{code:java}
// select sql
@Test
def myTest2(): Unit={
  check(
"SELECT c0 FROM test2",
Seq(
  row(2113554011),
  row(2113554022)
))
}
{code}
i got an exception :

java.lang.RuntimeException: Failed to fetch next 
resultjava.lang.RuntimeException: Failed to fetch next result
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
 at 
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
 at java.util.Iterator.forEachRemaining(Iterator.java:115) at 
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:109) at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:300)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:140)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:106)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.check(BatchFileSystemITCaseBase.scala:46)
 at 
org.apache.flink.table.planner.runtime.FileSystemITCaseBase$class.myTest2(FileSystemITCaseBase.scala:128)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.myTest2(BatchFileSystemITCaseBase.scala:33)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) at 
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at 
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.RunRules.evaluate(RunRules.java:20) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:413) at 
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
 at 
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
 at 
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
 at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

 

Caused by: java.io.IOException: Failed to fetch job execution result at 
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:184)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:121)
 at

[jira] [Created] (FLINK-24291) Decimal precision is lost when deserializing

2021-09-15 Thread xuyangzhong (Jira) 
xuyangzhong created FLINK-24291:
---

 Summary: Decimal precision is lost when deserializing
 Key: FLINK-24291
 URL: https://issues.apache.org/jira/browse/FLINK-24291
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Planner
Reporter: xuyangzhong


When added the test case following into FileSystemItCaseBase:
{code:java}
// create table
tableEnv.executeSql(
  s"""
 |create table test2 (
 |  c0 decimal(10,0), c1 int
 |) with (
 |  'connector' = 'filesystem',
 |  'path' = '/Users/zhongxuyang/test/test',
 |  'format' = 'testcsv'
 |)
   """.stripMargin
)

//test file content is:
//2113554011,1
//2113554022,2
{code}
and
{code:java}
// select sql
@Test
def myTest2(): Unit={
  check(
"SELECT c0 FROM test2",
Seq(
  row(2113554011),
  row(2113554022)
))
}
{code}
i got an exception :

java.lang.RuntimeException: Failed to fetch next 
resultjava.lang.RuntimeException: Failed to fetch next result at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
 at 
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
 at 
org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370)
 at java.util.Iterator.forEachRemaining(Iterator.java:115) at 
org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:109) at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:300)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:140)
 at 
org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:106)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.check(BatchFileSystemITCaseBase.scala:46)
 at 
org.apache.flink.table.planner.runtime.FileSystemITCaseBase$class.myTest2(FileSystemITCaseBase.scala:128)
 at 
org.apache.flink.table.planner.runtime.batch.sql.BatchFileSystemITCaseBase.myTest2(BatchFileSystemITCaseBase.scala:33)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) at 
org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at 
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at 
org.junit.rules.RunRules.evaluate(RunRules.java:20) at 
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:413) at 
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
 at 
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
 at 
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
 at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)Caused by: 
java.io.IOException: Failed to fetch job execution result at 
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:184)
 at

[jira] [Commented] (FLINK-22449) Casting an invalid constant string to int throws exception from SinkNotNullEnforcer

2021-09-14 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-22449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415315#comment-17415315
 ] 

xuyangzhong commented on FLINK-22449:
-

link https://issues.apache.org/jira/browse/CALCITE-4771

> Casting an invalid constant string to int throws exception from 
> SinkNotNullEnforcer
> ---
>
> Key: FLINK-22449
> URL: https://issues.apache.org/jira/browse/FLINK-22449
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Affects Versions: 1.13.0
>Reporter: Caizhi Weng
>Priority: Minor
>  Labels: auto-deprioritized-major
>
> Add the following test case to {{CalcITCase}} to reproduce this bug:
> {code:scala}
> @Test
> def myTest(): Unit = {
>   checkResult("SELECT CAST('haha' AS INT)", Seq(row(null)))
> }
> {code}
> The exception stack is
> {code}
> Caused by: org.apache.flink.table.api.TableException: Column 'EXPR$0' is NOT 
> NULL, however, a null value is being written into it. You can set job 
> configuration 'table.exec.sink.not-null-enforcer'='drop' to suppress this 
> exception and drop such records silently.
>   at 
> org.apache.flink.table.runtime.operators.sink.SinkNotNullEnforcer.filter(SinkNotNullEnforcer.java:56)
>   at 
> org.apache.flink.table.runtime.operators.sink.SinkNotNullEnforcer.filter(SinkNotNullEnforcer.java:30)
>   at 
> org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:38)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:112)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:93)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)
>   at 
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
>   at 
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
>   at BatchExecCalc$33.processElement(Unknown Source)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:112)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:93)
>   at 
> org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)
>   at 
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
>   at 
> org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
>   at 
> org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317)
>   at 
> org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411)
>   at 
> org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:92)
>   at 
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
>   at 
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)
>   at 
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)
> {code}
> This is because the result type of CAST is inferred as NOT NULL (see 
> {{SqlCastFunction#inferReturnType}} and 
> {{StandardConvertletTable#convertCast}}, the nullability is the same with the 
> input argument), however parsing an invalid string to int will produce null 
> values.
> One way I could think of is to change the result type of CAST to always 
> nullable (at least for some specific types of casting, for example casting 
> from string to int), but as CAST is a very low-level function this might have 
> a big impact (for example, if a rule adds casting, the resulting row type 
> might not be equal to the original row type due to mismatch in nullability).
> So it seems that at the current stage we should set all columns in a select 
> sink to be nullable. However this indicates that one cannot truly rely on the 
> nullability of any result type.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-22601) PushWatermarkIntoScan should produce digest created by Expression

2021-09-09 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412935#comment-17412935
 ] 

xuyangzhong commented on FLINK-22601:
-

link FLINK-22603

> PushWatermarkIntoScan should produce digest created by Expression
> -
>
> Key: FLINK-22601
> URL: https://issues.apache.org/jira/browse/FLINK-22601
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Planner
>Reporter: Jingsong Lee
>Priority: Minor
>  Labels: auto-deprioritized-major, pull-request-available
> Fix For: 1.14.0
>
>
> In PushWatermarkIntoTableSourceScanRuleBase, the digest is created by RexNode 
> instead of Expression, RexNode rely on field index but Expression rely on 
> field name.
> We should adjust it to names.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-22455) FlinkRelBuilder#windowAggregate will throw ClassCastException when function reuse

2021-09-08 Thread xuyangzhong (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-22455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412307#comment-17412307
 ] 

xuyangzhong commented on FLINK-22455:
-

Hi, [~tartarus]. I'm trying to solve this issue recently, and i can't trigger 
this bug using real case by sql/table api and reproduce this question.

It seems like that only using table api will enter this function while using 
sql api don't use FlinkRelBuilder(you can see the code in Function 
LogicalWindowAggregateRuleBase and line 100-113). 

I create some tests like the following: 
{code:java}
// test1
Table result = tenv
 .from("orders")
 .window(Tumble.over(lit(4).seconds()).on($("rowtime")).as("mywindow"))
 .groupBy($("it1"), $("mywindow"))
 .select($("it1"), $("it2").count().as("b"), $("it2").count().as("b"), 
$("mywindow").start(), $("mywindow").end());
{code}
 

 
{code:java}
// test2
Table result = tenv
 .from("orders")
 .window(Tumble.over(lit(4).seconds()).on($("rowtime")).as("mywindow"))
 .groupBy($("it1"), $("mywindow"))
 .select($("it1"), $("it2").count().as("b1"), $("it2").count().as("b2"), 
$("mywindow").start(), $("mywindow").end());{code}
 

Test1 throws Exception directly because in function validateAndGetUniqueNames 
in ProjectionOperationFactory, names will be checked to avoid duplication. 
Test2 enters the function the issue mentioned but _aggCalls_ has one element 
because in the previous optimization, the $("it2").count will be merged to one 
expression although they have the different name "b1" and "b2".

According to the above, i think this code this issue mentioned perhaps have the 
logical bug indeed but won't be trigged by the real case by table/sql api.

So can you have a test case used by sql/table api  to help me solve the issue?

> FlinkRelBuilder#windowAggregate will throw ClassCastException when function 
> reuse
> -
>
> Key: FLINK-22455
> URL: https://issues.apache.org/jira/browse/FLINK-22455
> Project: Flink
>  Issue Type: Bug
>  Components: Table SQL / Planner
>Reporter: tartarus
>Priority: Minor
>  Labels: auto-deprioritized-major
> Attachments: FlinkRelBuilderTest.scala
>
>
> If the input parameter aggCalls of FlinkRelBuilder#windowAggregate contains 
> the same aggregate function. Then it will throw ClassCastException, because 
> of the optimization of aggregate function reuse. We did not judge the return 
> value type, but direct type conversion;
> {code:java}
> val aggregate = super.transform(
>   new UnaryOperator[RelBuilder.Config] {
> override def apply(t: RelBuilder.Config)
>   : RelBuilder.Config = t.withPruneInputOfAggregate(false)
>   })
>   .push(build())
>   .aggregate(groupKey, aggCalls)
>   .build()
>   .asInstanceOf[LogicalAggregate]
> {code}
> I wrote a test that triggered this problem.
> You can use the attached code to reproduce this problem.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)