[jira] [Created] (CALCITE-6399) The predicate IN is transformed by the sqltorelConverter into join
kate created CALCITE-6399:
-
Summary: The predicate IN is transformed by the sqltorelConverter
into join
Key: CALCITE-6399
URL: https://issues.apache.org/jira/browse/CALCITE-6399
Project: Calcite
Issue Type: Wish
Components: core
Affects Versions: 1.36.0
Reporter: kate
*Simple Example：*
My SqlToRelConverter：
{code:java}
SqlToRelConverter.Config config = new SqlToRelNodeConfig()
.withTrimUnusedFields(true)
.withRelBuilderFactory(RelFactories.LOGICAL_BUILDER)
.withRelBuilderConfigTransform(relBuilderConfig -> relBuilderConfig
.withSimplify(false))
.withHintStrategyTable(HintStrategyTable.EMPTY);
RelRoot root = sqlToRelConverter.convertQuery(sqlNode, false, false);
{code}
SQL
{code:java}
SELECT cd_gender,
cd_marital_status,
cd_education_status, cd_purchase_estimate,
cd_credit_rating, cd_dep_count,
cd_dep_employed_count, cd_dep_college_countFROM
pg.tpcds.customer c,
pg.tpcds. customer_address ca,
pg.tpcds.customer_demographics
WHERE c.c_current_addr_sk = ca.ca_address_sk
AND cd_demo_sk = c.c_current_cdemo_sk
AND ca_county IN ( 'Lycoming County', 'Sheridan County', 'Kandiyohi
County',Pike County','Greene County' )
{code}
RelNode tree after converting
{code:java}
LogicalProject(cd_gender=[$32], cd_marital_status=[$33],
cd_education_status=[$34], cd_purchase_estimate=[$35], cd_credit_rating=[$36],
cd_dep_count=[$37], cd_dep_employed_count=[$38], cd_dep_college_count=[$39])
LogicalFilter(condition=[AND(=($4, $18), true, =($31, $2))])
LogicalJoin(condition=[=($25, $40)], joinType=[inner])
LogicalJoin(condition=[true], joinType=[inner])
LogicalJoin(condition=[true], joinType=[inner])
LogicalTableScan(table=[[tpcds, customer]])
LogicalTableScan(table=[[tpcds, customer_address]])
LogicalTableScan(table=[[tpcds, customer_demographics]])
LogicalAggregate(group=[{0}])
LogicalUnion(all=[true])
LogicalValues(tuples=[[{ 'Lycoming County' }]])
LogicalValues(tuples=[[{ 'Sheridan County' }]])
LogicalValues(tuples=[[{ 'Kandiyohi County' }]])
LogicalValues(tuples=[[{ 'Pike County' }]])
LogicalValues(tuples=[[{ 'Greene County' }]]) {code}
I'm trying to figure out how to keep in as a filter instead of converting it to
a join value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6398) Spark's ifnull function has only one parameter and the return value is Boolean
[
https://issues.apache.org/jira/browse/CALCITE-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843270#comment-17843270
]
Caican Cai commented on CALCITE-6398:
-
I mistook isnull for ifnull, sorry
> Spark's ifnull function has only one parameter and the return value is Boolean
> --
>
> Key: CALCITE-6398
> URL: https://issues.apache.org/jira/browse/CALCITE-6398
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Priority: Major
> Fix For: 1.37.0
>
>
> Spark's ifnull function has only one parameter and the return value is Boolean
> [https://spark.apache.org/docs/2.3.0/api/sql/index.html#isnull]
>
> But calcite's ifnull has two parameters and returns expr
> {code:java}
> | b s | IFNULL(value1, value2) | Equivalent to
> `NVL(value1, value2)`{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (CALCITE-6398) Spark's ifnull function has only one parameter and the return value is Boolean
[
https://issues.apache.org/jira/browse/CALCITE-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caican Cai closed CALCITE-6398.
---
Resolution: Not A Bug
> Spark's ifnull function has only one parameter and the return value is Boolean
> --
>
> Key: CALCITE-6398
> URL: https://issues.apache.org/jira/browse/CALCITE-6398
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Priority: Major
> Fix For: 1.37.0
>
>
> Spark's ifnull function has only one parameter and the return value is Boolean
> [https://spark.apache.org/docs/2.3.0/api/sql/index.html#isnull]
>
> But calcite's ifnull has two parameters and returns expr
> {code:java}
> | b s | IFNULL(value1, value2) | Equivalent to
> `NVL(value1, value2)`{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6398) Spark's ifnull function has only one parameter and the return value is Boolean
[
https://issues.apache.org/jira/browse/CALCITE-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843267#comment-17843267
]
Caican Cai commented on CALCITE-6398:
-
Sorry, I saw it wrong
> Spark's ifnull function has only one parameter and the return value is Boolean
> --
>
> Key: CALCITE-6398
> URL: https://issues.apache.org/jira/browse/CALCITE-6398
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Priority: Major
> Fix For: 1.37.0
>
>
> Spark's ifnull function has only one parameter and the return value is Boolean
> [https://spark.apache.org/docs/2.3.0/api/sql/index.html#isnull]
>
> But calcite's ifnull has two parameters and returns expr
> {code:java}
> | b s | IFNULL(value1, value2) | Equivalent to
> `NVL(value1, value2)`{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-6398) Spark's ifnull function has only one parameter and the return value is Boolean
Caican Cai created CALCITE-6398:
---
Summary: Spark's ifnull function has only one parameter and the
return value is Boolean
Key: CALCITE-6398
URL: https://issues.apache.org/jira/browse/CALCITE-6398
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.36.0
Reporter: Caican Cai
Fix For: 1.37.0
Spark's ifnull function has only one parameter and the return value is Boolean
[https://spark.apache.org/docs/2.3.0/api/sql/index.html#isnull]
But calcite's ifnull has two parameters and returns expr
{code:java}
| b s | IFNULL(value1, value2) | Equivalent to
`NVL(value1, value2)`{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-6352) The map_contains_key function may return true when the key and mapkeytype types are different.
[
https://issues.apache.org/jira/browse/CALCITE-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843235#comment-17843235
]
Caican Cai edited comment on CALCITE-6352 at 5/3/24 3:40 PM:
-
{code:java}
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2):
boolean]scala> df.show()
+--+
|map_contains_key(map(1, a, 2.0, b), 2)|
+--+
| true|
+--+
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2f)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2.0):
boolean]scala> df.show()
++
|map_contains_key(map(1, a, 2.0, b), 2.0)|
++
| true|
++
{code}
[~mbudiu] Hello, It seems that as long as the numbers are the same, true can be
returned, regardless of whether the number type is double, int, or float.
i submit a pr
was (Author: JIRAUSER302115):
{code:java}
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2):
boolean]scala> df.show()
+--+
|map_contains_key(map(1, a, 2.0, b), 2)|
+--+
| true|
+--+
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2f)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2.0):
boolean]scala> df.show()
++
|map_contains_key(map(1, a, 2.0, b), 2.0)|
++
| true|
++
{code}
[~mbudiu] Hello, It seems that as long as the numbers are the same, true can be
returned, regardless of whether the number type is double, int, or float.
> The map_contains_key function may return true when the key and mapkeytype
> types are different.
> --
>
> Key: CALCITE-6352
> URL: https://issues.apache.org/jira/browse/CALCITE-6352
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Assignee: Caican Cai
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
>
> {code:java}
> scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2, 'b'),
> 2.0)")
> val df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2, b),
> 2.0): boolean]
> scala> df.show()
> +--+
> |map_contains_key(map(1, a, 2, b), 2.0)|
> +--+
> | true|
> +--+
> {code}
> calcite return false
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6352) The map_contains_key function may return true when the key and mapkeytype types are different.
[
https://issues.apache.org/jira/browse/CALCITE-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-6352:
Labels: pull-request-available (was: )
> The map_contains_key function may return true when the key and mapkeytype
> types are different.
> --
>
> Key: CALCITE-6352
> URL: https://issues.apache.org/jira/browse/CALCITE-6352
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Assignee: Caican Cai
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
>
> {code:java}
> scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2, 'b'),
> 2.0)")
> val df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2, b),
> 2.0): boolean]
> scala> df.show()
> +--+
> |map_contains_key(map(1, a, 2, b), 2.0)|
> +--+
> | true|
> +--+
> {code}
> calcite return false
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6352) The map_contains_key function may return true when the key and mapkeytype types are different.
[
https://issues.apache.org/jira/browse/CALCITE-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843235#comment-17843235
]
Caican Cai commented on CALCITE-6352:
-
{code:java}
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2):
boolean]scala> df.show()
+--+
|map_contains_key(map(1, a, 2.0, b), 2)|
+--+
| true|
+--+
scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2.0, 'b'), 2f)")
df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2.0, b), 2.0):
boolean]scala> df.show()
++
|map_contains_key(map(1, a, 2.0, b), 2.0)|
++
| true|
++
{code}
[~mbudiu] Hello, It seems that as long as the numbers are the same, true can be
returned, regardless of whether the number type is double, int, or float.
> The map_contains_key function may return true when the key and mapkeytype
> types are different.
> --
>
> Key: CALCITE-6352
> URL: https://issues.apache.org/jira/browse/CALCITE-6352
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Caican Cai
>Assignee: Caican Cai
>Priority: Critical
> Fix For: 1.37.0
>
>
>
> {code:java}
> scala> val df = spark.sql("select map_contains_key(map(1, 'a', 2, 'b'),
> 2.0)")
> val df: org.apache.spark.sql.DataFrame = [map_contains_key(map(1, a, 2, b),
> 2.0): boolean]
> scala> df.show()
> +--+
> |map_contains_key(map(1, a, 2, b), 2.0)|
> +--+
> | true|
> +--+
> {code}
> calcite return false
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6397) Add NVL2 function (enabled in Spark library)
[ https://issues.apache.org/jira/browse/CALCITE-6397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-6397: Labels: pull-request-available (was: ) > Add NVL2 function (enabled in Spark library) > > > Key: CALCITE-6397 > URL: https://issues.apache.org/jira/browse/CALCITE-6397 > Project: Calcite > Issue Type: New Feature >Reporter: Caican Cai >Priority: Major > Labels: pull-request-available > > Add NVL2 function (enabled in Spark library) > > https://spark.apache.org/docs/2.3.0/api/sql/index.html#nvl2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6397) Add NVL2 function (enabled in Spark library)
Caican Cai created CALCITE-6397: --- Summary: Add NVL2 function (enabled in Spark library) Key: CALCITE-6397 URL: https://issues.apache.org/jira/browse/CALCITE-6397 Project: Calcite Issue Type: New Feature Reporter: Caican Cai Add NVL2 function (enabled in Spark library) https://spark.apache.org/docs/2.3.0/api/sql/index.html#nvl2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (CALCITE-6387) Calcite build while compiliation with jdk17+
[ https://issues.apache.org/jira/browse/CALCITE-6387 ]
Caican Cai deleted comment on CALCITE-6387:
-
was (Author: JIRAUSER302115):
[~Sergey Nuyanzin] Hello, does the corresponding document need to be modified?
./gradlew build seems to be unable to compile successfully.
> Calcite build while compiliation with jdk17+
>
>
> Key: CALCITE-6387
> URL: https://issues.apache.org/jira/browse/CALCITE-6387
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.36.0
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
> The issue appears with newly added Arrow adapter which requires
> {noformat}
> --add-opens=java.base/java.nio=ALL-UNNAMED
> {noformat}
> could be fixed with adding
> {noformat}
> plugins.withType {
> tasks {
> configureEach {
> jvmArgs("-XX:+IgnoreUnrecognizedVMOptions")
> jvmArgs("--add-opens=java.base/java.nio=ALL-UNNAMED")
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6387) Calcite build while compiliation with jdk17+
[
https://issues.apache.org/jira/browse/CALCITE-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843161#comment-17843161
]
Caican Cai commented on CALCITE-6387:
-
[~Sergey Nuyanzin] Hello, does the corresponding document need to be modified?
./gradlew build seems to be unable to compile successfully.
> Calcite build while compiliation with jdk17+
>
>
> Key: CALCITE-6387
> URL: https://issues.apache.org/jira/browse/CALCITE-6387
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.36.0
>Reporter: Sergey Nuyanzin
>Assignee: Sergey Nuyanzin
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
> The issue appears with newly added Arrow adapter which requires
> {noformat}
> --add-opens=java.base/java.nio=ALL-UNNAMED
> {noformat}
> could be fixed with adding
> {noformat}
> plugins.withType {
> tasks {
> configureEach {
> jvmArgs("-XX:+IgnoreUnrecognizedVMOptions")
> jvmArgs("--add-opens=java.base/java.nio=ALL-UNNAMED")
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6396) Add ADD_MONTHS function (enabled in Spark library)
[ https://issues.apache.org/jira/browse/CALCITE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-6396: Labels: pull-request-available (was: ) > Add ADD_MONTHS function (enabled in Spark library) > -- > > Key: CALCITE-6396 > URL: https://issues.apache.org/jira/browse/CALCITE-6396 > Project: Calcite > Issue Type: New Feature > Components: core >Affects Versions: 1.36.0 >Reporter: Caican Cai >Priority: Minor > Labels: pull-request-available > Fix For: 1.37.0 > > > Add ADD_MONTHS function (enabled in Spark library) > > https://spark.apache.org/docs/2.3.0/api/sql/index.html#add_months -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6396) Add ADD_MONTHS function (enabled in Spark library)
Caican Cai created CALCITE-6396: --- Summary: Add ADD_MONTHS function (enabled in Spark library) Key: CALCITE-6396 URL: https://issues.apache.org/jira/browse/CALCITE-6396 Project: Calcite Issue Type: New Feature Components: core Affects Versions: 1.36.0 Reporter: Caican Cai Fix For: 1.37.0 Add ADD_MONTHS function (enabled in Spark library) https://spark.apache.org/docs/2.3.0/api/sql/index.html#add_months -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6393) Byte code of SqlFunctions is invalid
[
https://issues.apache.org/jira/browse/CALCITE-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843139#comment-17843139
]
Ruben Q L commented on CALCITE-6393:
Thanks for creating the PR [~zabetak]!
[~MasseGuillaume], do you think this new check would need to verify both cases
on your original code snippet (checkClass true/false), considering that both
have unveiled different exceptions?
> Byte code of SqlFunctions is invalid
>
>
> Key: CALCITE-6393
> URL: https://issues.apache.org/jira/browse/CALCITE-6393
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.36.0
>Reporter: Sergey Nuyanzin
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
>
> The issue is a result of testing of Apache Calcite 1.37.0 rc 4 in this thread
> [1]
> There is test project andprocedure provided by [~MasseGuillaume] [2] (see
> also original thread where this was first discussed [3])
> it shows that since Calcite 1.36.0 it starts failing as
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of bounds for
> length 297
> at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2695)
> at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2711)
> at
> org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2777)
> at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1929)
> at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1515)
> at org.objectweb.asm.ClassReader.accept(ClassReader.java:745)
> {noformat}
> Also since Calcite 1.27.0 it starts failing as
> {noformat}
> java.lang.IllegalArgumentException: Invalid end label (must be visited
> first)
> at
> org.objectweb.asm.util.CheckMethodAdapter.checkLabel(CheckMethodAdapter.java:1453)
> at
> org.objectweb.asm.util.CheckMethodAdapter.visitLocalVariableAnnotation(CheckMethodAdapter.java:996)
> at
> org.objectweb.asm.MethodVisitor.visitLocalVariableAnnotation(MethodVisitor.java:757)
> at
> org.objectweb.asm.commons.MethodRemapper.visitLocalVariableAnnotation(MethodRemapper.java:257)
> at org.objectweb.asm.ClassReader.readCode(ClassReader.java:2614)
> at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1515)
> {noformat}
> [1] https://lists.apache.org/thread/n6cs1l86mt6fc5q8pcxr97czs3p6w65f
> [2] https://github.com/MasseGuillaume/asm-remapper-bug
> [3] https://lists.apache.org/thread/o736wz4qnr4l285bj5gv073cy0qll9t0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
