[jira] [Commented] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722754#comment-16722754 ] Stamatis Zampetakis commented on CALCITE-2741: -- Thanks for the clarification [~hhlai1990]! It appears that you have made quite a lot of progress on it. Maybe it would be a good idea to submit a PR so that we can discuss on something more concrete. > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Assignee: Julian Hyde >Priority: Minor > > [~julianhyde], > I extended the native enummerable implemention of calcite to support Hive sql > ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache > Drills. > I modified the parser,type systems,and bridge the hive operator . > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722744#comment-16722744 ] Lai Zhou edited comment on CALCITE-2741 at 12/17/18 7:31 AM: - [~julianhyde], I have seen what you did for Oracle functions . But We need to do more to support hive operators. Here is what I did: 1.Add 'fun=hive' config, register all op to a HiveSqlOperatorTable, so a SqlCall can be converted dynamically to a RexCall which holds a hive operator instance. I think it'd be better not to reuse the op instances of SqlStdOperatorTable, because the types of a hive operator is always not deterministic, the data type of a hive operator's output result depends on the data type of it's input parameters. We use a HiveOperatorWrapper to resolve the hive GenericUDF dynamically , when deriveType for a SqlCall ,it creates a instance of GenericUDF, call the initialize method to get the correct result type. So we need to add a sql type mapping from java to hive. 2.Define Implementor for hive operators.We have to modify the RexImplTable to define Implementor for hive operator, because we want to reuse the enumerable implemention as far as possible. It will be reasonable to give an extension hook to define user's Implementor , may be I have not found the right way . 3.To improve performance of the executing for a EnumerableRel, we inject some final fields into the generated class 'Baz',which holds the GenericUDF instance, to avoid creating new instance repeatedly . Besides，we do a little modification for the Parser to support some special operator ,such as rlike ,regexp (where c rlike '...') All above modifications is done on calcite 1.17.0. Now can you give me some suggestions, what's the right way to get all these things done ? was (Author: hhlai1990): [~julianhyde], I have seen what you did for Oracle functions . But We need to do more to support hive operators. Here is what I did: 1.Add 'fun=hive' config, register all op to a HiveSqlOperatorTable, so a SqlCall can be converted dynamically to a RexCall which holds a hive operator instance. I think it'd be better not to reuse the op instances of SqlStdOperatorTable, because the types of a hive operator is always not deterministic, the data type of a hive operator's output result depends on the data type of it's input parameters. We use a HiveOperatorWrapper to resolve the hive GenericUDF dynamically , when deriveType for a SqlCall ,it creates a instance of GenericUDF, call the initialize method to get the correct result type. So we need to add a sql type mapping from java to hive. 2.Define Implementor for hive operators.We have to modify the RexImplTable to define Implementor for hive operator, because we want to reuse the enumerable implemention as far as possible. It will be reasonable to give an extension hook to define user's Implementor , may be I have not found the right way . 3.To improve performance of the executing for a EnumerableRel, we inject some final fields to the generated class 'Baz',which holds the GenericUDF instance, to avoid creating new instance repeatedly . Besides，we do a little modification for the Parser to support some special operator ,such as rlike ,regexp (where c rlike '...') All above modifications is done on calcite 1.17.0. Now can you give me some suggestions, what's the right way to get all these things done ? > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Assignee: Julian Hyde >Priority: Minor > > [~julianhyde], > I extended the native enummerable implemention of calcite to support Hive sql > ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache > Drills. > I modified the parser,type systems,and bridge the hive operator . > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722744#comment-16722744 ] Lai Zhou commented on CALCITE-2741: --- [~julianhyde], I have seen what you did for Oracle functions . But We need to do more to support hive operators. Here is what I did: 1.Add 'fun=hive' config, register all op to a HiveSqlOperatorTable, so a SqlCall can be converted dynamically to a RexCall which holds a hive operator instance. I think it'd be better not to reuse the op instances of SqlStdOperatorTable, because the types of a hive operator is always not deterministic, the data type of a hive operator's output result depends on the data type of it's input parameters. We use a HiveOperatorWrapper to resolve the hive GenericUDF dynamically , when deriveType for a SqlCall ,it creates a instance of GenericUDF, call the initialize method to get the correct result type. So we need to add a sql type mapping from java to hive. 2.Define Implementor for hive operators.We have to modify the RexImplTable to define Implementor for hive operator, because we want to reuse the enumerable implemention as far as possible. It will be reasonable to give an extension hook to define user's Implementor , may be I have not found the right way . 3.To improve performance of the executing for a EnumerableRel, we inject some final fields to the generated class 'Baz',which holds the GenericUDF instance, to avoid creating new instance repeatedly . Besides，we do a little modification for the Parser to support some special operator ,such as rlike ,regexp (where c rlike '...') All above modifications is done on calcite 1.17.0. Now can you give me some suggestions, what's the right way to get all these things done ? > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Assignee: Julian Hyde >Priority: Minor > > [~julianhyde], > I extended the native enummerable implemention of calcite to support Hive sql > ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache > Drills. > I modified the parser,type systems,and bridge the hive operator . > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Zhong updated CALCITE-2744:
---
Summary: Throws RuntimeException in RelDecorrelator when optimizing a
Semi-Join query with a multi-param aggregate function in subquery (was: throws
RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a
multi-param aggregate function in subquery)
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.18.0
>
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
[jira] [Created] (CALCITE-2744) throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
Wei Zhong created CALCITE-2744:
--
Summary: throws RuntimeException in RelDecorrelator when
optimizing a Semi-Join query with a multi-param aggregate function in subquery
Key: CALCITE-2744
URL: https://issues.apache.org/jira/browse/CALCITE-2744
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.17.0
Reporter: Wei Zhong
Assignee: Julian Hyde
Fix For: 1.18.0
when running the following test in org.apache.calcite.test.RelOptRulesTest, we
will get a RuntimeException:
{code:java}
@Test public void testDecorrelateWith2ParamsAgg() {
final HepProgram preProgram =
HepProgram.builder()
.addRuleInstance(FilterProjectTransposeRule.INSTANCE)
.addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
.addRuleInstance(ProjectMergeRule.INSTANCE)
.build();
final HepProgram program =
HepProgram.builder()
.addRuleInstance(SemiJoinRule.PROJECT)
.build();
final String sql = "select * from dept where exists (\n"
+ "select UDF_AVG(sal, sal + 1) from sales.emp\n"
+ "where emp.deptno = dept.deptno\n"
+ "and emp.sal > 100)";
Sql sqlTest = sql(sql)
.withDecorrelation(true)
.withTrim(true)
.withPre(preProgram)
.with(program);
sqlTest.check();
}
{code}
UDF_AVG code in MockSqlOperatorTable:
{code:java}
public static void addRamp(MockSqlOperatorTable opTab) {
// Don't use anonymous inner classes. They can't be instantiated
// using reflection when we are deserializing from JSON.
opTab.addOperator(new RampFunction());
opTab.addOperator(new DedupFunction());
opTab.addOperator(UDF_AVG);
}
public static final SqlFunction UDF_AVG = new SqlAggFunction(
"UDF_AVG",
null,
SqlKind.OTHER_FUNCTION,
ReturnTypes.AVG_AGG_FUNCTION,
null,
OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
SqlFunctionCategory.NUMERIC,
false,
false,
Optionality.FORBIDDEN) {
@Override public boolean isDeterministic() {
return false;
}
};
{code}
The RuntimeExcpetion detail:
{code:java}
java.lang.RuntimeException: While invoking method 'public
org.apache.calcite.sql2rel.RelDecorrelator$Frame
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
at
org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
at
org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
at
org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
at
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
at org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
at
org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
[jira] [Created] (CALCITE-2743) TimeStamp confused in avatica jdbc
shining created CALCITE-2743:
Summary: TimeStamp confused in avatica jdbc
Key: CALCITE-2743
URL: https://issues.apache.org/jira/browse/CALCITE-2743
Project: Calcite
Issue Type: Bug
Components: avatica
Affects Versions: 1.10.0
Reporter: shining
I use Phoenix Query Server through avatica, Operation is as follows：
1. create table with sqlline-thin
CREATE TABLE test_timezone(log_time TIMESTAMP NOT NULL PRIMARY KEY, id
VARCHAR(40));
2. Upset data
upsert into test_timezone values('2018-11-27 11:01:59.000','1’);
3. Query
0: jdbc:phoenix:thin:url=http://localhost:876> select * from test_timezone;
+--+-+
| LOG_TIME | ID |
+-+--+
| 2018-11-27 03:01:59 | 1 |
+--+——+
My local timeZone is GMT+8, and configured Phoenix
“phoenix.query.dateFormatTimeZone=GMT+8”
I also view code of avatica， when the timezone is GMT+8， getTimeStamp method
will lose 8 hours：
public Timestamp getTimestamp(Calendar calendar) throws SQLException {
java.util.Date date = (java.util.Date) getObject();
if (date == null) {
return null;
}
long v = date.getTime();
if (calendar != null) {
v -= calendar.getTimeZone().getOffset(v);
}
return new Timestamp(v);
}
sqlline-thin use getString() method get the timestamp，it pass a null timezone
to timestampAsString()
So I have two doubtful places here：
1）I get correct time from phoenixResultSet， why reduce 8 hours ？
2）Can getString method be returned by getTimeStamp().toString():
public String getString() throws SQLException {
final long v = getLong();
if (v == 0 && wasNull()) {
return null;
}
return getTimeStamp(v, null).toString();
}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2741) Add operator table with Hive-specific built-in functions
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lai Zhou updated CALCITE-2741: -- Summary: Add operator table with Hive-specific built-in functions (was: extend the native enumerable implemention of calcite to support Hive SQL ) > Add operator table with Hive-specific built-in functions > > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Assignee: Julian Hyde >Priority: Minor > > [~julianhyde], > I extended the native enummerable implemention of calcite to support Hive sql > ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache > Drills. > I modified the parser,type systems,and bridge the hive operator . > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[
https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722578#comment-16722578
]
Jacques Nadeau commented on CALCITE-2736:
-
{quote}Regardless of which release it goes into, the change is producing
warnings relating to calls to deprecated APIs, so it shouldn't be in master. I
have put it onto a branch
[https://github.com/julianhyde/calcite/tree/2736-reduce-options] and
force-pushed to remove it from master. Let's merge it after the release, and
after the warnings are fixed.
{quote}
Can you share a link where this was decided (master should have no deprecation
warnings)? I'd like to review to better understand the thinking behind having
this rule but not enforcing in the build or pre-commit checks.
{quote}Policy is to release from master, not a spur. It produces a simple
change log, and holding back merges around release time is not too onerous for
a project of this velocity.
{quote}
I agree that a policy shouldn't be unilaterally dismissed without discussion.
That was definitely not my intent with this merge. The merge of Jesus's patch
without any discussion on the release thread was the second indicator that the
branch was open to me (beyond the lack of communication for some time on the
release thread). Per my comments above, there should be a clearer way to expose
this status. It would also preferably be time-bounded.
Side note: I disagree with your thoughts on the onerousness of this approach of
branch management. Two plus weeks of master lock is too long. I've hit this
several times in Calcite and I'd be surprised if I was alone. There is minimal
committer benefit of having a pure linear history for a few commit release spur
(even if you have to do a couple of cherry picks). I'll consider bringing this
up on the mailing list.
{quote}I don't want even minor changes to be added to this RC. Hive's regress
is in progress, and takes several days.
{quote}
It would have been nice to have clearer communication on the mailing list
around your goals with regards to the release and the timing. Until your post
today, your intentions, actions and timing weren't super clear.
> ReduceExpressionsRule never reduces dynamic expressions but this should be
> configurable
> ---
>
> Key: CALCITE-2736
> URL: https://issues.apache.org/jira/browse/CALCITE-2736
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jacques Nadeau
>Assignee: Jacques Nadeau
>Priority: Major
> Fix For: 1.19.0
>
>
> There are situations where it is helpful to reduce dynamic SqlCalls. Right
> now, the ReduceExpressionsRule always avoids doing this. We should enhance
> the rule so this can be configurable depending on where in planning the rule
> is used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2741) extend the native enumerable implemention of calcite to support Hive SQL
[ https://issues.apache.org/jira/browse/CALCITE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722570#comment-16722570 ] Julian Hyde commented on CALCITE-2741: -- Could you also please add the Oracle-specific functions to the SQL reference. They are listed in https://calcite.apache.org/apidocs/org/apache/calcite/sql/fun/OracleSqlOperatorTable.html. > extend the native enumerable implemention of calcite to support Hive SQL > -- > > Key: CALCITE-2741 > URL: https://issues.apache.org/jira/browse/CALCITE-2741 > Project: Calcite > Issue Type: New Feature > Components: core >Reporter: Lai Zhou >Assignee: Julian Hyde >Priority: Minor > > [~julianhyde], > I extended the native enummerable implemention of calcite to support Hive sql > ,include UDF、UDAF and all the SqlSpecialOperator,which inspired by apache > Drills. > I modified the parser,type systems,and bridge the hive operator . > How do you think of supporting a direct implemention of hive sql like this? > I think it will be valueable when someone want to migrate his hive etl jobs > to real-time scene. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2674) SqlIdentifier same name with built-in function but with escape character should be still resolved as an identifier
[
https://issues.apache.org/jira/browse/CALCITE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-2674:
-
Fix Version/s: (was: 1.18.0)
1.19.0
> SqlIdentifier same name with built-in function but with escape character
> should be still resolved as an identifier
> --
>
> Key: CALCITE-2674
> URL: https://issues.apache.org/jira/browse/CALCITE-2674
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Yuzhao Chen
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.19.0
>
>
> Now If user defines table as schema below:
> {code:java}
> create table tt(
> `current_time` int,
> b int);
> {code}
> Then start a query:
> {code:java}
> select `current_time` from tt;
> {code}
> Calcite parser will parse the *current_time* as a normal *SqlIdentifier*,
> then the SqlValidator will recognize it as a builtin function with below code
> snippet:
> {code:java}
> public static SqlCall makeCall(
> SqlOperatorTable opTab,
> SqlIdentifier id) {
> if (id.names.size() == 1) {
> final List list = new ArrayList<>();
> opTab.lookupOperatorOverloads(id, null, SqlSyntax.FUNCTION, list);
> for (SqlOperator operator : list) {
> if (operator.getSyntax() == SqlSyntax.FUNCTION_ID) {
> // Even though this looks like an identifier, it is a
> // actually a call to a function. Construct a fake
> // call to this function, so we can use the regular
> // operator validation.
> return new SqlBasicCall(
> operator,
> SqlNode.EMPTY_ARRAY,
> id.getParserPosition(),
> true,
> null);
> }
> }
> }
> return null;
> }{code}
> While i tried MYSQL and such query can work properly, so is this a bug ?
> Cause if the matched function's return type is same with the column, then the
> query will output wrong results.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde reopened CALCITE-2736: -- Regardless of which release it goes into, the change is producing warnings relating to calls to deprecated APIs, so it shouldn't be in master. I have put it onto a branch https://github.com/julianhyde/calcite/tree/2736-reduce-options and force-pushed to remove it from master. Let's merge it after the release, and after the warnings are fixed. Policy is to release from master, not a spur. It produces a simple change log, and holding back merges around release time is not too onerous for a project of this velocity. I don't want even minor changes to be added to this RC. Hive's regress is in progress, and takes several days. > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Fix For: 1.19.0 > > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722532#comment-16722532 ] Jacques Nadeau commented on CALCITE-2736: - Hey [~julianhyde], sorry about that. I didn't realize 1.18 wasn't already out. I looked for the release thread and saw it wasn't active for 10 days or something and foolishly assumed that meant the release was done. What do you think is the best way to track that master is closed so others don't miss that the way I did? Also, how important is locking? Seems like having a little release spur would be fine. Another option is to include this patch in next release candidate. It doesn't change much except configurability of the one rule. > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Fix For: 1.19.0 > > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
