[jira] [Created] (CALCITE-2161) Add "types" collection to the schema element in the JSON model

[Shuyi Chen (JIRA)]

Shuyi Chen created CALCITE-2161:
---

 Summary: Add "types" collection to the schema element in the JSON 
model
 Key: CALCITE-2161
 URL: https://issues.apache.org/jira/browse/CALCITE-2161
 Project: Calcite
  Issue Type: Task
  Components: core
Affects Versions: 1.16.0
Reporter: Shuyi Chen
Assignee: Julian Hyde


As discussed in CALCITE-2045, we need to add "types" collection in the schema 
element in the JSON model.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2160) Spatial grid index

[Julian Hyde (JIRA)]

Julian Hyde created CALCITE-2160:


 Summary: Spatial grid index
 Key: CALCITE-2160
 URL: https://issues.apache.org/jira/browse/CALCITE-2160
 Project: Calcite
  Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde


Recognize materialized views that convert a geometry into a grid of rectangles, 
and use such materialized views for point-to-polygon and polgyon-to-polygon 
joins.

Implement the [ST_MakeGrid|http://www.h2gis.org/docs/dev/ST_MakeGrid] and 
[ST_MakeGridPoints|http://www.h2gis.org/docs/dev/ST_MakeGridPoints] functions.

Add a test data set based upon western U.S. states and national parks. Given a 
materialized view

{code}CREATE MATERIALIZED VIEW StateGrids (name, x, y, PRIMARY KEY (name)) AS
SELECT s.name, ST_XMin(g.the_geom), ST_YMin(g.the_geom)
FROM States AS s
CROSS APPLY TABLE(ST_MakeGrid(s.geom, 1, 1)) AS g{code}

and a similar materialized view ParkGrids on Parks, the query

{code}SELECT p.name AS park, s.name AS state
FROM Parks AS p
JOIN States AS s ON ST_Overlaps(s.geom, p.geom)
ORDER BY 1, 2{code}

should return

{noformat}
Park State
 =
Death Valley CA
Death Valley NV
Yellowstone  ID
Yellowstone  MT
Yellowstone  WY
Yosemite CA
{noformat}

and should semi-join to the {{StateGrids}} and {{ParkGrids}} tables to reduce 
the size of the input before applying St_Overlaps: {code}SELECT p.name AS park, 
s.name AS state
FROM Parks AS p
JOIN States AS s ON ST_Overlaps(s.geom, p.geom)
WHERE (p.name, s.name) IN (
  SELECT DISTINCT pg.name, sg.name
  FROM ParkGrids AS pg
  JOIN StateGrids AS sg ON pg.id = sg.id){code}

Note the semi-join, to remove duplicates in case a park and a state have 
several cells that overlap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2159) UNNEST to support 'ANY' type

[Chunhui Shi (JIRA)]

Chunhui Shi created CALCITE-2159:


 Summary: UNNEST to support 'ANY' type
 Key: CALCITE-2159
 URL: https://issues.apache.org/jira/browse/CALCITE-2159
 Project: Calcite
  Issue Type: Bug
Reporter: Chunhui Shi
Assignee: Julian Hyde


Not all data source has type information about the input of UNNEST during 
parsing stage. In Drill, if we want to support UNNEST(table.column) syntax for 
a document with nested structure, for now, these two things will happen:

SqlUnnestOperator.inferReturnType will use unknown operand's type 'ANY' so 
isStruct will be false, thus the the following code will hit NULL reference.

 

Another issue is, Should UnnestnameSpace.getTable return a table so when other 
parts of the query tried to refer to some columns coming out of UNNEST, we know 
the query is asking for a column from the table, so the parser could add the 
column to the RowType of UNNEST? An example query is like this:

SELECT AVG(o.o_amount) AS avg_orders FROM  UNNEST(c.orders) AS o

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Pull requests

[Michael Mior]

I've been working on the Geode adapter but I'm waiting for a response from
Christian. I merged the updates to calcite-test-dataset but the integration
tests in the PR are failing and there's a few stylistic issues that need to
be resolved.

--
Michael Mior
mm...@apache.org

2018-01-30 15:40 GMT-05:00 Julian Hyde :

> Can someone please review and commit https://issues.apache.org/
> jira/browse/CALCITE-508 
> (Avatica) and https://issues.apache.org/jira/browse/CALCITE-2059 <
> https://issues.apache.org/jira/browse/CALCITE-2059> (Geode adapter). I am
> burning out.
>
> Also, we need an Avatica release.
>
> Julian
>
>

Pull requests

[Julian Hyde]

Can someone please review and commit 
https://issues.apache.org/jira/browse/CALCITE-508 
 (Avatica) and 
https://issues.apache.org/jira/browse/CALCITE-2059 
 (Geode adapter). I am 
burning out.

Also, we need an Avatica release. 

Julian

[jira] [Created] (CALCITE-2158) SubQuery with EXISTS clause creates redundant aggregate call

[Volodymyr Vysotskyi (JIRA)]

Volodymyr Vysotskyi created CALCITE-2158:


 Summary: SubQuery with EXISTS clause creates redundant aggregate 
call
 Key: CALCITE-2158
 URL: https://issues.apache.org/jira/browse/CALCITE-2158
 Project: Calcite
  Issue Type: Bug
Reporter: Volodymyr Vysotskyi
Assignee: Julian Hyde


When {{SqlToRelConverter.Config.isExpand()}} returns true, subqueries are 
expanded in {{SqlToRelConverter}}.
Then for the queries, like this:
{code:sql}
SELECT cs1.sal
FROM emp cs1
WHERE EXISTS
(SELECT *
 FROM emp cs2
 WHERE cs1.sal = cs2.sal
   AND cs1.deptno <> cs2.deptno)
{code}
Calcite returns logical plan with excessive aggregate calls:
{noformat}
LogicalProject(SAL=[$5])
  LogicalFilter(condition=[IS NOT NULL($9)])
LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{5, 
7}])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{}], agg#0=[MIN($0)])
LogicalProject($f0=[true])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], 
HIREDATE=[$4], SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO, 
$7))])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
But when {{SqlToRelConverter.Config.isExpand()}} returns false and 
SubQueryRemoveRule rules are applied to the logical plan with RexSubQuery, the 
resulting logical plan is correct and does not contain excessive aggregate 
calls:
{noformat}
LogicalProject(SAL=[$5])
  LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4], 
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
LogicalCorrelate(correlation=[$cor0], joinType=[inner], 
requiredColumns=[{5, 7}])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0}])
LogicalProject(i=[true])
  LogicalFilter(condition=[AND(=($cor0.SAL, $5), <>($cor0.DEPTNO, $7))])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{noformat}
These cases may be observed using this unit test:
{code:java}
  @Test public void testExistsExpand() {
final HepProgram preProgram = HepProgram.builder()
.addRuleInstance(SubQueryRemoveRule.FILTER)
.addRuleInstance(SubQueryRemoveRule.PROJECT)
.addRuleInstance(SubQueryRemoveRule.JOIN)
.build();
final HepProgram program = HepProgram.builder()
.build();
final String sql = "SELECT cs1.sal\n"
+ "FROM emp cs1 \n" 
+ "WHEREEXISTS\n" 
+ "(SELECT *\n" 
+ "FROM   emp cs2\n" 
+ "WHERE  cs1.sal = cs2.sal\n" 
+ "ANDcs1.deptno <> cs2.deptno)";
sql(sql)
.withDecorrelation(false)
.withTrim(false)
.expand(true) // change to false
.withPre(preProgram)
.with(program)
.checkUnchanged();
  }
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2157) ClickHouse dialect implementation

[Chris Baynes (JIRA)]

Chris Baynes created CALCITE-2157:
-

 Summary: ClickHouse dialect implementation
 Key: CALCITE-2157
 URL: https://issues.apache.org/jira/browse/CALCITE-2157
 Project: Calcite
  Issue Type: New Feature
  Components: jdbc-adapter
Reporter: Chris Baynes
Assignee: Chris Baynes


ClickHouse is a really fast columnar DBMS for OLAP: 
[https://clickhouse.yandex/.|https://clickhouse.yandex/]

It has a jdbc adapter and uses mostly standard sql, though there are 
differences (e.g. join syntax, datatypes, function name case-sensitivity).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2156) DateRangeRules not handling timestamp with timezone properly

[Nishant Bangarwa (JIRA)]

Nishant Bangarwa created CALCITE-2156:
-

 Summary: DateRangeRules not handling timestamp with timezone 
properly
 Key: CALCITE-2156
 URL: https://issues.apache.org/jira/browse/CALCITE-2156
 Project: Calcite
  Issue Type: Bug
Reporter: Nishant Bangarwa
Assignee: Nishant Bangarwa


DateRangeRules calculate floor and ceil on timestamps in UTC timezone, instead 
it should do the floor or ceil on timestamp in local timezone when data type is 
timestamp with local timezone. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2155) upgrade slf4j

[Alexey Roytman (JIRA)]

Alexey Roytman created CALCITE-2155:
---

 Summary: upgrade slf4j
 Key: CALCITE-2155
 URL: https://issues.apache.org/jira/browse/CALCITE-2155
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.15.0
Reporter: Alexey Roytman
Assignee: Julian Hyde


Apache Calcite depends on slf4j-api version 1.7.13 which was released on 
2015-11, more than two years ago.

It does not have known CVE vulnerabilities, but we may wish to upgrade it to 
later version. For 2018-01-30, it's 1.7.25.

I hope that fixing pom.xml files and running tests is enough.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-2154) upgrade jackson

[Alexey Roytman (JIRA)]

Alexey Roytman created CALCITE-2154:
---

 Summary: upgrade jackson 
 Key: CALCITE-2154
 URL: https://issues.apache.org/jira/browse/CALCITE-2154
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.15.0
Reporter: Alexey Roytman
Assignee: Julian Hyde


Calcite now uses FasterXML Jackson 2.6.3 that has known security 
vulnerabilities:
 * CVE-2017-7525 is prone to a remote-code execution vulnerability. 
Successfully exploiting this issue allows attackers to execute arbitrary code 
in the context of the affected application. Failed exploits will result in 
denial-of-service conditions.
 * CVE-2017-15095 describes more deserialization exploits for jackson-databind 
as a follow-up to CVE-2017-7525.
 * CVE-2017-17485 is about jackson-databind up to 2.9.3 allowing 
unauthenticated remote code execution because of an incomplete fix for the 
CVE-2017-7525 deserialization flaw.
 * CVE-2018-5968 is about jackson-databind up to 2.9.3 allowing unauthenticated 
remote code execution because of an incomplete fix for the CVE-2017-7525 and 
CVE-2017-17485 deserialization flaws.

Please upgrade to last version of FasterXML Jackson (on 2018-01-30 it's version 
2.9.4).

I hope that fixing pom.xml files and running tests is enough.

(See also JIRA:CALCITE-1021, JIRA:KYLIN-3027)

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)