Calcite-Master - Build # 269 - Failure

2018-05-07 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #269) Status: Failure Check console output at https://builds.apache.org/job/Calcite-Master/269/ to view the results.

Re: Approximate query processing in Calcite

2018-05-07 Thread Xiening Dai
However, we are interested in exploring ways that a deeper integration might be beneficial especially in the case of federated query processing across multiple database backends. Can you elaborate a bit more on this? I can understand that there are common functionalities can be shared and make s

Re: Approximate query processing in Calcite

2018-05-07 Thread Michael Mior
That's I question I can raise with the VerdictDB folks. However, I doubt it makes sense for them to invest the time to make the change without a clear tangible benefit for them. They already have a working system and it's not clear what they would gain by swapping out so much of their internals for

Re: Approximate query processing in Calcite

2018-05-07 Thread Edmon Begoli
A perhaps idealistic question, but would it be possible to do a little bit of engineering and architectural swap, where we incorporate VerdictDB’s unique parsing, optimization and other components, and then have them use Calcite? Again, this is probably an easy thing to say, and much harder to do,

Re: Approximate query processing in Calcite

2018-05-07 Thread Julian Hyde
I also noticed that they have a Veeline module [1], a fork of SQLline [2] that I maintain. No complains about that — copy-paste is re-use, and re-use is good! — but if they want to contribute their changes back I’d be glad to have them. Julian [1] https://github.com/mozafari/verdictdb/tree/mas

Re: Approximate query processing in Calcite

2018-05-07 Thread Michael Mior
You are correct that there are a lot of pieces the systems could probably share. In fact, mentioning some of the other systems using Calcite's parser drew immediate interest so I think that's something they're exploring. It seems as though they may also want to exploring using Calcite's relational

Re: Approximate query processing in Calcite

2018-05-07 Thread Michael Mior
Xiening, You are correct that VerdictDB is currently completely DBMS independent. However, we are interested in exploring ways that a deeper integration might be beneficial especially in the case of federated query processing across multiple database backends. The first step would be to simply all

Re: Approximate query processing in Calcite

2018-05-07 Thread Julian Hyde
In many ways VerdictDB has a similar architecture to Calcite - a query mediation layer that understands SQL can sends modified SQL to the back-end. I think of approximate query processing as a form of materialized view rewrite. In order to answer the query you obviously have to read some data, b

Re: Approximate query processing in Calcite

2018-05-07 Thread Xiening Dai
Hi Michael, AQP is valuable in our business scenarios. Often time our data scientist would issue exploratory queries to get a basic sense of the data (means, aggregation on certain groupings, etc). But from my understanding, Verdict DB is independent to the database system and the query planner

Re: Approximate query processing in Calcite

2018-05-07 Thread Michael Mior
Edmon (and others), I'd be curious to hear more about your specific use cases if you're able to share. Especially those who have companies which may benefit from using AQP with Calcite to lower costs. -- Michael Mior mm...@uwaterloo.ca Le jeu. 3 mai 2018 à 18:58, Edmon Begoli a écrit : > I am

Re: Are all the statistics given to calcite, need to be exact or approximate?

2018-05-07 Thread Julian Hyde
Not sure what you mean by “depict”. If you want a description of the statistics, read their documentation. Hopefully it’s clear which statistics are exact and which are not. Obviously we’d all like statistics to be exact. But for many statistics, it’s impossible to get exact answer unless you a

Re: How to disable EnumerableThetaJoin?

2018-05-07 Thread Julian Hyde
It sounds like EnumerableThetaJoin is necessary for that query. I don’t believe that an equi-join would give the right answer. Because of the outer it’s difficult to push down or pull up the condition. > On May 7, 2018, at 2:42 AM, aishwaryaa...@gmail.com wrote: > > In case of queries having ou

Re: Are all the statistics given to calcite, need to be exact or approximate?

2018-05-07 Thread aishwaryaanns
Can you depict which column statistics should be exact and which are all can be approximate to get a decent plan? On 2018/05/04 05:31:30, aishwaryaa...@gmail.com wrote: > Yes ColStatistics is in Hive and it holds all statistics about the columns. > > On 2018/05/03 16:26:02, Julian Hyde wrot

How to disable EnumerableThetaJoin?

2018-05-07 Thread aishwaryaanns
In case of queries having outer joins with one condition on equality between two columns and another condition on one particular column, then in optimization, LogicalJoin has been converted to EnumerableThetaJoin instead of EnumerableJoin. How to make changes such that LogicalJoin will not be co

[jira] [Created] (CALCITE-2302) Implicit type cast support

2018-05-07 Thread Yuzhao Chen (JIRA)
Yuzhao Chen created CALCITE-2302: Summary: Implicit type cast support Key: CALCITE-2302 URL: https://issues.apache.org/jira/browse/CALCITE-2302 Project: Calcite Issue Type: Improvement