Re: question about views

2018-03-19 Thread Ted Dunning
Aman, That is exactly the clarification that I needed. I had a hazy memory of a problem in this area, but not enough to actually figure out the current state. In case anybody cares, being able to do this is really handy. The basic idea is to keep long history in files and recent history in a DB.

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

2018-03-19 Thread Parth Chandra
Hi Anup, I don't have full context for the proposed hack, and it might have worked, but looks like Vlad has addressed the issue in the right place. Perhaps you can try out 1.13.0 and let us all know. Thanks Parth On Sat, Mar 17, 2018 at 11:43 AM, Anup Tiwari wrote:

Re: QSQL via jdbc (python3 and JayDeBeApi) wraps with SELECT ... LIMIT 0

2018-03-19 Thread Kunal Khatua
Francis I'm certain this is the result of JayDeBeApi using the  preparedStatement command.  (DRILL-5316. See the comments in the JIRA) I was thinking of creating a fork and using the standard Connection.getStatement() API instead, before compiling. However, I'm currently on a time crunch and

Re: QSQL via jdbc (python3 and JayDeBeApi) wraps with SELECT ... LIMIT 0

2018-03-19 Thread Francis McGregor-Macdonald
Thanks Kunal and Charles, I rebuilt the script / environment inside a container to see if I could replicate and I have the same result. The container is running on an EC2 "next to" the cluster. Charles was there any additional configuration you had done? I have in the Dockerfile: ... conda

Re: question about views

2018-03-19 Thread Aman Sinha
Due to an infinite loop occurring in Calcite planning, we had to disable the filter pushdown past the union (SetOps). See https://issues.apache.org/jira/browse/DRILL-3855. Now that we have rebased on Calcite 1.15.0, we should re-enable this and test and if the pushdown works then the partition

Re: question about views

2018-03-19 Thread Kunal Khatua
I think Ted's question is 2 fold, with the former being more important. 1. Can we push filters past a union.  2. Will Drill push filters down to the source. For the latter, it depends on the source.  For the former, it depends primarily on whether Calcite supports this. I haven't tried it, so I

Re: question about views

2018-03-19 Thread rahul challapalli
First I would suggest to ignore the view and try out a query which has the required filters as part of the subqueries on both sides of the union (for both the database and partitioned parquet data). The plan for such a query should have the answers to your question. If both the subqueries

question about views

2018-03-19 Thread Ted Dunning
IF I create a view that is a union of partitioned parquet files and a database that has secondary indexes, will Drill be able to properly push down query limits into both parts of the union? In particular, if I have lots of archival data and parquet partitioned by time but my query only asks for

Re: Channel closed exception

2018-03-19 Thread Kunal Khatua
The exception in your snippet is basically saying that one of the communication channels on which the client was communicating with the server to fetch sever metadata closed.  Usually, this is observed if a system is under load (e.g. many concurrent queries, etc). We'd need to trace back

How to get data from mongo database into saiku using apache drill

2018-03-19 Thread Sonu Kumawat
Hi, I am trying to get data from mongo database using apache drill in Saiku tool. Right now I am able to get tables from mongo using apache drill but columns (fields) are not coming in tables ( empty tables are coming ). Please help me out of this problem ASAP. Thank you

Re: Channel closed exception

2018-03-19 Thread Divya Gehlot
Hi , The file size is quite small in KBs only . If you could tell me few scenarios when it happens will help me debug it ? Thanks, Divya On 15 March 2018 at 15:00, Kunal Khatua wrote: > There could be multiple reasons for why the ChannelClosedException is > thrown. What

Re: [ANNOUNCE] Apache Drill release 1.13.0

2018-03-19 Thread Arina Ielchiieva
One more point, from this release Apache Drill no longer supports JDK7 and fully moved to JDK8. Kind regards Arina On Mon, Mar 19, 2018 at 5:51 AM, Abhishek Girish wrote: > Congratulations everyone, on yet another great release of Apache Drill! > On Mon, Mar 19, 2018 at