Re: Different output for the same date field

2018-03-14 Thread Jiang Wu
I will file a Jira issue for this problem with a more detailed description. https://issues.apache.org/jira/browse/DRILL-6242 I put some solution possibilities in the Jira. Someone with more knowledge can take a look at see if any of the solution approach makes sense first. Thanks. -- Jiang

Re: Single (embedded) drillbit as a service

2018-03-14 Thread Jiang Wu
We use an upstart (http://upstart.ubuntu.com/getting-started.html) script to launch drill-embedded as a service. It works fine on local Ubuntu machines and AWS ec2 instances. The command used inside the upstart script is similar to what you have: exec /bin/sh "a script that eventually calls

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

2018-03-14 Thread Kunal Khatua
Hi Anup Can you share this as a file ? There seems to be some truncation of the contents. Share it using some online service like Google Drive or Dropbox, since the mailing list might not allow for attachments. Thanks ~ Kunal On Tue, Mar 13, 2018 at 11:44 PM, Anup Tiwari

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

2018-03-14 Thread Kunal Khatua
Hi Anup It helps if you can share the profile (*.sys.drill / *.json files) to help explain. I don't think the user mailing list allows attachments, so you could use an online document sharing service (e.g. Google Drive, etc) to do the same. Coming back to your description, it seems like you are

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

2018-03-14 Thread Anup Tiwari
Also i have observed one thing, the query which is taking time is creating ~30-40 fragments and 99.9% of record is getting written into only one fragment. On Wed, Mar 14, 2018 1:37 PM, Anup Tiwari anup.tiw...@games24x7.com wrote: Hi Padma, Please find my highlighted answer w.r.t.

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

2018-03-14 Thread Anup Tiwari
Hi Padma, Please find my highlighted answer w.r.t. your question :- Connection loss error can happen when zookeeper thinks that a node is dead becauseit did not get heartbeat from the node. It can be because the node is busy or you havenetwork problems. Q) Did anything changed in your network ?

Re: hive connection as generic jdbc

2018-03-14 Thread Arjun kr
Looks like hive-jdbc-1.1.1-standalone.jar has 'slf4j-log4j' bundled.You may try cloning below repo for hive uber jar that tried earlier. It does not include above the jar. You can try removing 'org.apache.httpcomponents:httpclient' and 'httpcore' from artifact include list and build new jar.

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

2018-03-14 Thread Anup Tiwari
JSON Profile when Succeeded :- {"id":{"part1":2690693429455769721,"part2":6509382378722762087},"type":1,"start":1521007764471,"end":1521007906770,"query":"create table a_games_log_visit_utm as\nselect\ndistinct\nglv.sessionid,\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=',

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

2018-03-14 Thread Anup Tiwari
Hi Kunal, Please find below cluster/platform details :- Number of Nodes : 5 RAM/Node : 32GBCore/Node : 8DRILL_MAX_DIRECT_MEMORY="20G"DRILL_HEAP="8G"DRILL VERSION = 1.12.0HADOOP VERSION = 2.7.3ZOOKEEPER VERSION = 3.4.8(Installed in Distributed Mode on 3

Re: Single (embedded) drillbit as a service

2018-03-14 Thread Kunal Khatua
The trouble with your approach is that making it a background process pauses the process in an interactive mode, until you bring it to the foreground. You cannot use an Embedded Drillbit with Zookeeper. You should start it off as a regular drillbit instance. Zookeeper only serves the purpose of