Hi Ramana,
We've isolated the problem! Below is our simple test case whereby a SQL
query only runs local and not distributed.
Our single zookeeper instance is running only on node Drill1 amd the
foreman drillbit is running on Drill2.
SQL -->
use dfs;
select xbin,ybin,numberOfConnections from
`/geo/appdata/rawdata/2014/SameNamedata.json`;
where SameNamedata.json is present on nodes : Drill1, Drill2 and Drill3,
and is of the form -->
{ "NumberOfConnections": 1, "StartTime": "2014-10-
02T14:30:00Z", "EndTime": "2014-10-02T14:45:00Z", "XBin": 566150, "YBin":
3270100 }
{ "NumberOfUniqueSubscribers": 2, "NumberOfConnections":
2, "StartTime": "2014-10-02T14:30:00Z", "EndTime": "2014-10-
02T14:45:00Z", "XBin": 571350, "YBin": 3268100, "AverageSnr": -
6.2870720195528573 }
On the 'foreman' node (Drill2) , drillbit.log reveals the following
trace -->
2014-10-28 10:55:06,883 [UserServer-1] DEBUG
o.a.drill.exec.rpc.user.UserServer - Received query to run. Returning
query handle.
2014-10-28 10:55:06,893 [WorkManager Event Thread] DEBUG
o.apache.drill.exec.work.WorkManager - Starting pending task
org.apache.drill.exec.work.WorkManager$RunnableWrapper@bec4752
2014-10-28 10:55:06,911 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman]
DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig - Initializing HBase
StoragePlugin configuration with zookeeper quorum 'localhost', port '2181'.
2014-10-28 10:55:06,912 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman]
DEBUG o.a.drill.exec.store.SchemaFactory - Took 17 ms to register schemas.
2014-10-28 10:55:06,953 [091c7c1d-fb6b-4685-9f1d-04f08afd1053:foreman]
DEBUG o.a.d.e.p.s.h.DefaultSqlHandler - Optiq Logical :
ProjectRel(xbin=[$1], ybin=[$2], numberOfConnections=[$3]): rowcount =
100.0, cumulative cost = {200.0 rows, 401.0 cpu, 0.0 io, 0.0 network, 0.0
memory}, id = 1259
EnumerableTableAccessRel(table=
[[dfs, /geo/appdata/rawdata/2014/SameNamedata.json]]): rowcount = 100.0,
cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory},
id = 1256
The line -->
2014-10-28 10:55:06,911 [091c7c1d-fb6b-4685-9f1d-
04f08afd1053:foreman] DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig -
Initializing HBase StoragePlugin configuration with zookeeper
quorum 'localhost', port '2181'.
may be the key.
We're not using HBase and their is no Zookeeper instance running local on
Drill2 !!
May be a bug ? (Although it might still be our configuration I suppose).
Cheers,
Chris
