Can you help me understand what "local to the cluster" means in the context of a 5 node cluster? In the plan, the files are all file:// Are the files replicated to each node? is it a common shared filesystem? Do all 5 nodes have equal access to the 10 files? I wonder if using a local FS in a distributed cluster is having some effect on the planning...
On Fri, Mar 3, 2017 at 6:08 AM, PROJJWAL SAHA <[email protected]> wrote: > I did not change the default values used by drill. > Are you talking of changing planner.memory_limit > and planner.memory.max_query_memory_per_node ? > If there are any other debug work that I can do, pls suggest > > Regards > > On Fri, Mar 3, 2017 at 5:14 PM, Nitin Pawar <[email protected]> > wrote: > > > how much memory have you set for planner ? > > > > On Fri, Mar 3, 2017 at 5:06 PM, PROJJWAL SAHA <[email protected]> > wrote: > > > > > Hello all, > > > > > > I am quering select * from dfs.xxx where yyy (filter condition) > > > > > > I am using dfs storage plugin that comes out of the box from drill on a > > > 1GB file, local to the drill cluster. > > > The 1GB file is split into 10 files of 100 MB each. > > > As expected I see 11 minor and 2 major fagments. > > > The drill cluster is 5 nodes cluster with 4 cores, 32 GB each. > > > > > > One observation is that the query plan time is more than 30 seconds. I > > ran > > > the explain plan query to validate this. > > > The query execution time is 2 secs. > > > total time taken is 32secs > > > > > > I wanted to understand how can i minimise the query plan time. > > Suggestions > > > ? > > > Is the time taken described above expected ? > > > Attached is result from explain plan query > > > > > > Regards, > > > Projjwal > > > > > > > > > > > > -- > > Nitin Pawar > > >
