Hmm...
Each line of my file has one json document. The file is in HDFS. Though I am
not familiar with Drill's code, but looks like it reads each line as separate
record and process them. Not sure why it can't split the files with 1 million
json records. Anyway it treats each line as independent record.
> Date: Wed, 29 Oct 2014 21:54:24 -0700
> Subject: RE: Enable caching in Drill
> From: [email protected]
> To: [email protected]
>
> Drill doesn't currently cache data and relies on the underlying file system
> cache.
>
> Also, json is not splittable so adding nodes with a single json file will
> generally have little impact.
> On Oct 29, 2014 9:48 PM, "Tridib Samanta" [email protected] wrote:
>
> > select count(*) from myhdfs.json.`x00.json`;
> >
> > Surprising thing is, I get same performance when I use 1 drillbit compare
> > to 4 drillbits.
> >
> > > Date: Thu, 30 Oct 2014 10:08:04 +0530
> > > Subject: Re: Enable caching in Drill
> > > From: [email protected]
> > > To: [email protected]
> > >
> > > The query didn't get through :-).
> > >
> > >
> > > ---
> > > Mufeed Usman
> > > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
> > > Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
> > > <http://mufeed.livejournal.com>
> > >
> > >
> > >
> > >
> > > On Thu, Oct 30, 2014 at 2:54 AM, Tridib Samanta <[email protected]
> > >
> > > wrote:
> > >
> > > > Hello,
> > > > I am doing a count query like bellow. I understand that it will take
> > long
> > > > time at first attempt. But not sure why it takes same time in
> > subsequent
> > > > execution. Will I have to enable caching or something like that?
> > > >
> > > > Thanks
> > > > Tridib
> > > >
> >
