Also, could I be added to the Slack channel? On Wed, Sep 23, 2020 at 2:57 PM Carolina Gomes <[email protected]> wrote:
> Hi Paul, > > > That would be great even if you can just copy the discussion here. Being > able to do that would greatly optimize the performance of our product. > > On Mon, Sep 21, 2020 at 5:57 PM Paul Rogers <[email protected]> wrote: > >> Hi Carolina, >> >> This issue came up recently in one of the Drill Slack channels. I wonder, >> can anyone here summarize the findings from that Slack discussion? >> >> Thanks, >> >> - Paul >> >> >> On Mon, Sep 21, 2020 at 7:35 AM Carolina Gomes <[email protected]> >> wrote: >> >> > Also if it helps, I’m using Drill 1.16 in single-node mode. >> > >> > On Mon, Sep 21, 2020 at 10:32 AM Carolina Gomes <[email protected]> >> > wrote: >> > >> > > Hi all, >> > > >> > > I have a question about push down of limit and offset clauses on >> Drill. >> > > For my use case, I’d always like for limit and offset clauses to be >> > pushed >> > > down to the data sources, which are always RDBMS databases like SQL >> > Server, >> > > Oracle etc. >> > > >> > > >> > > However, I have noticed the decision to push down seems to happen >> > > depending on the size of the limit clause, and on the number of >> columns >> > > being projected. >> > > >> > > >> > > As an example, I have a table of about 250 columns with about 50 >> million >> > > rows. If I do: >> > > >> > > >> > > select * from table limit 1000 —-> limit push down does not happen, >> query >> > > takes 30s while if I change the physical plan to push down the limit >> > > clause, it takes less than 1s. >> > > >> > > select * from table limit 1000000 —-> limit push down does happen, >> query >> > > takes roughly same time as if I queried directly on the source DB. >> > > >> > > Is there a way of easily telling Drill to always pushdown? >> > > -- >> > > [Carolina Gomes] >> > > CEO, AfterData.ai <https://www.afterdata.ai/> >> > > +1 (416) 931 4774 >> > > >> > > >> > > >> > > -- >> > [Carolina Gomes] >> > CEO, AfterData.ai <https://www.afterdata.ai/> >> > +1 (416) 931 4774 >> > >> > -- > [Carolina Gomes] > CEO, AfterData.ai <https://www.afterdata.ai/> > +1 (416) 931 4774 > > > > -- [Carolina Gomes] CEO, AfterData.ai <https://www.afterdata.ai/> +1 (416) 931 4774
