Sounds good. Thank you Charles. On Fri, Jan 28, 2022 at 12:30 PM Charles Givre <[email protected]> wrote:
> Hey Daniel, > I looked at the mongo docs, and I'm going to add a config option to add > the "spill to disk" for mongo. I'll do it over the weekend and submit a PR > to Drill. This is a very minor fix. > Best, > -- C > > > > > On Jan 28, 2022, at 12:28 PM, Daniel Clark <[email protected]> wrote: > > > > I've attached the json version of the profile. Is that what you are > requesting? The latest run ended in another error. > > > > On Fri, Jan 28, 2022 at 9:47 AM luoc <[email protected] <mailto: > [email protected]>> wrote: > > > > Hi Daniel, > > The 1.20 is actually included a feature to improve performance using > the new push down framework. > > For describe the specific things, could you please post the profile > file of query? > > > > > On Jan 28, 2022, at 22:32, Daniel Clark <[email protected] <mailto: > [email protected]>> wrote: > > > > > > Hello, > > > > > > While evaluating 1.20.0-SNAPSHOT release performance, I ran a mongo > query that runs in 15 minutes in the 1.19 release (below). > > > > > > SELECT `Elements_Efforts`.`EffortTypeName` AS `EffortTypeName`, > > > `Elements`.`ElementSubTypeName` AS `ElementSubTypeName`, > > > `Elements`.`ElementTypeName` AS `ElementTypeName`, > > > `Elements`.`PlanID` AS `PlanID` > > > FROM `mongo.grounds`.`Elements` `Elements` > > > INNER JOIN `mongo.grounds`.`Elements_Efforts` `Elements_Efforts` ON > (`Elements`.`_id` = `Elements_Efforts`.`_id`) > > > WHERE (`Elements`.`PlanID` = '1623263140') > > > GROUP BY `Elements_Efforts`.`EffortTypeName`, > > > `Elements`.`ElementSubTypeName`, > > > `Elements`.`ElementTypeName`, > > > `Elements`.`PlanID` > > > > > > The query runs for 34 minutes before returning this error; "Sort > exceeded memory limit of 104857600 bytes, but did not opt in to external > sorting. Aborting operation. Pass allowDiskUse:true to opt in.' on server > localhost:27017." Any ideas? I realize that it's a mongodb error, but the > mongo database doesn't raise this error with the 1.19 release. I was > expecting improved performance with the mongo storage plugin in the > upcoming 1.20 release. Nothing in my environment has changed. I've attached > the full stacktrace. > > > > > > <stacktrace.txt> > > <profile.json> > >
