Hey Daniel, I looked at the mongo docs, and I'm going to add a config option to add the "spill to disk" for mongo. I'll do it over the weekend and submit a PR to Drill. This is a very minor fix. Best, -- C
> On Jan 28, 2022, at 12:28 PM, Daniel Clark <[email protected]> wrote: > > I've attached the json version of the profile. Is that what you are > requesting? The latest run ended in another error. > > On Fri, Jan 28, 2022 at 9:47 AM luoc <[email protected] > <mailto:[email protected]>> wrote: > > Hi Daniel, > The 1.20 is actually included a feature to improve performance using the > new push down framework. > For describe the specific things, could you please post the profile file of > query? > > > On Jan 28, 2022, at 22:32, Daniel Clark <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hello, > > > > While evaluating 1.20.0-SNAPSHOT release performance, I ran a mongo query > > that runs in 15 minutes in the 1.19 release (below). > > > > SELECT `Elements_Efforts`.`EffortTypeName` AS `EffortTypeName`, > > `Elements`.`ElementSubTypeName` AS `ElementSubTypeName`, > > `Elements`.`ElementTypeName` AS `ElementTypeName`, > > `Elements`.`PlanID` AS `PlanID` > > FROM `mongo.grounds`.`Elements` `Elements` > > INNER JOIN `mongo.grounds`.`Elements_Efforts` `Elements_Efforts` ON > > (`Elements`.`_id` = `Elements_Efforts`.`_id`) > > WHERE (`Elements`.`PlanID` = '1623263140') > > GROUP BY `Elements_Efforts`.`EffortTypeName`, > > `Elements`.`ElementSubTypeName`, > > `Elements`.`ElementTypeName`, > > `Elements`.`PlanID` > > > > The query runs for 34 minutes before returning this error; "Sort exceeded > > memory limit of 104857600 bytes, but did not opt in to external sorting. > > Aborting operation. Pass allowDiskUse:true to opt in.' on server > > localhost:27017." Any ideas? I realize that it's a mongodb error, but the > > mongo database doesn't raise this error with the 1.19 release. I was > > expecting improved performance with the mongo storage plugin in the > > upcoming 1.20 release. Nothing in my environment has changed. I've attached > > the full stacktrace. > > > > <stacktrace.txt> > <profile.json>
