Hi Daniel, I just submitted DRILL-8118 (https://github.com/apache/drill/issues/2440 <https://github.com/apache/drill/issues/2440>). Please take a look at let us know if this solves your issue. Best, -- C
> On Jan 28, 2022, at 12:35 PM, Daniel Clark <[email protected]> wrote: > > Sounds good. Thank you Charles. > > On Fri, Jan 28, 2022 at 12:30 PM Charles Givre <[email protected] > <mailto:[email protected]>> wrote: > >> Hey Daniel, >> I looked at the mongo docs, and I'm going to add a config option to add >> the "spill to disk" for mongo. I'll do it over the weekend and submit a PR >> to Drill. This is a very minor fix. >> Best, >> -- C >> >> >> >>> On Jan 28, 2022, at 12:28 PM, Daniel Clark <[email protected]> wrote: >>> >>> I've attached the json version of the profile. Is that what you are >> requesting? The latest run ended in another error. >>> >>> On Fri, Jan 28, 2022 at 9:47 AM luoc <[email protected] >>> <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>>> wrote: >>> >>> Hi Daniel, >>> The 1.20 is actually included a feature to improve performance using >> the new push down framework. >>> For describe the specific things, could you please post the profile >> file of query? >>> >>>> On Jan 28, 2022, at 22:32, Daniel Clark <[email protected] >>>> <mailto:[email protected]> <mailto: >> [email protected]>> wrote: >>>> >>>> Hello, >>>> >>>> While evaluating 1.20.0-SNAPSHOT release performance, I ran a mongo >> query that runs in 15 minutes in the 1.19 release (below). >>>> >>>> SELECT `Elements_Efforts`.`EffortTypeName` AS `EffortTypeName`, >>>> `Elements`.`ElementSubTypeName` AS `ElementSubTypeName`, >>>> `Elements`.`ElementTypeName` AS `ElementTypeName`, >>>> `Elements`.`PlanID` AS `PlanID` >>>> FROM `mongo.grounds`.`Elements` `Elements` >>>> INNER JOIN `mongo.grounds`.`Elements_Efforts` `Elements_Efforts` ON >> (`Elements`.`_id` = `Elements_Efforts`.`_id`) >>>> WHERE (`Elements`.`PlanID` = '1623263140') >>>> GROUP BY `Elements_Efforts`.`EffortTypeName`, >>>> `Elements`.`ElementSubTypeName`, >>>> `Elements`.`ElementTypeName`, >>>> `Elements`.`PlanID` >>>> >>>> The query runs for 34 minutes before returning this error; "Sort >> exceeded memory limit of 104857600 bytes, but did not opt in to external >> sorting. Aborting operation. Pass allowDiskUse:true to opt in.' on server >> localhost:27017." Any ideas? I realize that it's a mongodb error, but the >> mongo database doesn't raise this error with the 1.19 release. I was >> expecting improved performance with the mongo storage plugin in the >> upcoming 1.20 release. Nothing in my environment has changed. I've attached >> the full stacktrace. >>>> >>>> <stacktrace.txt> >>> <profile.json>
