Just ran RelOptRulesTest with a breakpoint in
AbstractRelNode#computeDigest() and I'm able to observe those kind of
digest:
"LogicalProject(input=rel#6:LogicalWindow(input=rel#0:LogicalTableScan(table=[CATALOG,
SALES, EMP]),window#0=window(partition {0} order by [0] range between
UNBOUNDED PRECEDING and CURRENT ROW aggs [COUNT()])),$0=$9)"On Wed, Aug 15, 2018 at 2:09 PM Laurent Goujon <[email protected]> wrote: > Here's one (partial) example (truncated because it contains potential > sensitive info, and didn't obfuscate or try to reproduce locally with non > sensitive data): > > > "rel#8643738:LogicalProject.NONE.ANY([]).[](input=rel#8643736:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643702:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643668:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643634:LogicalProject.NONE.ANY([]).[](input=rel#8643632:LogicalAggregate.NONE.ANY([]).[](input=rel#8643630:LogicalAggregate.NONE.ANY([]).[](input=rel#8643628:LogicalProject.NONE.ANY([]).[](input=rel#8643626:LogicalFilter.NONE.ANY([]).[](input=rel#8643624:LogicalProject.NONE.ANY([]).[](input=rel#8643622:LogicalProject.NONE.ANY([]).[](input=rel#8643842:MultiJoin.NONE.ANY([]).[](input#0=rel#8643838:LogicalProject.NONE.ANY([]).[](input=rel#8643615:MultiJoin.NONE.ANY([]).[](input#0=rel#8643603:LogicalProject.NONE.ANY([]).[](input=rel#8643601:SampleCrel.NONE.ANY([]).[](input=rel#8639853:ScanCrel.NONE.ANY([]).[](table="... > > The Logical* relnodes don't override computeDigest method, so this is > basically whatever AbstractRelNode#computeDigest is doing: > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/AbstractRelNode.java#L415 > > Laurent > > > > On Wed, Aug 15, 2018 at 1:57 PM Julian Hyde <[email protected]> wrote: > >> I thought the digest only included the IDs of the inputs, not the digest >> of the inputs. Am I mistaken? >> >> Could you give an example of large description & digest? >> >> > On Aug 15, 2018, at 1:46 PM, Laurent Goujon <[email protected]> wrote: >> > >> > Hi folks, >> > >> > I'm looking for some guidance here before opening JIRAs/pull requests. >> > >> > I'm examining a memory dump during a planning operation and a >> significant >> > amount of memory are strings used for RelNode digest and description >> (some >> > strings being around 130kb). In that particular case, the relnode tree >> is >> > particularly deep, and since the digest is basically done recursively, >> the >> > deepest/widest the tree, the longer the digest. >> > >> > The easy solution would be to not go deep when adding inputs to the >> digest, >> > and instead of adding the input description to only add their type, id >> and >> > traits (and also not recurse). Would this break parts of calcite, or >> cause >> > other inconvenience because some use-cases rely on digest/description >> to be >> > basically the whole tree in a textual form? >> > >> > Laurent >> >>
