It looks to happen when building the final plan: the hep planner goes recursively to each node to recompute the digest. In that relnode tree, there's no more HepRelVertex nodes, and the digest now includes the whole input(s) description.
On Wed, Aug 15, 2018 at 2:33 PM Julian Hyde <[email protected]> wrote: > When I run that test I get > > LogicalProject(input=HepRelVertex#10,$0=$9) > > Have you screwed something up? > > > On Aug 15, 2018, at 2:23 PM, Laurent Goujon <[email protected]> wrote: > > > > Just ran RelOptRulesTest with a breakpoint in > > AbstractRelNode#computeDigest() and I'm able to observe those kind of > > digest: > > > "LogicalProject(input=rel#6:LogicalWindow(input=rel#0:LogicalTableScan(table=[CATALOG, > > SALES, EMP]),window#0=window(partition {0} order by [0] range between > > UNBOUNDED PRECEDING and CURRENT ROW aggs [COUNT()])),$0=$9)" > > > > On Wed, Aug 15, 2018 at 2:09 PM Laurent Goujon <[email protected]> > wrote: > > > >> Here's one (partial) example (truncated because it contains potential > >> sensitive info, and didn't obfuscate or try to reproduce locally with > non > >> sensitive data): > >> > >> > "rel#8643738:LogicalProject.NONE.ANY([]).[](input=rel#8643736:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643702:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643668:LogicalUnion.NONE.ANY([]).[](input#0=rel#8643634:LogicalProject.NONE.ANY([]).[](input=rel#8643632:LogicalAggregate.NONE.ANY([]).[](input=rel#8643630:LogicalAggregate.NONE.ANY([]).[](input=rel#8643628:LogicalProject.NONE.ANY([]).[](input=rel#8643626:LogicalFilter.NONE.ANY([]).[](input=rel#8643624:LogicalProject.NONE.ANY([]).[](input=rel#8643622:LogicalProject.NONE.ANY([]).[](input=rel#8643842:MultiJoin.NONE.ANY([]).[](input#0=rel#8643838:LogicalProject.NONE.ANY([]).[](input=rel#8643615:MultiJoin.NONE.ANY([]).[](input#0=rel#8643603:LogicalProject.NONE.ANY([]).[](input=rel#8643601:SampleCrel.NONE.ANY([]).[](input=rel#8639853:ScanCrel.NONE.ANY([]).[](table="... > >> > >> The Logical* relnodes don't override computeDigest method, so this is > >> basically whatever AbstractRelNode#computeDigest is doing: > >> > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/AbstractRelNode.java#L415 > >> > >> Laurent > >> > >> > >> > >> On Wed, Aug 15, 2018 at 1:57 PM Julian Hyde <[email protected]> wrote: > >> > >>> I thought the digest only included the IDs of the inputs, not the > digest > >>> of the inputs. Am I mistaken? > >>> > >>> Could you give an example of large description & digest? > >>> > >>>> On Aug 15, 2018, at 1:46 PM, Laurent Goujon <[email protected]> > wrote: > >>>> > >>>> Hi folks, > >>>> > >>>> I'm looking for some guidance here before opening JIRAs/pull requests. > >>>> > >>>> I'm examining a memory dump during a planning operation and a > >>> significant > >>>> amount of memory are strings used for RelNode digest and description > >>> (some > >>>> strings being around 130kb). In that particular case, the relnode tree > >>> is > >>>> particularly deep, and since the digest is basically done recursively, > >>> the > >>>> deepest/widest the tree, the longer the digest. > >>>> > >>>> The easy solution would be to not go deep when adding inputs to the > >>> digest, > >>>> and instead of adding the input description to only add their type, id > >>> and > >>>> traits (and also not recurse). Would this break parts of calcite, or > >>> cause > >>>> other inconvenience because some use-cases rely on digest/description > >>> to be > >>>> basically the whole tree in a textual form? > >>>> > >>>> Laurent > >>> > >>> > >
