Hi Chunwei, Thank you for your suggestion and insight!
-Maytas On Wed, Jul 1, 2020 at 8:16 PM Chunwei Lei <[email protected]> wrote: > Hi, Maytas. Our system is MaxCompute[1], which is a big data computing and > storage platform. We use > Calcite for cost-based query optimization. > > To reduce the search space, before entering VolcanoPlanner we use > HepPlanner to do some > optimizations, such as column pruning, predicate pushdown, and so on. > > > > [1]https://www.alibabacloud.com/product/maxcompute > > > > Best, > Chunwei > > > On Wed, Jul 1, 2020 at 1:29 PM Maytas Monsereenusorn <[email protected]> > wrote: > > > Hi Chunwei, > > > > Thank you for your reply. Can you share a little bit more about your > > particular system? > > For context, our system is Apache Druid. > > > > Best Regards, > > Maytas > > > > On Tue, Jun 30, 2020 at 7:57 PM Chunwei Lei <[email protected]> > > wrote: > > > > > Hi, Maytas. > > > > > > In my system, we set the matchLimit to 1200, which can cover most > cases. > > > > > > Best, > > > Chunwei > > > > > > > > > On Tue, Jun 30, 2020 at 4:30 PM Maytas Monsereenusorn < > > [email protected]> > > > wrote: > > > > > > > Hi, > > > > > > > > I am encountering a problem in Calcite that causes my query to result > > in > > > an > > > > StackOverflow exception. A simple query that can reproduces this > issue > > is > > > > something liken SELECT ARRAY ['Hello', NULL] > > > > > > > > I believe this is the cause (and also the fix) > > > > https://issues.apache.org/jira/browse/CALCITE-3845 > > > > However, we are still currently using Calcite 1.21.0 which does not > > have > > > > this fix (fix above is in 1.23.0). I found that a workaround that can > > > work > > > > without upgrading Calcite is to set currentProgram.matchLimit. I was > > able > > > > to confirm that the workaround (setting a limit on matchLimit) fixes > > this > > > > issue without needing to upgrade. I am wondering if there is any > > > > recommendation on a sensible value to use for the matchLimit > > > > for HepProgramBuilder? > > > > > > > > The default value is Integer.MAX_VALUE which I don't think will ever > be > > > > possible due to stack / memory limitations. > > > > > > > > - Maytas > > > > > > > > > >
