Hi Boaz, Couple of updates. *Merged In:* DRILL-6542: (May be Ready2Commit soon) IndexOutOfBounds exception for multilevel lateral ((Sorabh / Parth))
*In Review:* *DRILL-6475: Query with UNNEST causes a Null Pointer . (( Hanumath ))* Thanks, Sorabh On Fri, Jul 13, 2018 at 1:17 PM, Parth Chandra <par...@apache.org> wrote: > Our (unwritten) rule has been that a commit cannot even go in unless unit > _and_ regression tests pass. > Releases are stricter, all tests, longevity tests, UI, are required to > pass. In addition, any performance regression needs to be discussed. > > So far we have not made any exceptions, but that is not to say we cannot. > > On Fri, Jul 13, 2018 at 1:03 PM, Vlad Rozov <vro...@apache.org> wrote: > > > My 2 cents: > > > > From Apache point of view it is OK to do a release even if unit tests do > > not pass at all or there is a large number of regression introduced. > Apache > > release is a source release and as long as it compiles and does not have > > license issues, it is up to community (PMC) to decide on any other > criteria > > for a release. > > > > The issue in DRILL-6453 is not limited to a large number of hash joins. > It > > should be possible to reproduce it even with a single hash join as long > as > > left and right sides are getting batches from one(many) to many exchanges > > (broadcast or hash partitioner senders). > > > > Thank you, > > > > Vlad > > > > > > On 7/13/18 08:41, Aman Sinha wrote: > > > >> I would say we have to take a measured approach to this and decide on a > >> case-by-case which issue is a show stopper. > >> While of course we have to make every effort to avoid regression, we > >> cannot > >> claim that a particular release will not cause any regression. > >> I believe there are 10000+ passing tests, so that should provide a > level > >> of confidence. The TPC-DS 72 is a 10 table join which in the hadoop > >> world > >> of > >> denormalized schemas is not relatively common. The main question is > does > >> the issue reproduce with fewer joins having the same type of > distribution > >> plan ? > >> > >> > >> Aman > >> > >> On Fri, Jul 13, 2018 at 7:36 AM Arina Yelchiyeva < > >> arina.yelchiy...@gmail.com> > >> wrote: > >> > >> We cannot release with existing regressions, especially taking into > >>> account > >>> the there are not minor issues. > >>> As far as I understand reverting is not an option since hash join spill > >>> feature are extended into several commits + subsequent fixes. > >>> I guess we need to consider postponing the release until issues are > >>> resolved. > >>> > >>> Kind regards, > >>> Arina > >>> > >>> On Fri, Jul 13, 2018 at 5:14 PM Boaz Ben-Zvi <b...@apache.org> wrote: > >>> > >>> (Guessing ...) It is possible that the root cause for DRILL-6606 is > >>>> similar to that in DRILL-6453 -- that is the new "early sniffing" in > >>>> the > >>>> Hash-Join, which repeatedly invokes next() on the two "children" of > the > >>>> join *during schema discovery* until non-empty data is returned (or > >>>> NONE, > >>>> STOP, etc). Last night Salim, Vlad and I briefly discussed > >>>> alternatives, > >>>> like postponing the "sniffing" to a later time (beginning of the build > >>>> > >>> for > >>> > >>>> the right child, and beginning of the probe for the left child). > >>>> > >>>> However this would require some work time. So what should we do about > >>>> > >>> 1.14 > >>> > >>>> ? > >>>> > >>>> Thanks, > >>>> > >>>> Boaz > >>>> > >>>> On Fri, Jul 13, 2018 at 3:46 AM, Arina Yelchiyeva < > >>>> arina.yelchiy...@gmail.com> wrote: > >>>> > >>>> During implementing late limit 0 optimization, Bohdan has found one > more > >>>>> regression after Hash Join spill to disk. > >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > apache.org_jira_browse_DRILL-2D6606&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r= > gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=TGqnVoxNweQMiHTgP4J- > rOnguFThVKShnQqHE_CmySI&s=aHUGrClgE_9UsRpRlNM95TbW91ivkqGdF1hV0EDc3xU&e= > >>>>> < > >>>>> > >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > >>> apache.org_jira_browse_DRILL-2D6606&d=DwMFaQ&c=cskdkSMqhcnjZ > >>> xdQVpwTXg&r=7lXQnf0aC8VQ0iMXwVgNHw&m=OHnyHeZpNk3hcwkG-JoQG6E > >>> 90tKdoS47J1rv5x-hJzw&s=wm5zpJf9K2zYzrqRB1LqLpKcvmBK5y6XC0ZUqVmSjko&e= > >>> > >>>> Boaz please take a look. > >>>>> > >>>>> Kind regards, > >>>>> Arina > >>>>> > >>>>> > >>>> > > >
