Thanks for the explanation Sorabh, that clears things up for me. Tim
On Thu, Aug 2, 2018 at 11:38 PM, Sorabh Hamirwasia <[email protected]> wrote: > When next is called on upstream operator then 2 things can happen: > > 1. Current operator either work on incoming and produces/copy the > records to its outgoing batch. In this case there is no transfer of > ownership of incoming batch buffer from upstream to current operator > allocator. Current operator will allocate separate memory footprint for > its > outgoing batch buffer. Also current operator is supposed to release the > incoming batch buffer once its done working on it. > 2. Current operator does a transfer of buffers from incoming batch value > vectors to outgoing value vectors (like in Filter, limit (see [1]), > etc). > In this case ownership of buffers in incoming batch is transferred to > current operator allocator. > > But I have seen different operator behaving differently. For Hash Join > since join operators has to evaluate join condition for each probe side > row, I don't think it will do any transfers. For build side it will build > hash table on column involved in join condition but also has to store other > columns if in projection list of query. So probably it might do transfer > for those columns only (haven't looked into code though). > > [1]: > https://urldefense.proofpoint.com/v2/url?u=https-3A__github. > com_apache_drill_blob_006dc10a88c1708b793e3a38ac52a0 > 266bb07deb_exec_java-2Dexec_src_main_java_org_apache_ > drill_exec_physical_impl_limit_LimitRecordBatch.java-23L181&d=DwIBaQ&c= > cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m= > 3PkPHZrX33YZhmngIGbjYtgGn_KYzXNVUyJPEUQVFyo&s=7YyEpHwWOp2I8zQjNEdFXLu- > QnVCq7neR7lyg7bfPVs&e= > > Thanks, > Sorabh > > On Thu, Aug 2, 2018 at 9:43 PM, Timothy Farkas <[email protected]> wrote: > > > Hi All, > > > > What is the expected behavior for HashJoin when it calls next for its > left > > or right upstream record batches. Is ownership of an upstream > > VectorContainer supposed to pass from from the left or right upstream > > record batches to HashJoin immediately after a call to next? Or is > > ownership of a VectorContainer supposed to stay with an upstream record > > batch immediately after a call to next? > > > > Thanks, > > Tim > > >
