Yeah, that will be easier. But for example, we have N nodes and in each node, it will generate a Bloom Filter(BF) for its own data. We need to send these BFs to one node for constructing a complete BF and then send the BF back to each node. I am not sure we can use multiple stage job for this, because there should be a 1->N and a N->1 connecter among nodes. If in one job, there may be no way to transfer data among nodes. This is my idea. If this can be implemented by one multiple stage job, that will decrease a lot of my work :-)
Bests, Mingda On Mon, Oct 10, 2016 at 8:59 PM, Mike Carey <dtab...@gmail.com> wrote: > Is there a reason for wanting two jobs? I would think that one multiple > stage job would be preferable. > > On Oct 10, 2016 1:21 PM, "mingda li" <limingda1...@gmail.com> wrote: > > > Oh, thanks Kim~ > > > > On Mon, Oct 10, 2016 at 12:55 PM, Taewoo Kim <wangs...@gmail.com> wrote: > > > > > Forwarded to dev. > > > > > > Best, > > > Taewoo > > > > > > ---------- Forwarded message ---------- > > > From: mingda li <limingda1...@gmail.com> > > > Date: Mon, Oct 10, 2016 at 11:21 AM > > > Subject: Let one Operator finished the job before another one begin in > > > Hyracks > > > To: us...@asterixdb.apache.org > > > > > > > > > Hi, > > > > > > Now,I am trying to build a Bloom Filter(BF) before join. The BF is > build > > in > > > each node and sent to one node to combine. I want to set a stop sign > > there > > > before sending the BF in each node. The stop sign means it can only > send > > > the BF after it is build. > > > The class HyracksConnection.waitForCompletion may help this. But I am > > not > > > sure how to use it. > > > Should I build two jobs: hcc.waitForCompletion(jobBuildBF); > > > jobidSendBF=hcc.startJob(); ? > > > Has anyone ever used the HyracksConnection.waitForCompletion? > > > > > > Thanks, > > > Mingda > > > > > >
