You can search the usage of waitForCompletion in the code base, e.g.:
APIFramework.java:
public void executeJobArray(IHyracksClientConnection hcc,
JobSpecification[] specs, PrintWriter out)
throws Exception {
for (JobSpecification spec : specs) {
spec.setMaxReattempts(0);
JobId jobId = hcc.startJob(spec);
long startTime = System.currentTimeMillis();
hcc.waitForCompletion(jobId);
long endTime = System.currentTimeMillis();
double duration = (endTime - startTime) / 1000.00;
out.println("<pre>Duration: " + duration + " sec</pre>");
}
}
You start a job and get the job Id, and then you can wait on the job id.
Best,
Yingyi
On Tue, Oct 11, 2016 at 1:45 AM, 李文海 <[email protected]> wrote:
> Hi, Mingda.
> What you need is quite familiar with what I and Presten have done.
> Actually, I think we just need a shared
> object accommodated by joblet or task which should be also driven by a
> broadcast connector inbetween its input
> and output operators. We can talk about this by skype if needed.
> Best, Wenhai
>
>
> > -----原始邮件-----
> > 发件人: "Mike Carey" <[email protected]>
> > 发送时间: 2016年10月11日 星期二
> > 收件人: [email protected]
> > 抄送:
> > 主题: Re: Let one Operator finished the job before another one begin in
> Hyracks
> >
> > And both Wenhai and Preston have examples of doing the
> > fan-in-and-compute/fan-back-out pattern with blocking until the latter
> > part is done - Wenhai for finding range split points for parallel
> > sorting and Preston for similar things that arise in interval joins.
> > Can you guys chime in when you have a chance? (Preston may be busy from
> > what I saw on Skype on Friday :-), with congrats being due!)
> >
> >
> > On 10/11/16 12:22 AM, Jianfeng Jia wrote:
> > > Based on the described example, it seems possible to implement it in
> one job by using MToNPartitioningConnectorDescriptor.
> > > You can force that merge-BF-operator only runs in one partition by
> using PartitionConstraintHelper.addAbsoluteLocationConstraint() function.
> > >
> > >> On Oct 10, 2016, at 11:43 PM, mingda li <[email protected]>
> wrote:
> > >>
> > >> Yeah, that will be easier. But for example, we have N nodes and in
> each
> > >> node, it will generate a Bloom Filter(BF) for its own data. We need
> to send
> > >> these BFs to one node for constructing a complete BF and then send
> the BF
> > >> back to each node. I am not sure we can use multiple stage job for
> this,
> > >> because there should be a 1->N and a N->1 connecter among nodes. If
> in one
> > >> job, there may be no way to transfer data among nodes.
> > >> This is my idea. If this can be implemented by one multiple stage
> job, that
> > >> will decrease a lot of my work :-)
> > >>
> > >> Bests,
> > >> Mingda
> > >>
> > >> On Mon, Oct 10, 2016 at 8:59 PM, Mike Carey <[email protected]>
> wrote:
> > >>
> > >>> Is there a reason for wanting two jobs? I would think that one
> multiple
> > >>> stage job would be preferable.
> > >>>
> > >>> On Oct 10, 2016 1:21 PM, "mingda li" <[email protected]> wrote:
> > >>>
> > >>>> Oh, thanks Kim~
> > >>>>
> > >>>> On Mon, Oct 10, 2016 at 12:55 PM, Taewoo Kim <[email protected]>
> wrote:
> > >>>>
> > >>>>> Forwarded to dev.
> > >>>>>
> > >>>>> Best,
> > >>>>> Taewoo
> > >>>>>
> > >>>>> ---------- Forwarded message ----------
> > >>>>> From: mingda li <[email protected]>
> > >>>>> Date: Mon, Oct 10, 2016 at 11:21 AM
> > >>>>> Subject: Let one Operator finished the job before another one
> begin in
> > >>>>> Hyracks
> > >>>>> To: [email protected]
> > >>>>>
> > >>>>>
> > >>>>> Hi,
> > >>>>>
> > >>>>> Now,I am trying to build a Bloom Filter(BF) before join. The BF is
> > >>> build
> > >>>> in
> > >>>>> each node and sent to one node to combine. I want to set a stop
> sign
> > >>>> there
> > >>>>> before sending the BF in each node. The stop sign means it can only
> > >>> send
> > >>>>> the BF after it is build.
> > >>>>> The class HyracksConnection.waitForCompletion may help this. But
> I am
> > >>>> not
> > >>>>> sure how to use it.
> > >>>>> Should I build two jobs: hcc.waitForCompletion(jobBuildBF);
> > >>>>> jobidSendBF=hcc.startJob(); ?
> > >>>>> Has anyone ever used the HyracksConnection.waitForCompletion?
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Mingda
> > >>>>>
> >
>
>
