bq. From a priority point of view, which is more important for Pig - a
history UI for completed applications or an AM UI to monitor a running
application?
Both are important. But I guess if there is a choice to be made,
probably will go with history UI as you can at least look at it and debug
later looking at the logs and config. It would be difficult to always catch
the AM in action and difficult to debug before the run finishes.
bq. Ambrose could provide both if someone hooked it up to app history
server.
I am not sure how that would work, unless Tez support is added to
Ambrose.
Regards,
Rohini
On Thu, Jan 23, 2014 at 6:00 PM, Dmitriy Ryaboy <dvrya...@gmail.com> wrote:
> Ambrose could provide both if someone hooked it up to app history server.
>
>
> On Thu, Jan 23, 2014 at 5:11 PM, Hitesh Shah <hit...@apache.org> wrote:
>
> > At the moment, we are looking at integrating Tez with YARN's application
> > history server mostly for data-related integration. YARN will be
> providing
> > webservices that provides access to this data and therefore allow anyone
> to
> > build a UI. However, there are no concrete plans/timelines around the UI
> > implementation.
> >
> > From a priority point of view, which is more important for Pig - a
> history
> > UI for completed applications or an AM UI to monitor a running
> application?
> >
> > thanks
> > -- Hitesh
> >
> > On Jan 23, 2014, at 4:53 PM, Rohini Palaniswamy wrote:
> >
> > > Bikas came to the Pig Sprint Planning and we had some good discussions
> on
> > > the approaches we are taking.
> > >
> > > Prioritized set of requirements from the Pig Team:
> > >
> > > Q1:
> > > - Partitioned unsorted output
> > > - API to start Input fetch (TEZ-668)
> > > - Fix for any issues with 1-1 Edge found by us as we plan to use them
> > > for one stage in order-by.
> > > - UI for Tez jobs
> > >
> > > Q2:
> > > - Advanced Memory management
> > > - Memory manager for inputs based on map output sizes.
> > > - Memory manager for outputs
> > > As a workaround till then we will set io.sort.mb on the input and
> > > output descriptors ourselves manually based on number of edges.
> > >
> > > Q3/Q4:
> > > - Partial aggregator to determine number of reduces
> > > - Split edge support - same output to multiple vertices
> > >
> > >
> > > Hitesh,
> > > Bikas said you would have information on the UI for Tez as you are
> > > driving it with YARN team. Can you brief us on what we can expect and
> > > possibly when?
> > >
> > >
> > > Regards,
> > > Rohini
> >
> >
>
