Thanks Hitesh. That explains the DAG. When you said completed vs total tasks for a given vertex, does it mean, there was a total of 0/2 + 0/8 = 0/10 ( 10 tasks ) for this tez job. Which means, when i ran the same query in hive MR, it launched 16 tasks and now it is launching only 10 tasks. Also, can you please explain how the number of tasks got reduced here ?
Thanks. On Thu, Jul 31, 2014 at 9:20 PM, Hitesh Shah <[email protected]> wrote: > Hi > > This looks like a 3-vertex DAG. It could be possibly be a linear DAG such > as Map1 -> Map2 -> Reduce3 or a Join DAG where > Map1 -> Reduce3 and Map2 -> Reduce3. > > If you can get the application logs from YARN ( using bin/yarn logs > -applicationId application_1404180111945_438880 ), you will be able to get > a .dot file from the logs which will allow you to > visualize the DAG using a tool like graphviz. > > As for the console output, 0/2 or 0/8 just implies the no. of completed vs > total tasks for a given vertex. > > thanks > — Hitesh > > > On Jul 31, 2014, at 12:04 AM, igotux igotux <[email protected]> wrote: > > > Hello Everyone, > > > > Can someone help me explain what are the numbers next to Map 1 / Map 2 > and Reducer 3 . > > > > ~~~~~~~~~~~~~~~ > > Status: Running (application id: application_1404180111945_438880) > > > > Map 1: -/- Map 2: -/- Reducer 3: 0/1 > > Map 1: 0/2 Map 2: -/- Reducer 3: 0/1 > > Map 1: 0/2 Map 2: 0/8 Reducer 3: 0/1 > > Map 1: 0/2 Map 2: 0/8 Reducer 3: 0/1 > > Map 1: 0/2 Map 2: 0/8 Reducer 3: 0/1 > > Map 1: 1/2 Map 2: 0/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 0/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 2/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 3/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 4/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 6/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 8/8 Reducer 3: 0/1 > > Map 1: 2/2 Map 2: 8/8 Reducer 3: 1/1 > > Status: Finished successfully > > OK > > ~~~~~~~~~~~~~~~ > > > > The MR hive job runs with 16 mappers and one reducer. > >
