https://issues.apache.org/jira/browse/TEZ-1157
-----Original Message----- From: Johannes Zillmann [mailto:[email protected]] Sent: Friday, July 25, 2014 2:51 AM To: [email protected] Cc: Gopal V Subject: Re: Task count Ok, will try this, thanks! Can you say the jira number so i can track progress on that ? You know for which version of Tez this is planned ? And no, no more use cases currently! best Johannes On 24 Jul 2014, at 20:38, Bikas Saha <[email protected]> wrote: > The patch should work for all types of vertices because it gives each > task the total number of tasks for its vertex. Do you have any other use case? > > Bikas > > -----Original Message----- > From: Johannes Zillmann [mailto:[email protected]] > Sent: Thursday, July 24, 2014 6:58 AM > To: Gopal V > Cc: [email protected] > Subject: Re: Task count > > Hey Gopal, > > using the task count basically for 2 things (in mr for both the map > stage and the reduce stage): > - each task samples its output-data up to a certain number. This > number is the desired sample count divided by the number of tasks > - also we use the task count in some scenarios to let the last task > (of a stage or a vertex) do some extra logic. That plays in > combination of the task-index. > > Looking at your patch it looks like it will do the job for kind of the > map-like vertex but not for the aggregation vertex, right ? > Also what jira issue is that ? > > best > Johannes > > On 24 Jul 2014, at 07:40, Gopal V <[email protected]> wrote: > >> On 7/23/14, 6:07 PM, Johannes Zillmann wrote: >>> Hey Tez team, >>> >>> is there some way to get the task count within a vertex from within >>> a > task ? >>> Some equivalent to mapred.map.tasks and mapred.reduce.tasks for > map-reduce ? >> >> Could you explain the use-case for this particular requirement? >> >> I intend to add the vertex parallelism to the task context as part of > one of my WIP branches. >> >> I uploaded my base patch-set as is (including the TODO markers). >> >> > https://issues.apache.org/jira/secure/attachment/12657536/TEZ-broadcas > t-sh > uffle%2Bvertex-parallelism.patch >> >> If you can explain what you are actually looking to do with this > information, perhaps I can roll the two feature reqs together. >> >> Cheers, >> Gopal >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable > law. If the reader of this message is not the intended recipient, you > are hereby notified that any printing, copying, dissemination, > distribution, disclosure or forwarding of this communication is > strictly prohibited. If you have received this communication in error, > please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
