[
https://issues.apache.org/jira/browse/TEZ-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290595#comment-17290595
]
Rajesh Balamohan commented on TEZ-4250:
---------------------------------------
[~maheshk114],
Issue is that, "TestVertexImpl::testCounterLimits" fails, as it tries to set
the counters in TaskImpl.
Patch for this ticket tries to reduce reading from TaskImpl counters for
non-speculative execution and it causes failure for
"TestVertexImpl::testCounterLimits".
Lines of interest:
https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java#L1515
https://github.com/apache/tez/blob/master/tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestVertexImpl.java#L7113
https://github.com/apache/tez/blob/master/tez-dag/src/test/java/org/apache/tez/dag/app/dag/impl/TestVertexImpl.java#L7126
https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java#L467
"TaskImpl::setCounters" is specifically for testing. It would be good to choose
the best attempt and increment its counters in TaskImpl::setCounters.
(i.e "selectBestAttempt().getCounters().incrAllCounters(this.counters);" )
> Optimise TaskImpl::getCounters
> -------------------------------
>
> Key: TEZ-4250
> URL: https://issues.apache.org/jira/browse/TEZ-4250
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Major
> Attachments: TEZ-4250.01.patch, TEZ-4250.patch,
> image-2020-11-17-14-05-09-294.png
>
>
> It would be good to have the temp counters only when speculation is enabled.
> Current impl causes overhead for short running queries/tasks.
>
> !image-2020-11-17-14-05-09-294.png|width=794,height=355!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
