[
https://issues.apache.org/jira/browse/TEZ-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996457#comment-14996457
]
Bikas Saha commented on TEZ-2918:
---------------------------------
Attached new patch that addresses Jason's final nit.
Added member var to TA to read the conf once. Opened TEZ-2934 to consolidate
such vars into the vertex to reduce memory load by having them per TA.
Updating progress is just like updating stats or counters and should be always
done. There is no need for a config to enable or disable this. Taking actions
on that information may be optional and that is already supported.
Added some code to observe perf characteristics with the change. Its added as
an Ignored test in TestMROutput. MROutput was easy to verify since it has the
least setup complexity. Nothing interesting showed up on Yourkit. Used a
modified version of the test code to run some JMH benchmarks and they show no
noticeable difference in write operations on the kvWriter.
{noformat}
==============With change================
Benchmark Mode Cnt Score Error Units
MyBenchmark.testWriter avgt 15 ≈ 10⁻⁷ s/op
Benchmark Mode Cnt Score Error Units
MyBenchmark.testWriter thrpt 15 14514176.254 ± 3121190.866 ops/s
============Without change===============
Benchmark Mode Cnt Score Error Units
MyBenchmark.testWriter avgt 15 ≈ 10⁻⁷ s/op
Benchmark Mode Cnt Score Error Units
MyBenchmark.testWriter thrpt 15 14260616.729 ± 3354735.691 ops/s
{noformat}
Please let me know if there are further comments. Barring which I will commit
the patch based on Jason's approval.
> Make progress notifications in IOs
> ----------------------------------
>
> Key: TEZ-2918
> URL: https://issues.apache.org/jira/browse/TEZ-2918
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2918.1.patch, TEZ-2918.2.patch, TEZ-2918.3.patch,
> TEZ-2918.4.patch, jmh-microbenchmark.tar.gz
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
