[
https://issues.apache.org/jira/browse/HADOOP-13435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16223396#comment-16223396
]
Steve Loughran commented on HADOOP-13435:
-----------------------------------------
h3. main code
I'd like to be able to have a single thread access its stats in isolation. This
will let a single threaded task execution build up the stats which I can then
include in the {{_SUCCESS}} manifest. for later aggregation
{code}
setupTask(taskContext): stats0 = fs.getThreadStats()
...do work
commitTask(taskContext): stats1 = fs.getThreadStats(); diff =
stats1.diff(stats0), log(diff), _SUCCESS.stats=diff
{code}
other: New FS keynames should go into CommonStatisticNames if we plan to use
them more broadly, referencing from the enum.
h3. tests should
* hand in some -ve read ranges
* try to read nonexistent key
* look @ using Callable & futures for testIncrementAndGet for errors to come up
(); if a child has an error, just throw the exception directly, rather than an
assertNull
* could refactor out common {{assertEquals(0, stats.get(MyStatType.FIRST))}}
tome {{assertStatEquals(expected, sts, key)}} with error text to include key
name
> Add thread local mechanism for aggregating file system storage stats
> --------------------------------------------------------------------
>
> Key: HADOOP-13435
> URL: https://issues.apache.org/jira/browse/HADOOP-13435
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HADOOP-13435.000.patch, HADOOP-13435.001.patch,
> HADOOP-13435.002.patch, HADOOP-13435.003.patch, HADOOP-13435.004.patch
>
>
> As discussed in [HADOOP-13032], this is to add thread local mechanism for
> aggregating file system storage stats. This class will also be used in
> [HADOOP-13031], which is to separate the distance-oriented rack-aware read
> bytes logic from {{FileSystemStorageStatistics}} to new
> DFSRackAwareStorageStatistics as it's DFS-specific. After this patch, the
> {{FileSystemStorageStatistics}} can live without the to-be-removed
> {{FileSystem$Statistics}} implementation.
> A unit test should also be added.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
