[
https://issues.apache.org/jira/browse/HDFS-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525569#comment-16525569
]
Todd Lipcon commented on HDFS-13702:
------------------------------------
{quote}I do want a trace layer in there; I do want it broader than just HDFS,
and I do want it to be used from the layers above. I don't want stuff taken
from DFSClient until there's a better story there. Otherwise: it'll get cut,
nobody will replace it, and it'll get lost in folklore.
{quote}
Sure, I agree tracing is nice. I recently worked on adding OpenTracing support
to the HMS (HIVE-19685). I think we could do the same here. That said,
regardless of trace framework, I think we can all agree that the current code
path is too expensive as written today. I don't have time to do the heavy
lifting of moving entirely to some new tracing framework, so in the meantime I
think we should fix this perf issue by removing this heavy code path. I didn't
remove all the other FS-operation-level trace annotations (eg
open/close/listStatus/etc) since those are likely to be less high throughput
and therefore overhead not an issue.
> HTrace hooks taking 10-15% CPU in DFS client when disabled
> ----------------------------------------------------------
>
> Key: HDFS-13702
> URL: https://issues.apache.org/jira/browse/HDFS-13702
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: performance
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
> Attachments: hdfs-13702.patch, hdfs-13702.patch, hdfs-13702.patch
>
>
> I am seeing DFSClient.newReaderTraceScope take ~15% CPU in a teravalidate
> workload even when HTrace is disabled. This is because it stringifies several
> integers. We should avoid all allocation and stringification when htrace is
> disabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
