[
https://issues.apache.org/jira/browse/HADOOP-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18844:
------------------------------------
Parent: HADOOP-19353 (was: HADOOP-18477)
> Add a way to get the IOStatistics of active filesystems in long-lived
> processes
> -------------------------------------------------------------------------------
>
> Key: HADOOP-18844
> URL: https://issues.apache.org/jira/browse/HADOOP-18844
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, fs/s3
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Priority: Major
>
> we can configure abfs and s3a to print out the filesystem IOStats in close(),
> but this doesn't let us see the state of long-lived processes, including
> latencies, error rates, and other important information.
> Proposed: we add a means by which all cached filesystems which provide
> IOStatistics can log them to a file/console *while the process continues to
> run*
> * a configurable scheduled thread which enumerates all open filesystem
> instances, gets their iostats and if non empty dumps to a log...log
> configuration can set that to the console or elsewhere. Good: can run without
> intervention; bad: one more thread to manage.
> * option to add a signal handler which will do this whenever a SIGUSR2 is
> raised. JVM signal handling is a bit of a brittle feature, so it should be
> off by default.
> {{org.apache.hadoop.service.launcher.IrqHandler}} supports signal wire-up.
> good: gives us an equivalent of kill -quit to get a view of iostats. bad: you
> need to be on the machine
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
