[
https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063081#comment-16063081
]
Steve Loughran commented on HADOOP-14475:
-----------------------------------------
# {{S3AInstrumentation}}: revert the .* import to the explicit classes.
# I think we should use the bucket & number as cues, with a format like
{{String.format("s3afilesystem-%04d-%s",
metricsSourceNameCounter.incrementAndGet, bucket }}. That's always with the
counter; 3 digits of numbering so even long-lived LLAP clusters will take a
while to lose the indexing, and with all remaning bucket name (a hostname)
coming after. That way, even when the bucket has a hyphen in it, scan logic
will work trivially.
{{s3aFilesystem-0001-stevel-ireland}}.
I've been thinking about testing this, specifically, how to insert it into an
existing test. Azure does this in {{TestNativeAzureFileSystemClientLogging}},
though note the issues there with HADOOP-14573. We can do a capture-the-logs
run & parse the output, or have things get saved to a file. Either way, having
that test will not only show things work, it'll stop people (myself included)
breaking it.
> Metrics of S3A don't print out when enable it in Hadoop metrics property file
> ------------------------------------------------------------------------------
>
> Key: HADOOP-14475
> URL: https://issues.apache.org/jira/browse/HADOOP-14475
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.8.0
> Environment: uname -a
> Linux client01 4.4.0-74-generic #95-Ubuntu SMP Wed Apr 12 09:50:34 UTC 2017
> x86_64 x86_64 x86_64 GNU/Linux
> cat /etc/issue
> Ubuntu 16.04.2 LTS \n \l
> Reporter: Yonger
> Assignee: Yonger
> Attachments: failsafe-report-s3a-it.html,
> failsafe-report-s3a-scale.html, HADOOP-14475.002.patch, s3a-metrics.patch1,
> stdout.zip
>
>
> *.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink
> #*.sink.file.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> #*.sink.influxdb.url=http:/xxxxxxxxxx
> #*.sink.influxdb.influxdb_port=8086
> #*.sink.influxdb.database=hadoop
> #*.sink.influxdb.influxdb_username=hadoop
> #*.sink.influxdb.influxdb_password=hadoop
> #*.sink.ingluxdb.cluster=c1
> *.period=10
> #namenode.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> #S3AFileSystem.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink
> S3AFileSystem.sink.file.filename=s3afilesystem-metrics.out
> I can't find the out put file even i run a MR job which should be used s3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
