[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16252804#comment-16252804 ]
Yonger edited comment on HADOOP-14475 at 11/15/17 1:44 AM: ----------------------------------------------------------- For the # 2 point of my update ---------------------------------- without this update,the metrics be flush into file like this(each record with same URI mark the same "context+registry name" as begin): {code:java} 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx {code} Note, these three records written at the same time instead of different intervals, but they should be consider three different kinds of metrics(different fsid,different values in real logs). (Additional, I don't know why the metrics be registered many times within a process,like why we need to method "newMetricsSourceName") with this update, each record will be easy to distinguish: {code:java} 1510631110244 s3afs.S3aFileSystemMetrics: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3aFileSystemMetrics2: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3aFileSystemMetrics3: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx {code} was (Author: iyonger): For the # 2 point of my update ---------------------------------- without this update,the metrics be flush into file like this(each record with same URI mark the same "context+registry name" as begin): {code:java} 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3AFileSystem: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx {code} Note, these three records written at the same time instead of different intervals, so they should be consider three different kinds of metrics. (Additional, I don't know why the metrics be registered many times within a process,like why we need to method "newMetricsSourceName") with this update, each record will be easy to distinguish: {code:java} 1510631110244 s3afs.S3aFileSystemMetrics: Context=s3afs, FileSystemId=dd721794-8269-44eb-911f-a39caa2e6b34-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3aFileSystemMetrics2: Context=s3afs, FileSystemId=8fe27f60-8b3d-41c1-aa84-8bb812315d55-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx 1510631110244 s3afs.S3aFileSystemMetrics3: Context=s3afs, FileSystemId=6bcf9a67-2dcd-4533-aa5b-2ded7a8ba038-bdaas-demo-dfs, fsURI=s3a://bdaas-demo-dfs/, Hostname=client01, xxx {code} > Metrics of S3A don't print out when enable it in Hadoop metrics property file > ------------------------------------------------------------------------------ > > Key: HADOOP-14475 > URL: https://issues.apache.org/jira/browse/HADOOP-14475 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 > Affects Versions: 2.8.0 > Environment: uname -a > Linux client01 4.4.0-74-generic #95-Ubuntu SMP Wed Apr 12 09:50:34 UTC 2017 > x86_64 x86_64 x86_64 GNU/Linux > cat /etc/issue > Ubuntu 16.04.2 LTS \n \l > Reporter: Yonger > Assignee: Yonger > Attachments: HADOOP-14475-003.patch, HADOOP-14475.002.patch, > HADOOP-14475.005.patch, HADOOP-14475.006.patch, HADOOP-14475.008.patch, > HADOOP-14775.007.patch, failsafe-report-s3a-it.html, > failsafe-report-s3a-scale.html, failsafe-report-scale.html, > failsafe-report-scale.zip, s3a-metrics.patch1, stdout.zip > > > *.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink > #*.sink.file.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink > #*.sink.influxdb.url=http:/xxxxxxxxxx > #*.sink.influxdb.influxdb_port=8086 > #*.sink.influxdb.database=hadoop > #*.sink.influxdb.influxdb_username=hadoop > #*.sink.influxdb.influxdb_password=hadoop > #*.sink.ingluxdb.cluster=c1 > *.period=10 > #namenode.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink > #S3AFileSystem.sink.influxdb.class=org.apache.hadoop.metrics2.sink.influxdb.InfluxdbSink > S3AFileSystem.sink.file.filename=s3afilesystem-metrics.out > I can't find the out put file even i run a MR job which should be used s3. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org