jeeteshm commented on a change in pull request #1611: HADOOP-16612 Track Azure
Blob File System client-perceived latency
URL: https://github.com/apache/hadoop/pull/1611#discussion_r341793602
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##########
@@ -265,182 +264,223 @@ public AbfsConfiguration getAbfsConfiguration() {
}
public Hashtable<String, String> getFilesystemProperties() throws
AzureBlobFileSystemException {
- LOG.debug("getFilesystemProperties for filesystem: {}",
- client.getFileSystem());
+ try (AbfsPerfInfo tracker = startTracking("getFilesystemProperties",
"getFilesystemProperties")) {
+ LOG.debug("getFilesystemProperties for filesystem: {}",
+ client.getFileSystem());
+
+ final Hashtable<String, String> parsedXmsProperties;
- final Hashtable<String, String> parsedXmsProperties;
+ final AbfsRestOperation op = client.getFilesystemProperties();
+ tracker.registerResult(op.getResult());
- final AbfsRestOperation op = client.getFilesystemProperties();
- final String xMsProperties =
op.getResult().getResponseHeader(HttpHeaderConfigurations.X_MS_PROPERTIES);
+ final String xMsProperties =
op.getResult().getResponseHeader(HttpHeaderConfigurations.X_MS_PROPERTIES);
- parsedXmsProperties = parseCommaSeparatedXmsProperties(xMsProperties);
+ parsedXmsProperties = parseCommaSeparatedXmsProperties(xMsProperties);
+ tracker.registerSuccess(true);
- return parsedXmsProperties;
+ return parsedXmsProperties;
+ }
}
public void setFilesystemProperties(final Hashtable<String, String>
properties)
throws AzureBlobFileSystemException {
- if (properties == null || properties.isEmpty()) {
- return;
- }
+ try (AbfsPerfInfo tracker = startTracking("setFilesystemProperties",
"setFilesystemProperties")) {
+ if (properties == null || properties.isEmpty()) {
+ return;
+ }
- LOG.debug("setFilesystemProperties for filesystem: {} with properties: {}",
- client.getFileSystem(),
- properties);
+ LOG.debug("setFilesystemProperties for filesystem: {} with properties:
{}",
+ client.getFileSystem(),
+ properties);
- final String commaSeparatedProperties;
- try {
- commaSeparatedProperties =
convertXmsPropertiesToCommaSeparatedString(properties);
- } catch (CharacterCodingException ex) {
- throw new InvalidAbfsRestOperationException(ex);
- }
+ final String commaSeparatedProperties;
+ try {
Review comment:
The objective here is to optimize the end-to-end latency observed by the
client. By wrapping everything inside the try-catch block, we get truer
end-ti-end latency.
So my thinking is to consider each method as a blackbox and measure the
performance of the entirety of the methods. Tracker should interpret everything
inside the method as executable code subject to optimization. This also makes
these lines consistent with other methods where we don't have many setup steps
before the real client.DoSomething() call.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
