saxenapranav commented on code in PR #6314:
URL: https://github.com/apache/hadoop/pull/6314#discussion_r1510666424
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -1411,6 +1448,97 @@ protected AccessTokenProvider getTokenProvider() {
return tokenProvider;
}
+ public AzureBlobFileSystem getMetricFilesystem() throws IOException {
+ if (metricFs == null) {
+ try {
+ Configuration metricConfig = abfsConfiguration.getRawConfiguration();
+ String metricAccountKey =
metricConfig.get(FS_AZURE_METRIC_ACCOUNT_KEY);
+ final String abfsMetricUrl = metricConfig.get(FS_AZURE_METRIC_URI);
+ if (abfsMetricUrl == null) {
+ return null;
+ }
+ metricConfig.set(FS_AZURE_ACCOUNT_KEY_PROPERTY_NAME, metricAccountKey);
+ metricConfig.set(AZURE_CREATE_REMOTE_FILESYSTEM_DURING_INITIALIZATION,
"false");
+ URI metricUri;
+ metricUri = new URI(FileSystemUriSchemes.ABFS_SCHEME, abfsMetricUrl,
null, null, null);
+ metricFs = (AzureBlobFileSystem) FileSystem.newInstance(metricUri,
metricConfig);
Review Comment:
Was curious on this, tried
https://github.com/saxenapranav/hadoop/commit/0cced3f0d92e9248072b0c18ec1e321d7250b81f.
Chaining would happen, reason being, there is no knowledge with the
application, if a metricFS has been created or not for a given fs. So, what
happens is: when metricFs is created, it will create an client object, which
would again create a new metricFs (at this instance 2), but this chain will
grow up infinitely.
There is a problem further: we can not control how many fileSystem objects
application want to create for same url. So, keeping 1:1 relatationship of
metricFs to uri would not be the best one. Reason being, each fs object created
by app has diff values of abfsCounter.
I would recommend again, that we should not have fileSystem created out of
client. But we just create a new method in client which understands metricUri.
This method just creates an restOperation with the metricUri and execute it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
