[
https://issues.apache.org/jira/browse/HADOOP-15432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468114#comment-16468114
]
Esfandiar Manii commented on HADOOP-15432:
------------------------------------------
[~mackrorysd] Thank you for your comment. The big patch is broken into 6 parts
(sub JIRAs). I divided all the code by the functionality type, i.e. the patches
won't necessary be compile-able. To help understanding the different parts I
would suggest to review in the following order: Contracts, Constants, Base
Package, Diagnostics, Services, and finally tests.
- I definitely would like to move towards that path. But at the same time we
wanted to be very careful about configurations. Our goal was to have the new
driver be "out of the box" usable for the existing customers which have
configured their clusters with the old version of driver (WASB), so there won't
be any hassles for them.
- Regarding LOG, we only use the LOG in the logging service. There were reasons
we encapsulated it into logging service, e.g. the implementation details and
the way loggers are constructed will be hidden from the callers, also they will
be much easier to replace for testing and etc.
- Regarding those libraries, I will take a look to see if I can upgrade. Our
code had been initially built on top of a local Hadoop 2.7 and I ported into
trunk.
- Regarding shading, I am fine either way. Will convert them to be more
specific.
- The auto generated code is generated by an open source tool (AutoRest) and if
you take a look over the generated code in the other JIRA, you'll notice the
amount of generated code is quite big. Updating the style for that amount code
which is subject to change anytime would not be relatively easy. But we are
very open to suggestions! :)
- IsSecure is called in AbfsHttpClientFactoryImpl.java in the services patch.
if (abfs.isSecure() {
scheme = FileSystemUriSchemes.HTTPS_SCHEME;
}
- I agree, we followed the same pattern as WASB configuration for back
compatibility reasons. Once we pass WASB deprecation phases, we will remove
extra ones.
- Thanks for letting me know about it, those tests were not implemented in
Hadoop 2.7. I will implement it them in the next change.
- If you are interested in running tests yourself, you can just directly
contact me and I will provide the access info through email.
[email protected]
> AzureBlobFS - Base package classes and configuration files
> ----------------------------------------------------------
>
> Key: HADOOP-15432
> URL: https://issues.apache.org/jira/browse/HADOOP-15432
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 3.2.0
> Reporter: Esfandiar Manii
> Assignee: Esfandiar Manii
> Priority: Major
> Attachments: HADOOP-15432-001.patch, HADOOP-15432-003.patch
>
>
> Patch contains:
> - AzureBlobFileSystem and SecureAzureBlobFileSystem classes which are the
> main interfaces Hadoop interacts with.
> - Updated Azure pom.xml with updated dependencies, updated parallel tests
> configurations and maven shader plugin.
> - Checkstyle suppression file. Since http layer is generated automatically by
> another libraries, it will not follow hadoop coding guidelines. Therefore a
> few rules for checkstyles have been disabled.
> - Added test configuration file template to be used by the consumers. Similar
> to wasb, all the configurations will go into this file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
