[jira] [Created] (HADOOP-14450) ADLS Python client inconsistent when used in tandem with AdlFileSystem
Sailesh Mukil created HADOOP-14450: -- Summary: ADLS Python client inconsistent when used in tandem with AdlFileSystem Key: HADOOP-14450 URL: https://issues.apache.org/jira/browse/HADOOP-14450 Project: Hadoop Common Issue Type: Bug Components: fs/adl Reporter: Sailesh Mukil Impala uses the AdlFileSystem connector to talk to ADLS. As a part of the Impala tests, we drop tables and verify that the files belonging to that table have been dropped for all filesystems that Impala supports. These tests however, fail with ADLS. If I use the Hadoop ADLS connector to delete a file, and then list the parent directory of that file using the above Python client within the second, the client still says that the file is available in ADLS. This is the Python client from Microsoft that we're using in our testing: https://github.com/Azure/azure-data-lake-store-python Their release notes say that it's still a "pre-release preview": https://github.com/Azure/azure-data-lake-store-python/releases Questions for the ADLS folks: Is this a known issue? If so, will it be fixed soon? Or is this expected behavior? I'm able to deterministically reproduce it in my tests, with Impala on ADLS. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14437) AdlFileSystem.getAclStatus() returns unexpected ACL information
Sailesh Mukil created HADOOP-14437: -- Summary: AdlFileSystem.getAclStatus() returns unexpected ACL information Key: HADOOP-14437 URL: https://issues.apache.org/jira/browse/HADOOP-14437 Project: Hadoop Common Issue Type: Bug Components: fs/adl Reporter: Sailesh Mukil The HDFS getAclStatus() returns ACL information of a file and how that pertains to a Hadoop user/group. However, the ADLS getAclStatus() returns ACL information of a file but does not map them to Hadoop users/groups and instead maps them to the client ID of the SPI. The components built around Hadoop use this API with the expectation that the ACL information returned will be mapped to Hadoop users/groups, i.e. the components expect it to have the same behavior as other filesystems that support this API. If not, the components need to have logic to handle this case where if they're talking to ADLS, they won't look for matching the Hadoop user/group, but instead the client ID. And once AdlFileSystem changes its API to be able to map to Hadoop users/groups, it will be a breaking change for all the components. Until such functionality is able to be provided, I would suggest that the getAclStatus() be unsupported, just as it is in S3AFileSystem. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
