Sailesh Mukil created HADOOP-14450:
--------------------------------------
Summary: ADLS Python client inconsistent when used in tandem with
AdlFileSystem
Key: HADOOP-14450
URL: https://issues.apache.org/jira/browse/HADOOP-14450
Project: Hadoop Common
Issue Type: Bug
Components: fs/adl
Reporter: Sailesh Mukil
Impala uses the AdlFileSystem connector to talk to ADLS. As a part of the
Impala tests, we drop tables and verify that the files belonging to that table
have been dropped for all filesystems that Impala supports. These tests
however, fail with ADLS.
If I use the Hadoop ADLS connector to delete a file, and then list the parent
directory of that file using the above Python client within the second, the
client still says that the file is available in ADLS.
This is the Python client from Microsoft that we're using in our testing:
https://github.com/Azure/azure-data-lake-store-python
Their release notes say that it's still a "pre-release preview":
https://github.com/Azure/azure-data-lake-store-python/releases
Questions for the ADLS folks:
Is this a known issue? If so, will it be fixed soon?
Or is this expected behavior?
I'm able to deterministically reproduce it in my tests, with Impala on ADLS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
