[jira] [Created] (HADOOP-17636) ABFS: Remove the workaround used for Http PATCH
Bilahari T H created HADOOP-17636: - Summary: ABFS: Remove the workaround used for Http PATCH Key: HADOOP-17636 URL: https://issues.apache.org/jira/browse/HADOOP-17636 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H JDK7 does not support PATCH, so to to achieve the same as a workaround we use PUT and specify the real method in the X-Http-Method-Override header. This needs to be changed. Since we are using java 8. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17536) ABFS: Suport for customer provided encrption key
[
https://issues.apache.org/jira/browse/HADOOP-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17536:
--
Summary: ABFS: Suport for customer provided encrption key (was: Suport for
customer provided encrption key)
> ABFS: Suport for customer provided encrption key
>
>
> Key: HADOOP-17536
> URL: https://issues.apache.org/jira/browse/HADOOP-17536
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.4.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> The data for a particular customer needs to be encrypted on account level. At
> server side the APIs will start accepting the encryption key as part of
> request headers. The data will be encrypted/decrypted with the given key at
> the server.
> Since the ABFS FileSystem APIs are implementations for Hadoop FileSystem APIs
> there is no direct way with which customer can pass the key to ABFS driver.
> In this case driver should have the following capabilities so that it can
> accept and pass the encryption key as one of the request headers.
> # There should be a way to configure the encryption key for different
> accounts.
> # If there is a key specified for a particular account, the same needs to be
> sent along with the request headers.
> *Config changes*
> They key for an account can be specified in the core-site as follows.
> fs.azure.account.client-provided-encryption-key.{account
> name}.dfs.core.windows.net
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17531) DistCp: Reduce memory usage on copying huge directories
[ https://issues.apache.org/jira/browse/HADOOP-17531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17317594#comment-17317594 ] Bilahari T H commented on HADOOP-17531: --- Hi [~ayushtkn], [~ste...@apache.org] The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out from ABFS contract tests. Requesting you to please take a look. https://issues.apache.org/jira/browse/HADOOP-17628 > DistCp: Reduce memory usage on copying huge directories > --- > > Key: HADOOP-17531 > URL: https://issues.apache.org/jira/browse/HADOOP-17531 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Labels: pull-request-available > Fix For: 3.3.1, 3.4.0 > > Attachments: MoveToStackIterator.patch, gc-NewD-512M-3.8ML.log > > Time Spent: 10h 20m > Remaining Estimate: 0h > > Presently distCp, uses the producer-consumer kind of setup while building the > listing, the input queue and output queue are both unbounded, thus the > listStatus grows quite huge. > Rel Code Part : > https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java#L635 > This goes on bredth-first traversal kind of stuff(uses queue instead of > earlier stack), so if you have files at lower depth, it will like open up the > entire tree and the start processing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17628) ABFS: Distcp contract test testDistCpWithIterator is timing out consistently
[ https://issues.apache.org/jira/browse/HADOOP-17628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17628: -- Description: The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out. was:The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out. > ABFS: Distcp contract test testDistCpWithIterator is timing out consistently > - > > Key: HADOOP-17628 > URL: https://issues.apache.org/jira/browse/HADOOP-17628 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > > The test case testDistCpWithIterator in AbstractContractDistCpTest is > consistently timing out. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17628) ABFS: Distcp contract test testDistCpWithIterator is timing out consistently
Bilahari T H created HADOOP-17628: - Summary: ABFS: Distcp contract test testDistCpWithIterator is timing out consistently Key: HADOOP-17628 URL: https://issues.apache.org/jira/browse/HADOOP-17628 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 3.4.0 Reporter: Bilahari T H The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17615) ADLFS: Update SDK version from 2.3.6 to 2.3.9
[ https://issues.apache.org/jira/browse/HADOOP-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17615: -- Status: Patch Available (was: Open) > ADLFS: Update SDK version from 2.3.6 to 2.3.9 > - > > Key: HADOOP-17615 > URL: https://issues.apache.org/jira/browse/HADOOP-17615 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Minor > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17615) ADLFS: Update SDK version from 2.3.6 to 2.3.9
Bilahari T H created HADOOP-17615: - Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9 Key: HADOOP-17615 URL: https://issues.apache.org/jira/browse/HADOOP-17615 Project: Hadoop Common Issue Type: Improvement Components: fs/adl Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.3.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17536) Suport for customer provided encrption key
[
https://issues.apache.org/jira/browse/HADOOP-17536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H reassigned HADOOP-17536:
-
Assignee: Bilahari T H
> Suport for customer provided encrption key
> --
>
> Key: HADOOP-17536
> URL: https://issues.apache.org/jira/browse/HADOOP-17536
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.4.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
>
> The data for a particular customer needs to be encrypted on account level. At
> server side the APIs will start accepting the encryption key as part of
> request headers. The data will be encrypted/decrypted with the given key at
> the server.
> Since the ABFS FileSystem APIs are implementations for Hadoop FileSystem APIs
> there is no direct way with which customer can pass the key to ABFS driver.
> In this case driver should have the following capabilities so that it can
> accept and pass the encryption key as one of the request headers.
> # There should be a way to configure the encryption key for different
> accounts.
> # If there is a key specified for a particular account, the same needs to be
> sent along with the request headers.
> *Config changes*
> They key for an account can be specified in the core-site as follows.
> fs.azure.account.client-provided-encryption-key.{account
> name}.dfs.core.windows.net
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17536) Suport for customer provided encrption key
Bilahari T H created HADOOP-17536:
-
Summary: Suport for customer provided encrption key
Key: HADOOP-17536
URL: https://issues.apache.org/jira/browse/HADOOP-17536
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/azure
Affects Versions: 3.4.0
Reporter: Bilahari T H
The data for a particular customer needs to be encrypted on account level. At
server side the APIs will start accepting the encryption key as part of request
headers. The data will be encrypted/decrypted with the given key at the server.
Since the ABFS FileSystem APIs are implementations for Hadoop FileSystem APIs
there is no direct way with which customer can pass the key to ABFS driver. In
this case driver should have the following capabilities so that it can accept
and pass the encryption key as one of the request headers.
# There should be a way to configure the encryption key for different accounts.
# If there is a key specified for a particular account, the same needs to be
sent along with the request headers.
*Config changes*
They key for an account can be specified in the core-site as follows.
fs.azure.account.client-provided-encryption-key.{account
name}.dfs.core.windows.net
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17475) Implement listStatusIterator
[ https://issues.apache.org/jira/browse/HADOOP-17475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17277251#comment-17277251 ] Bilahari T H commented on HADOOP-17475: --- TODO: https://issues.apache.org/jira/browse/HADOOP-17512 > Implement listStatusIterator > > > Key: HADOOP-17475 > URL: https://issues.apache.org/jira/browse/HADOOP-17475 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Major > Labels: pull-request-available > Time Spent: 5.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
[ https://issues.apache.org/jira/browse/HADOOP-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17512: -- Parent: HADOOP-17474 Issue Type: Sub-task (was: Bug) > Remove the enable/disable flag for ABFSRemoteListIterator > - > > Key: HADOOP-17512 > URL: https://issues.apache.org/jira/browse/HADOOP-17512 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
[ https://issues.apache.org/jira/browse/HADOOP-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17512: -- Parent: HADOOP-15763 Issue Type: Sub-task (was: Improvement) > Remove the enable/disable flag for ABFSRemoteListIterator > - > > Key: HADOOP-17512 > URL: https://issues.apache.org/jira/browse/HADOOP-17512 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
[ https://issues.apache.org/jira/browse/HADOOP-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17512: -- Parent: (was: HADOOP-15763) Issue Type: Bug (was: Sub-task) > Remove the enable/disable flag for ABFSRemoteListIterator > - > > Key: HADOOP-17512 > URL: https://issues.apache.org/jira/browse/HADOOP-17512 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
Bilahari T H created HADOOP-17512: - Summary: Remove the enable/disable flag for ABFSRemoteListIterator Key: HADOOP-17512 URL: https://issues.apache.org/jira/browse/HADOOP-17512 Project: Hadoop Common Issue Type: Improvement Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17503) ABFS: Fix auth failures getting counted as throttling errors in ABFS throttling interceptor
[ https://issues.apache.org/jira/browse/HADOOP-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17503: -- Summary: ABFS: Fix auth failures getting counted as throttling errors in ABFS throttling interceptor (was: ABFS: Fix auth failures gettng counted as throttling errors in ABFS throttling interceptor) > ABFS: Fix auth failures getting counted as throttling errors in ABFS > throttling interceptor > --- > > Key: HADOOP-17503 > URL: https://issues.apache.org/jira/browse/HADOOP-17503 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > > Currently the throttling intercepter counts the auth failures against > throttling errors. This needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17503) ABFS: Fix auth failures gettng counted as throttling errors
Bilahari T H created HADOOP-17503: - Summary: ABFS: Fix auth failures gettng counted as throttling errors Key: HADOOP-17503 URL: https://issues.apache.org/jira/browse/HADOOP-17503 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Currently the throttling intercepter counts the auth failures against throttling errors. This needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17503) ABFS: Fix auth failures gettng counted as throttling errors in ABFS throttling interceptor
[ https://issues.apache.org/jira/browse/HADOOP-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17503: -- Summary: ABFS: Fix auth failures gettng counted as throttling errors in ABFS throttling interceptor (was: ABFS: Fix auth failures gettng counted as throttling errors) > ABFS: Fix auth failures gettng counted as throttling errors in ABFS > throttling interceptor > -- > > Key: HADOOP-17503 > URL: https://issues.apache.org/jira/browse/HADOOP-17503 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > > Currently the throttling intercepter counts the auth failures against > throttling errors. This needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17502) IOStatistics collection for listStatusIterator()
Bilahari T H created HADOOP-17502: - Summary: IOStatistics collection for listStatusIterator() Key: HADOOP-17502 URL: https://issues.apache.org/jira/browse/HADOOP-17502 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Add IOStatistics collection for listStatusIterator -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17475) Implement listStatusIterator
Bilahari T H created HADOOP-17475: - Summary: Implement listStatusIterator Key: HADOOP-17475 URL: https://issues.apache.org/jira/browse/HADOOP-17475 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17473) Optimise abfs incremental listings
[ https://issues.apache.org/jira/browse/HADOOP-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17473: -- Description: [Uber JIRA|https://issues.apache.org/jira/browse/HADOOP-17474] > Optimise abfs incremental listings > -- > > Key: HADOOP-17473 > URL: https://issues.apache.org/jira/browse/HADOOP-17473 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > > [Uber JIRA|https://issues.apache.org/jira/browse/HADOOP-17474] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17474) Optimise abfs incremental listings
Bilahari T H created HADOOP-17474: - Summary: Optimise abfs incremental listings Key: HADOOP-17474 URL: https://issues.apache.org/jira/browse/HADOOP-17474 Project: Hadoop Common Issue Type: New Feature Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17473) Optimise abfs incremental listings
Bilahari T H created HADOOP-17473: - Summary: Optimise abfs incremental listings Key: HADOOP-17473 URL: https://issues.apache.org/jira/browse/HADOOP-17473 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17466) ADL Gen1: Fix the NoClassDefFound error for assertJ.
Bilahari T H created HADOOP-17466: - Summary: ADL Gen1: Fix the NoClassDefFound error for assertJ. Key: HADOOP-17466 URL: https://issues.apache.org/jira/browse/HADOOP-17466 Project: Hadoop Common Issue Type: Test Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17459) ADL Gen1: Fix the test case failures which are failing after the contract test update in hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-17459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17459: -- Status: Patch Available (was: Open) > ADL Gen1: Fix the test case failures which are failing after the contract > test update in hadoop-common > -- > > Key: HADOOP-17459 > URL: https://issues.apache.org/jira/browse/HADOOP-17459 > Project: Hadoop Common > Issue Type: Test > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Fix the following test case failures which are failing after the contract > test update in hadoop-common > [ERROR] Failures: > [ERROR] > TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 > expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected > with exception, but got false > [ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting > org.apache.hadoop.security.AccessControlException with text Parent path is > not a folder. but got : "void" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17459) ADL Gen1: Fix the test case failures which are failing after the contract test update in hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-17459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17459: -- Summary: ADL Gen1: Fix the test case failures which are failing after the contract test update in hadoop-common (was: Fix the test case failures which are failing after the contract test update in hadoop-common) > ADL Gen1: Fix the test case failures which are failing after the contract > test update in hadoop-common > -- > > Key: HADOOP-17459 > URL: https://issues.apache.org/jira/browse/HADOOP-17459 > Project: Hadoop Common > Issue Type: Test > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Fix the following test case failures which are failing after the contract > test update in hadoop-common > [ERROR] Failures: > [ERROR] > TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 > expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected > with exception, but got false > [ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting > org.apache.hadoop.security.AccessControlException with text Parent path is > not a folder. but got : "void" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17459) Fix the test case failures which are failing after the contract test update in hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-17459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17459: -- Summary: Fix the test case failures which are failing after the contract test update in hadoop-common (was: Fix the test case failures after the contract test update in hadoop-common) > Fix the test case failures which are failing after the contract test update > in hadoop-common > > > Key: HADOOP-17459 > URL: https://issues.apache.org/jira/browse/HADOOP-17459 > Project: Hadoop Common > Issue Type: Test > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > > Fix the following test case failures which are failing after the contract > test update in hadoop-common > [ERROR] Failures: > [ERROR] > TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 > expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected > with exception, but got false > [ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting > org.apache.hadoop.security.AccessControlException with text Parent path is > not a folder. but got : "void" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9
[ https://issues.apache.org/jira/browse/HADOOP-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17444: -- Fix Version/s: (was: 3.4.0) > ADLFS: Update SDK version from 2.3.6 to 2.3.9 > - > > Key: HADOOP-17444 > URL: https://issues.apache.org/jira/browse/HADOOP-17444 > Project: Hadoop Common > Issue Type: Bug > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Update SDK version from 2.3.6 to 2.3.9 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17459) Fix the test case failures after the contract test update in hadoop-common
Bilahari T H created HADOOP-17459: - Summary: Fix the test case failures after the contract test update in hadoop-common Key: HADOOP-17459 URL: https://issues.apache.org/jira/browse/HADOOP-17459 Project: Hadoop Common Issue Type: Test Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H Fix For: 3.4.0 Fix the following test case failures which are failing after the contract test update in hadoop-common [ERROR] Failures: [ERROR] TestAdlContractRenameLive>AbstractContractRenameTest.testRenameFileOverExistingFile:131->Assert.fail:88 expected rename(/test/source-256.txt, /test/dest-512.txt) to be rejected with exception, but got false [ERROR] TestAdlContractRenameLive.testRenameFileUnderFile:46 Expecting org.apache.hadoop.security.AccessControlException with text Parent path is not a folder. but got : "void" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9
[ https://issues.apache.org/jira/browse/HADOOP-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17444: -- Status: Patch Available (was: Open) > ADLFS: Update SDK version from 2.3.6 to 2.3.9 > - > > Key: HADOOP-17444 > URL: https://issues.apache.org/jira/browse/HADOOP-17444 > Project: Hadoop Common > Issue Type: Bug > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > > Update SDK version from 2.3.6 to 2.3.9 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9
[ https://issues.apache.org/jira/browse/HADOOP-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17444: - Assignee: Bilahari T H > ADLFS: Update SDK version from 2.3.6 to 2.3.9 > - > > Key: HADOOP-17444 > URL: https://issues.apache.org/jira/browse/HADOOP-17444 > Project: Hadoop Common > Issue Type: Bug > Components: fs/adl >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > > Update SDK version from 2.3.6 to 2.3.9 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17444) ADLFS: Update SDK version from 2.3.6 to 2.3.9
Bilahari T H created HADOOP-17444: - Summary: ADLFS: Update SDK version from 2.3.6 to 2.3.9 Key: HADOOP-17444 URL: https://issues.apache.org/jira/browse/HADOOP-17444 Project: Hadoop Common Issue Type: Bug Components: fs/adl Affects Versions: 3.4.0 Reporter: Bilahari T H Fix For: 3.4.0 Update SDK version from 2.3.6 to 2.3.9 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17347) ABFS: Read optimizations
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17347: -- Description: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last block if the read is for footer If the read is for the last 8 bytes, read the full file. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] Both these optimizations will be present under configs as follows # fs.azure.read.smallfilescompletely # fs.azure.read.optimizefooterread was: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last block if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] Both these optimizations will be present under configs as follows # fs.azure.read.smallfilescompletely # fs.azure.read.optimizefooterread > ABFS: Read optimizations > > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Optimize read performance for the following scenarios > # Read small files completely > Files that are of size smaller than the read buffer size can be considered > as small files. In case of such files it would be better to read the full > file into the AbfsInputStream buffer. > # Read last block if the read is for footer > If the read is for the last 8 bytes, read the full file. > This will optimize reads for parquet files. [Parquet file > format|https://www.ellicium.com/parquet-file-format-structure/] > Both these optimizations will be present under configs as follows > # fs.azure.read.smallfilescompletely > # fs.azure.read.optimizefooterread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17347) ABFS: Read optimizations
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17347: -- Description: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last block if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] Both these optimizations will be present under configs as follows # fs.azure.read.smallfilescompletely # fs.azure.read.optimizefooterread was: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last blok if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] Both these optimizations will be present under configs as follows # fs.azure.read.smallfilescompletely # fs.azure.read.optimizefooterread > ABFS: Read optimizations > > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Optimize read performance for the following scenarios > # Read small files completely > Files that are of size smaller than the read buffer size can be considered > as small files. In case of such files it would be better to read the full > file into the AbfsInputStream buffer. > # Read last block if the read is for footer > If the read is for the last 8 bytes, read the full file completely. > This will optimize reads for parquet files. [Parquet file > format|https://www.ellicium.com/parquet-file-format-structure/] > Both these optimizations will be present under configs as follows > # fs.azure.read.smallfilescompletely > # fs.azure.read.optimizefooterread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17347) ABFS: Read optimizations
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17347: -- Description: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last blok if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] Both these optimizations will be present under configs as follows # fs.azure.read.smallfilescompletely # fs.azure.read.optimizefooterread was: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last blok if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] > ABFS: Read optimizations > > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Optimize read performance for the following scenarios > # Read small files completely > Files that are of size smaller than the read buffer size can be considered > as small files. In case of such files it would be better to read the full > file into the AbfsInputStream buffer. > # Read last blok if the read is for footer > If the read is for the last 8 bytes, read the full file completely. > This will optimize reads for parquet files. [Parquet file > format|https://www.ellicium.com/parquet-file-format-structure/] > Both these optimizations will be present under configs as follows > # fs.azure.read.smallfilescompletely > # fs.azure.read.optimizefooterread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17347) ABFS: Read optimizations
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17347: -- Description: Optimize read performance for the following scenarios # Read small files completely Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. # Read last blok if the read is for footer If the read is for the last 8 bytes, read the full file completely. This will optimize reads for parquet files. [Parquet file format|https://www.ellicium.com/parquet-file-format-structure/] was:Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. > ABFS: Read optimizations > > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Optimize read performance for the following scenarios > # Read small files completely > Files that are of size smaller than the read buffer size can be considered > as small files. In case of such files it would be better to read the full > file into the AbfsInputStream buffer. > # Read last blok if the read is for footer > If the read is for the last 8 bytes, read the full file completely. > This will optimize reads for parquet files. [Parquet file > format|https://www.ellicium.com/parquet-file-format-structure/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17347) ABFS: Read optimizations
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17347: -- Summary: ABFS: Read optimizations (was: ABFS: Read small files completely) > ABFS: Read optimizations > > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Files that are of size smaller than the read buffer size can be considered as > small files. In case of such files it would be better to read the full file > into the AbfsInputStream buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17347) ABFS: Read small files completely
[ https://issues.apache.org/jira/browse/HADOOP-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17347: - Assignee: Bilahari T H > ABFS: Read small files completely > - > > Key: HADOOP-17347 > URL: https://issues.apache.org/jira/browse/HADOOP-17347 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > Files that are of size smaller than the read buffer size can be considered as > small files. In case of such files it would be better to read the full file > into the AbfsInputStream buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17347) ABFS: Read small files completely
Bilahari T H created HADOOP-17347: - Summary: ABFS: Read small files completely Key: HADOOP-17347 URL: https://issues.apache.org/jira/browse/HADOOP-17347 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H Files that are of size smaller than the read buffer size can be considered as small files. In case of such files it would be better to read the full file into the AbfsInputStream buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17311) ABFS: Logs should redact SAS signature
[ https://issues.apache.org/jira/browse/HADOOP-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17311: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Logs should redact SAS signature > -- > > Key: HADOOP-17311 > URL: https://issues.apache.org/jira/browse/HADOOP-17311 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, security >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Signature part of the SAS should be redacted for security purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17311) ABFS: Logs should redact SAS signature
[ https://issues.apache.org/jira/browse/HADOOP-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17311: -- Status: Patch Available (was: Open) > ABFS: Logs should redact SAS signature > -- > > Key: HADOOP-17311 > URL: https://issues.apache.org/jira/browse/HADOOP-17311 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, security >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Signature part of the SAS should be redacted for security purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17289) ABFS: Testcase failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
[ https://issues.apache.org/jira/browse/HADOOP-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17289: -- Status: Patch Available (was: Open) > ABFS: Testcase failure > ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics > > > Key: HADOOP-17289 > URL: https://issues.apache.org/jira/browse/HADOOP-17289 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The test case is failing when the fs.azure.test.appendblob.enabled=true. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17289) ABFS: Testcase failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
[ https://issues.apache.org/jira/browse/HADOOP-17289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17289: -- Summary: ABFS: Testcase failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics (was: ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics) > ABFS: Testcase failure > ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics > > > Key: HADOOP-17289 > URL: https://issues.apache.org/jira/browse/HADOOP-17289 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Bilahari T H >Priority: Minor > > The test case is failing when the fs.azure.test.appendblob.enabled=true. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17289) ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
Bilahari T H created HADOOP-17289: - Summary: ABFS: Fixing the testcase ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Key: HADOOP-17289 URL: https://issues.apache.org/jira/browse/HADOOP-17289 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H The test case is failing when the fs.azure.test.appendblob.enabled=true. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17183: -- Description: Enable check access on ABFS > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Enable check access on ABFS -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17183: -- Description: Enable check access on ABFS. Currently by default the same if disabled. (was: Enable check access on ABFS) > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Enable check access on ABFS. Currently by default the same if disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17183: -- Fix Version/s: (was: 3.4.0) > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work started] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17191 started by Bilahari T H. - > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Status: Patch Available (was: In Progress) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations ex: HNS account with SharedKey and OAuth, NonHNS account with SharedKey etc.. The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations automatically. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations ex: HNS account with SharedKey and OAuth, NonHNS account with SharedKey etc.. The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations. > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations ex: HNS account with SharedKey and OAuth, NonHNS account with SharedKey etc.. The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations. > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reopened HADOOP-17160: --- The issue is still present. > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. This will help the developer to run the integration tests with different variants of configurations. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. was: colored textADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The expectation is to automate these runs with different combinations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Summary: ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results (was: ABFS: Run tests with all the auth types) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > colored textADLS Gen 2 supports accounts with and without hierarchical > namespace support. ABFS driver supports various authorization mechanisms like > OAuth, haredKey, Shared Access Signature. The integration tests need to be > executed against accounts with and without hierarchical namespace support > using various authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The expectation is to automate these runs with different combinations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run tests with all the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The intention is to automate these runs with different combinations. > ABFS: Run tests with all the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The expectation is to automate these runs with different combinations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run tests with all the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The intention is to automate these runs with different combinations. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. > ABFS: Run tests with all the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The intention is to automate these runs with different combinations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run tests with all the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: colored textADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. was: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. The expectation is to automate these runs with different combinations. > ABFS: Run tests with all the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > colored textADLS Gen 2 supports accounts with and without hierarchical > namespace support. ABFS driver supports various authorization mechanisms like > OAuth, haredKey, Shared Access Signature. The integration tests need to be > executed against accounts with and without hierarchical namespace support > using various authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. > The expectation is to automate these runs with different combinations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run tests with all the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Description: ADLS Gen 2 supports accounts with and without hierarchical namespace support. ABFS driver supports various authorization mechanisms like OAuth, haredKey, Shared Access Signature. The integration tests need to be executed against accounts with and without hierarchical namespace support using various authorization mechanisms. Currently the developer has to manually run the tests with different combinations of configurations. > ABFS: Run tests with all the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17158) Intermittent test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters
[
https://issues.apache.org/jira/browse/HADOOP-17158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183550#comment-17183550
]
Bilahari T H commented on HADOOP-17158:
---
I will try that. But shouldn't we expect an outcome in 15 minutes(90
milliseconds)?
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters
> --
>
> Key: HADOOP-17158
> URL: https://issues.apache.org/jira/browse/HADOOP-17158
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.3.0
>Reporter: Mehakmeet Singh
>Assignee: Mehakmeet Singh
>Priority: Major
>
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters happening due to race
> conditions in readAhead threads.
> Test error:
> {code:java}
> [ERROR]
> testReadAheadCounters(org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics)
> Time elapsed: 30.723 s <<<
> ERROR!org.junit.runners.model.TestTimedOutException: test timed out after
> 3 milliseconds at java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics.testReadAheadCounters(ITestAbfsInputStreamStatistics.java:346)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748) {code}
> Possible Reasoning:
> - ReadAhead queue doesn't get completed and hence the counter values are not
> satisfied in 30 seconds time for some systems.
> - The condition that readAheadBytesRead and remoteBytesRead counter values
> need to be greater than or equal to 4KB and 32KB respectively doesn't occur
> in some machines due to the fact that sometimes instead of reading for
> readAhead Buffer, remote reads are performed due to Threads still being in
> the readAhead queue to fill that buffer. Thus resulting in either of the 2
> counter values to be not satisfying the condition and getting in an infinite
> loop and hence timing out the test eventually.
> Possible Fixes:
> - Write better test(That would pass under all conditions).
> - Maybe UT instead of IT?
> Possible fix to better the test would be preferable and UT as the last resort.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16915) ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance
[ https://issues.apache.org/jira/browse/HADOOP-16915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-16915: -- Description: Ref: https://issues.apache.org/jira/browse/HADOOP-16890 The following test fails randomly. This test compares the perf between Non HNS account against WASB. ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance was: Ref: https://issues.apache.org/jira/browse/HADOOP-16890 > ABFS: Test failure > ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance > --- > > Key: HADOOP-16915 > URL: https://issues.apache.org/jira/browse/HADOOP-16915 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: abfsactive > > Ref: https://issues.apache.org/jira/browse/HADOOP-16890 > The following test fails randomly. This test compares the perf between Non > HNS account against WASB. > ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16915) ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance
[ https://issues.apache.org/jira/browse/HADOOP-16915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-16915: -- Description: Ref: https://issues.apache.org/jira/browse/HADOOP-16890 was:Ref: https://issues.apache.org/jira/browse/HADOOP-16890 > ABFS: Test failure > ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance > --- > > Key: HADOOP-16915 > URL: https://issues.apache.org/jira/browse/HADOOP-16915 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: abfsactive > > Ref: https://issues.apache.org/jira/browse/HADOOP-16890 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16915) ABFS: Test failure ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance
[ https://issues.apache.org/jira/browse/HADOOP-16915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180511#comment-17180511 ] Bilahari T H commented on HADOOP-16915: --- ABFS accounts are primarily for customers to use with HNS property enabled. A non-HNS enabled ABFS account will not provide much value add in comparison to WASB account. The test case ITestAzureBlobFileSystemRandomRead#testRandomReadPerformance that tries to compare non-HNS to WASB performance hence isn't significant and can deviate in the perf in minor ways over time. Non-HNS not being a primary use case for ABFS, so ignoring this test which currently fails randomly as it crosses test perf threshold. > ABFS: Test failure > ITestAzureBlobFileSystemRandomRead.testRandomReadPerformance > --- > > Key: HADOOP-16915 > URL: https://issues.apache.org/jira/browse/HADOOP-16915 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Labels: abfsactive > > Ref: https://issues.apache.org/jira/browse/HADOOP-16890 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17158) Intermittent test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters
[
https://issues.apache.org/jira/browse/HADOOP-17158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180508#comment-17180508
]
Bilahari T H commented on HADOOP-17158:
---
Hi [~mehakmeetSingh], [~ste...@apache.org], I am still seeing this one.
I am testing against cental india region.
Regarding my test setup, I tested with both sharedkey as well as OAuth. And the
same is observed in both the scenarios.
I tried running the test individually from my IDE after removing the timeout
and sleep. PFB the stacktrace.
org.junit.runners.model.TestTimedOutException: test timed out after 90
milliseconds
at
app//org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics.testReadAheadCounters(ITestAbfsInputStreamStatistics.java:344)
at
java.base@11.0.8/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base@11.0.8/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base@11.0.8/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@11.0.8/java.lang.reflect.Method.invoke(Method.java:566)
at
app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
app//org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
app//org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at app//org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at
java.base@11.0.8/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base@11.0.8/java.lang.Thread.run(Thread.java:834)
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters
> --
>
> Key: HADOOP-17158
> URL: https://issues.apache.org/jira/browse/HADOOP-17158
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.3.0
>Reporter: Mehakmeet Singh
>Assignee: Mehakmeet Singh
>Priority: Major
>
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters happening due to race
> conditions in readAhead threads.
> Test error:
> {code:java}
> [ERROR]
> testReadAheadCounters(org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics)
> Time elapsed: 30.723 s <<<
> ERROR!org.junit.runners.model.TestTimedOutException: test timed out after
> 3 milliseconds at java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics.testReadAheadCounters(ITestAbfsInputStreamStatistics.java:346)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748) {code}
> Possible Reasoning:
> - ReadAhead queue doesn't get completed and hence the counter values are not
> satisfied in 30 seconds time for some systems.
> - The condition that readAheadBytesRead and remoteBytesRead counter values
> need to be greater than or equal to 4KB and 32KB respectively doesn't occur
> in some machines due to the fact that sometimes instead of reading for
> readAhead Buffer, remote reads are performed due to Threads still being in
> the readAhead queue to fill that buffer. Thus resulting in
[jira] [Created] (HADOOP-17213) ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics
Bilahari T H created HADOOP-17213: - Summary: ABFS: Test failure ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics Key: HADOOP-17213 URL: https://issues.apache.org/jira/browse/HADOOP-17213 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H The test ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics fails when the property fs.azure.test.appendblob.enabled is set to true -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17203) Test failures in ITestAzureBlobFileSystemCheckAccess in ABFS
[ https://issues.apache.org/jira/browse/HADOOP-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175451#comment-17175451 ] Bilahari T H commented on HADOOP-17203: --- Hi [~mehakmeetSingh] I assume that you are running the tests with OAuth. And the above exception means that the config fs.azure.account.oauth.provider.type (FS_AZURE_ACCOUNT_TOKEN_PROVIDER_TYPE_PROPERTY_NAME) is not set. https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java#L688 Please check and let me know. Also, Could you share me what is the authtype you are using and in case of OAuth what is the TokenProvider type you are using, so that I can try for repro. > Test failures in ITestAzureBlobFileSystemCheckAccess in ABFS > > > Key: HADOOP-17203 > URL: https://issues.apache.org/jira/browse/HADOOP-17203 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Mehakmeet Singh >Priority: Major > > ITestAzureBlobFileSystemCheckAccess is giving test failures while running > both in parallel as well as in stand-alone(in IDE). > Tested by: mvn -T 1C -Dparallel-tests=abfs clean verify > Region: East US -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172793#comment-17172793 ] Bilahari T H commented on HADOOP-17183: --- Commit: https://github.com/apache/hadoop/commit/a2610e21ed5289323d8a6f6359477a8ceb2db2eb > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17183: -- Resolution: Fixed Status: Resolved (was: Patch Available) > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17191) ABFS: Run tests with all the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17191: -- Summary: ABFS: Run tests with all the auth types (was: ABFS: Run tests with All the auth types) > ABFS: Run tests with all the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17191) ABFS: Run tests with All the auth types
[ https://issues.apache.org/jira/browse/HADOOP-17191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17191: - Assignee: Bilahari T H > ABFS: Run tests with All the auth types > --- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17191) ABFS: Run tests with All the auth types
Bilahari T H created HADOOP-17191: - Summary: ABFS: Run tests with All the auth types Key: HADOOP-17191 URL: https://issues.apache.org/jira/browse/HADOOP-17191 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Resolution: Fixed Status: Resolved (was: Patch Available) > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Resolution: Fixed Status: Resolved (was: Patch Available) > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > > The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] > has not yet been concluded. Untill then the logline could help debugging. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17137:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
> Fix For: 3.4.0
>
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
>
[jira] [Updated] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17183: -- Status: Patch Available (was: Open) > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17183) ABFS: Enable checkaccess API
[ https://issues.apache.org/jira/browse/HADOOP-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17183: - Assignee: Bilahari T H > ABFS: Enable checkaccess API > > > Key: HADOOP-17183 > URL: https://issues.apache.org/jira/browse/HADOOP-17183 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17183) ABFS: Enable checkaccess API
Bilahari T H created HADOOP-17183: - Summary: ABFS: Enable checkaccess API Key: HADOOP-17183 URL: https://issues.apache.org/jira/browse/HADOOP-17183 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Status: Patch Available (was: Open) > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > > The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] > has not yet been concluded. Untill then the logline could help debugging. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17166) ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size
[ https://issues.apache.org/jira/browse/HADOOP-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17166: -- Status: Patch Available (was: Open) > ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size > - > > Key: HADOOP-17166 > URL: https://issues.apache.org/jira/browse/HADOOP-17166 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17166) ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size
Bilahari T H created HADOOP-17166: - Summary: ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size Key: HADOOP-17166 URL: https://issues.apache.org/jira/browse/HADOOP-17166 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 3.3.0 Reporter: Bilahari T H Assignee: Bilahari T H Fix For: 3.4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17166) ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size
[ https://issues.apache.org/jira/browse/HADOOP-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17166: -- Component/s: fs/azure > ABFS: Add configs for maxConcurrentRequestCount and threadpool queue size > - > > Key: HADOOP-17166 > URL: https://issues.apache.org/jira/browse/HADOOP-17166 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Description: The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] has not yet been concluded. Untill then the logline could help debugging. > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > > The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] > has not yet been concluded. Untill then the logline could help debugging. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Component/s: fs/azure > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Affects Version/s: 3.3.0 > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Fix Version/s: 3.4.0 > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166560#comment-17166560 ] Bilahari T H commented on HADOOP-17160: --- [~ste...@apache.org] Sure. Thanks. > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17163: - Assignee: Bilahari T H > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17160: - Assignee: Steve Loughran (was: Bilahari T H) > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17160: - Assignee: Bilahari T H > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17163) ABFS: Add debug log for rename failures
Bilahari T H created HADOOP-17163: - Summary: ABFS: Add debug log for rename failures Key: HADOOP-17163 URL: https://issues.apache.org/jira/browse/HADOOP-17163 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
[ https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17150: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests > --- > > Key: HADOOP-17150 > URL: https://issues.apache.org/jira/browse/HADOOP-17150 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Major > Labels: abfsactive > > ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in > preview stage. The tests should not run until the API version reflects the > one in preview as when run against production clusters they will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16281) ABFS: Rename operation, GetFileStatus before rename operation and throw exception on the driver side
[ https://issues.apache.org/jira/browse/HADOOP-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-16281: - Assignee: Steve Loughran (was: Bilahari T H) > ABFS: Rename operation, GetFileStatus before rename operation and throw > exception on the driver side > - > > Key: HADOOP-16281 > URL: https://issues.apache.org/jira/browse/HADOOP-16281 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Steve Loughran >Priority: Major > Labels: abfsactive > > ABFS should add the rename with options: > [https://github.com/apache/hadoop/pull/743] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
[ https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-17150. --- Resolution: Works for Me Wrong observation > ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests > --- > > Key: HADOOP-17150 > URL: https://issues.apache.org/jira/browse/HADOOP-17150 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Labels: abfsactive > > ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in > preview stage. The tests should not run until the API version reflects the > one in preview as when run against production clusters they will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17160: -- Component/s: fs/azure > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17160: -- Affects Version/s: 3.3.0 > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17137:
--
Status: Patch Available (was: Open)
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
> Fix For: 3.4.0
>
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
>
[jira] [Updated] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17137:
--
Fix Version/s: 3.4.0
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
> Fix For: 3.4.0
>
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
>
[jira] [Assigned] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17149: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H reassigned HADOOP-17137:
-
Assignee: Bilahari T H (was: Sneha Vijayarajan)
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
>
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Status: Patch Available (was: Open) > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Affects Version/s: (was: 3.4.0) 3.3.1 > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
