[jira] [Commented] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166749#comment-17166749 ] Mingliang Liu commented on HADOOP-17159: This request makes sense to me. I have assigned this JIRA to you [~sandeep.guggilam] > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.3 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-17159: -- Assignee: Sandeep Guggilam > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.3 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Guggilam updated HADOOP-17159: -- Affects Version/s: 3.3.0 3.2.1 3.1.3 > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.3 >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Guggilam updated HADOOP-17159: -- Affects Version/s: 2.10.0 > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.10.0 >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166746#comment-17166746 ] Sandeep Guggilam commented on HADOOP-17159: --- Yes [~liuml07] this is applicable for 2.10.0 and other active versions of 3.x.y > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166738#comment-17166738 ] Mingliang Liu commented on HADOOP-17159: "Affects Version/s:" is 2.7.7 or newer? Since 2.7/2.8 has EoL, did you check if this is affecting any active version e.g. 2.10.0, [~sandeep.guggilam] Thanks, > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-17159: --- Fix Version/s: (was: 3.1.5) (was: 3.4.0) (was: 3.3.1) (was: 2.10.1) (was: 3.2.2) Target Version/s: 3.1.4, 3.2.2, 2.10.1, 3.3.1, 3.4.0 Moving values from the "Fixed version" (which should be updated when committing this change) to "Target version" filed. > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Guggilam updated HADOOP-17159: -- Component/s: security Fix Version/s: 3.1.5 3.4.0 3.3.1 2.10.1 3.2.2 > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Sandeep Guggilam >Priority: Major > Fix For: 3.2.2, 2.10.1, 3.3.1, 3.4.0, 3.1.5 > > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Description: The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] has not yet been concluded. Untill then the logline could help debugging. > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > > The JIRA [HADOOP-16281|https://issues.apache.org/jira/browse/HADOOP-16281] > has not yet been concluded. Untill then the logline could help debugging. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Component/s: fs/azure > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Affects Version/s: 3.3.0 > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17163: -- Fix Version/s: 3.4.0 > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166560#comment-17166560 ] Bilahari T H commented on HADOOP-17160: --- [~ste...@apache.org] Sure. Thanks. > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17163) ABFS: Add debug log for rename failures
[ https://issues.apache.org/jira/browse/HADOOP-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17163: - Assignee: Bilahari T H > ABFS: Add debug log for rename failures > --- > > Key: HADOOP-17163 > URL: https://issues.apache.org/jira/browse/HADOOP-17163 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17160: - Assignee: Steve Loughran (was: Bilahari T H) > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Steve Loughran >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17160: - Assignee: Bilahari T H > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17163) ABFS: Add debug log for rename failures
Bilahari T H created HADOOP-17163: - Summary: ABFS: Add debug log for rename failures Key: HADOOP-17163 URL: https://issues.apache.org/jira/browse/HADOOP-17163 Project: Hadoop Common Issue Type: Sub-task Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
[ https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17150: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests > --- > > Key: HADOOP-17150 > URL: https://issues.apache.org/jira/browse/HADOOP-17150 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Major > Labels: abfsactive > > ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in > preview stage. The tests should not run until the API version reflects the > one in preview as when run against production clusters they will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16281) ABFS: Rename operation, GetFileStatus before rename operation and throw exception on the driver side
[ https://issues.apache.org/jira/browse/HADOOP-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-16281: - Assignee: Steve Loughran (was: Bilahari T H) > ABFS: Rename operation, GetFileStatus before rename operation and throw > exception on the driver side > - > > Key: HADOOP-16281 > URL: https://issues.apache.org/jira/browse/HADOOP-16281 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Steve Loughran >Priority: Major > Labels: abfsactive > > ABFS should add the rename with options: > [https://github.com/apache/hadoop/pull/743] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.
[
https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reopened HADOOP-14124:
-
> S3AFileSystem silently deletes "fake" directories when writing a file.
> --
>
> Key: HADOOP-14124
> URL: https://issues.apache.org/jira/browse/HADOOP-14124
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
>Affects Versions: 2.6.0
>Reporter: Joel Baranick
>Priority: Minor
> Labels: filesystem, s3
>
> I realize that you guys probably have a good reason for {{S3AFileSystem}} to
> cleanup "fake" folders when a file is written to S3. That said, that fact
> that it silently does this feels like a separation of concerns issue. It
> also leads to weird behavior issues where calls to
> {{AmazonS3Client.getObjectMetadata}} for folders work before calling
> {{S3AFileSystem.create}} but not after. Also, there seems to be no mention
> in the javadoc that the {{deleteUnnecessaryFakeDirectories}} method is
> automatically invoked. Lastly, it seems like the goal of {{FileSystem}}
> should be to ensure that code built on top of it is portable to different
> implementations. This behavior is an example of a case where this can break
> down.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.
[
https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-14124:
Parent Issue: HADOOP-16829 (was: HADOOP-15620)
> S3AFileSystem silently deletes "fake" directories when writing a file.
> --
>
> Key: HADOOP-14124
> URL: https://issues.apache.org/jira/browse/HADOOP-14124
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
>Affects Versions: 2.6.0
>Reporter: Joel Baranick
>Priority: Minor
> Labels: filesystem, s3
>
> I realize that you guys probably have a good reason for {{S3AFileSystem}} to
> cleanup "fake" folders when a file is written to S3. That said, that fact
> that it silently does this feels like a separation of concerns issue. It
> also leads to weird behavior issues where calls to
> {{AmazonS3Client.getObjectMetadata}} for folders work before calling
> {{S3AFileSystem.create}} but not after. Also, there seems to be no mention
> in the javadoc that the {{deleteUnnecessaryFakeDirectories}} method is
> automatically invoked. Lastly, it seems like the goal of {{FileSystem}}
> should be to ensure that code built on top of it is portable to different
> implementations. This behavior is an example of a case where this can break
> down.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.
[
https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166540#comment-17166540
]
Steve Loughran commented on HADOOP-14124:
-
update: we're going to let people disable the act of deleting directories. This
will be completely incompatible with existing hadoop releases, but it will
avoid scale issues. HADOOP-13230
> S3AFileSystem silently deletes "fake" directories when writing a file.
> --
>
> Key: HADOOP-14124
> URL: https://issues.apache.org/jira/browse/HADOOP-14124
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
>Affects Versions: 2.6.0
>Reporter: Joel Baranick
>Priority: Minor
> Labels: filesystem, s3
>
> I realize that you guys probably have a good reason for {{S3AFileSystem}} to
> cleanup "fake" folders when a file is written to S3. That said, that fact
> that it silently does this feels like a separation of concerns issue. It
> also leads to weird behavior issues where calls to
> {{AmazonS3Client.getObjectMetadata}} for folders work before calling
> {{S3AFileSystem.create}} but not after. Also, there seems to be no mention
> in the javadoc that the {{deleteUnnecessaryFakeDirectories}} method is
> automatically invoked. Lastly, it seems like the goal of {{FileSystem}}
> should be to ensure that code built on top of it is portable to different
> implementations. This behavior is an example of a case where this can break
> down.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17150) ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests
[ https://issues.apache.org/jira/browse/HADOOP-17150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-17150. --- Resolution: Works for Me Wrong observation > ABFS: Test failure: Disable ITestAzureBlobFileSystemDelegationSAS tests > --- > > Key: HADOOP-17150 > URL: https://issues.apache.org/jira/browse/HADOOP-17150 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Labels: abfsactive > > ITestAzureBlobFileSystemDelegationSAS has tests for the SAS feature in > preview stage. The tests should not run until the API version reflects the > one in preview as when run against production clusters they will fail. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17160: -- Component/s: fs/azure > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17160: -- Affects Version/s: 3.3.0 > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17137:
--
Status: Patch Available (was: Open)
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
> Fix For: 3.4.0
>
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$
[jira] [Updated] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H updated HADOOP-17137:
--
Fix Version/s: 3.4.0
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
> Fix For: 3.4.0
>
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatemen
[jira] [Assigned] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17149: - Assignee: Bilahari T H (was: Sneha Vijayarajan) > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Bilahari T H >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17137) ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
[
https://issues.apache.org/jira/browse/HADOOP-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilahari T H reassigned HADOOP-17137:
-
Assignee: Bilahari T H (was: Sneha Vijayarajan)
> ABFS: Tests ITestAbfsNetworkStatistics need to be config setting agnostic
> -
>
> Key: HADOOP-17137
> URL: https://issues.apache.org/jira/browse/HADOOP-17137
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Sneha Vijayarajan
>Assignee: Bilahari T H
>Priority: Minor
> Labels: abfsactive
>
> Tess in ITestAbfsNetworkStatistics have asserts to a static number of
> network calls made from the start of fileystem instance creation. But this
> number of calls are dependent on the certain configs settings which allow
> creation of container or account is HNS enabled to avoid GetAcl call.
>
> The tests need to be modified to ensure that count asserts are made for the
> requests made by the tests alone.
>
> {code:java}
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[INFO]
> Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests
> run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.148 s <<<
> FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR]
> testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 4.148 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> get_responses expected:<8> but was:<7> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:207)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> [ERROR]
> testAbfsHttpSendStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)
> Time elapsed: 2.987 s <<< FAILURE!java.lang.AssertionError: Mismatch in
> connections_made expected:<6> but was:<5> at
> org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.failNotEquals(Assert.java:834) at
> org.junit.Assert.assertEquals(Assert.java:645) at
> org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445)
> at
> org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpSendStatistics(ITestAbfsNetworkStatistics.java:91)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStat
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Status: Patch Available (was: Open) > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Affects Version/s: (was: 3.4.0) 3.3.1 > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H updated HADOOP-17149: -- Affects Version/s: 3.4.0 > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.4.0 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17156) Clear abfs readahead requests on stream close
[ https://issues.apache.org/jira/browse/HADOOP-17156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17156: Summary: Clear abfs readahead requests on stream close (was: Clear readahead requests on stream close) > Clear abfs readahead requests on stream close > - > > Key: HADOOP-17156 > URL: https://issues.apache.org/jira/browse/HADOOP-17156 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Rajesh Balamohan >Priority: Minor > > It would be good to close/clear pending read ahead requests on stream close(). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17158) Intermittent test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters
[
https://issues.apache.org/jira/browse/HADOOP-17158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166392#comment-17166392
]
Steve Loughran commented on HADOOP-17158:
-
[~bilahari.th] is seeing this consistently, so he may be in a good position to
help debug the problem.
Bilihari -what's your test setup? Are you doing the parallel runs? And is your
test system close to the store network-wise?
Mehakmeet - does the test timeout more in parallel runs?
Imagine we had a gauge of queue size which could be accessed from the FS
instance...would that help in the tests?
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters
> --
>
> Key: HADOOP-17158
> URL: https://issues.apache.org/jira/browse/HADOOP-17158
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.3.0
>Reporter: Mehakmeet Singh
>Assignee: Mehakmeet Singh
>Priority: Major
>
> Intermittent test timeout for
> ITestAbfsInputStreamStatistics#testReadAheadCounters happening due to race
> conditions in readAhead threads.
> Test error:
> {code:java}
> [ERROR]
> testReadAheadCounters(org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics)
> Time elapsed: 30.723 s <<<
> ERROR!org.junit.runners.model.TestTimedOutException: test timed out after
> 3 milliseconds at java.lang.Thread.sleep(Native Method) at
> org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics.testReadAheadCounters(ITestAbfsInputStreamStatistics.java:346)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748) {code}
> Possible Reasoning:
> - ReadAhead queue doesn't get completed and hence the counter values are not
> satisfied in 30 seconds time for some systems.
> - The condition that readAheadBytesRead and remoteBytesRead counter values
> need to be greater than or equal to 4KB and 32KB respectively doesn't occur
> in some machines due to the fact that sometimes instead of reading for
> readAhead Buffer, remote reads are performed due to Threads still being in
> the readAhead queue to fill that buffer. Thus resulting in either of the 2
> counter values to be not satisfying the condition and getting in an infinite
> loop and hence timing out the test eventually.
> Possible Fixes:
> - Write better test(That would pass under all conditions).
> - Maybe UT instead of IT?
> Possible fix to better the test would be preferable and UT as the last resort.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17160. - Resolution: Duplicate HADOOP-17158 got there first. It's interesting you always see it; for Mehakmeet it is intermittent. Either way: needs fixing. And you will be able to verify the fix is permanent > ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always > -- > > Key: HADOOP-17160 > URL: https://issues.apache.org/jira/browse/HADOOP-17160 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Bilahari T H >Priority: Major > > The test ITestAbfsInputStreamStatistics#testReadAheadCounters timing out > always is timing out always -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17161) Make ipc.Client.stop() sleep configurable
[ https://issues.apache.org/jira/browse/HADOOP-17161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166387#comment-17166387 ] Steve Loughran commented on HADOOP-17161: - which workloads? > Make ipc.Client.stop() sleep configurable > - > > Key: HADOOP-17161 > URL: https://issues.apache.org/jira/browse/HADOOP-17161 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Priority: Major > > After identifying HADOOP-16126 might cause issues in few workloads, > ipc.Client.stop() sleep was identified to be configurable to better suit > multiple workloads -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17159) Ability for forceful relogin in UserGroupInformation class
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166386#comment-17166386 ] Steve Loughran commented on HADOOP-17159: - can you add component, version, target fix version. thanks. > Ability for forceful relogin in UserGroupInformation class > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Sandeep Guggilam >Priority: Major > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17162) Ozone /conf endpoint trigger kerberos replay error when SPNEGO is enabled
Xiaoyu Yao created HADOOP-17162:
---
Summary: Ozone /conf endpoint trigger kerberos replay error when
SPNEGO is enabled
Key: HADOOP-17162
URL: https://issues.apache.org/jira/browse/HADOOP-17162
Project: Hadoop Common
Issue Type: Improvement
Reporter: Nilotpal Nandi
Assignee: Xiaoyu Yao
{code}
curl -k --negotiate -X GET -u :
"https://quasar-jsajkc-8.quasar-jsajkc.root.hwx.site:9877/conf";
Error 403 GSSException: Failure unspecified at GSS-API level (Mechanism
level: Request is a replay (34))
HTTP ERROR 403 GSSException: Failure unspecified at GSS-API level
(Mechanism level: Request is a replay (34))
URI:/conf
STATUS:403
MESSAGE:GSSException: Failure unspecified at GSS-API level
(Mechanism level: Request is a replay (34))
SERVLET:conf
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
